Adsense Crawl Errors
Adsense Crawl Errors? So a lot of people are getting Double Click Publishers Google Adsense Crawl Errors. This article will help you with these issues. Many of my clients are having the same issues so I am writing an article on this subject to help with some headaches.
Step #1
When you log into your google adsense account you will scroll to this section on the first page under where is says "Scoreboard" . here you will find a link to fix your Adsense Crawl Errors. Click on that link
Step #2
After clicking on that link you will come to a page that looks like this. you probably freak out thinking that it hurts your SEO or something but the fact of the matter is that it does not. Here is a quote from google.
Q: Most of my 404s are for bizarro URLs that never existed on my site. What’s up with that? Where did they come from?
A: If Google finds a link somewhere on the web that points to a URL on your domain, it may try to crawl that link, whether any content actually exists there or not; and when it does, your server should return a 404 if there’s nothing there to find. These links could be caused by someone making a typo when linking to you, some type of misconfiguration (if the links are automatically generated, e.g. by a CMS), or by Google’s increased efforts to recognize and crawl links embedded in JavaScript or other embedded content; or they may be part of a quick check from our side to see how your server handles unknown URLs, to name just a few. If you see 404s reported in Webmaster Tools for URLs that don’t exist on your site, you can safely ignore them. We don’t know which URLs are important to you vs. which are supposed to 404, so we show you all the 404s we found on your site and let you decide which, if any, require your attention.
Source to article
Step #3
One of the things that you want to be sure to do so that Google Adsense can crawl you site to display relevant ads to the users is include this code in your robot.txt file.
User-agent: Mediapartners-Google
Disallow:
You should upload this file to your site under http://yoursite.com/robot.txt to avoid Adsense Crawl Errors.
Conclusion
Don't drive yourself crazy with the 404 crawl errors or Adsense Crawl Errors. Sometimes you can do something about it and sometimes you cannot. If you are displaying ads from other publishers and they are generating 404 errors in your website the only two things that you can do is contact that publisher and ask them to correct it or remove their ads from your site.
Remember also that Google adsense only allows 3 ads to be displayed on a webpage so if you are displaying more than 3 ads on a page it will generate crawl errors. Especially from doubleclick publishers.
How to Create a Google Adsense Account Video
Step #2
After clicking on that link you will come to a page that looks like this. you probably freak out thinking that it hurts your SEO or something but the fact of the matter is that it does not. Here is a quote from google.
Q: Most of my 404s are for bizarro URLs that never existed on my site. What’s up with that? Where did they come from?
A: If Google finds a link somewhere on the web that points to a URL on your domain, it may try to crawl that link, whether any content actually exists there or not; and when it does, your server should return a 404 if there’s nothing there to find. These links could be caused by someone making a typo when linking to you, some type of misconfiguration (if the links are automatically generated, e.g. by a CMS), or by Google’s increased efforts to recognize and crawl links embedded in JavaScript or other embedded content; or they may be part of a quick check from our side to see how your server handles unknown URLs, to name just a few. If you see 404s reported in Webmaster Tools for URLs that don’t exist on your site, you can safely ignore them. We don’t know which URLs are important to you vs. which are supposed to 404, so we show you all the 404s we found on your site and let you decide which, if any, require your attention. Source to article
Step #3
One of the things that you want to be sure to do so that Google Adsense can crawl you site to display relevant ads to the users is include this code in your robot.txt file.