Fix for Google AdSense not showing up on Category-Tag Pages - Wordpress

Google AdSense not showing up on Category-Tag Pages

Have you ever encountered this scenario ? In few pages of the website like Category / Tag pages, Google Adsense stopped showing up which were working before. It did happen in one of the websites that we support. Under the initial thought that it could be to do with the Google Adsense, tried google searches to drill down the problem. What not clear was that the google adunits appeared properly on the Home page or individual blog post. So as such, they were no problems in the Google adsense behavior. When we hit the pages with /category or /tag pages, adunits space was appearing, but the space was shown blank.

Solution for Google Adsense Not Showing Up:

Below robots.txt entry was causing the issue. Disallow:   */category/ Disallow:   */tag/ Updated the robots.txt to allow the Category and Tag pages to appear in Search results as below, Allow:  */category/ Allow:  */tag/

Just a single line incorrect entry in robots.txt caused Google Adsense not showing up on certain pages of the website. robots.txt is very sensitive and we need to be extra careful while updating it.

Root-Cause:

After a long search & hunt, the root cause for the problem was caught finally. Issue was due to robots.txt. It was not that easy to locate the cause as both are totally unrelated area. With not so clear understanding on the Website duplicate content issue, updated robots.txt to block the Category and Tag pages to be indexed in the Search engine and it has caused the ad units not showing up under Category and Tag pages of the website. Please refer to this link to know about Website duplicate content from Google Webmaster Blog. Also Read: Learn how to fix Google Adsense Yellow Box Issue

What is robots.txt ?

robots.txt is a text file that can be found under http://<yourWebsiteName>/robots.txt. This file is used by the Web Robots to gather the details of the website. It provides lot of instructions that would help the bots to perform the indexing of the site. In Wordpress Admin Dashboard, you can edit this file under Settings -> Reading Settings -> Additional robots.txt rules. You could also manage the robots.txt using Yoast Wordpress SEO plugin. We can determine the pages to appear in the search results and can also restrict pages using the Allow and Disallow commands. For instance, we don't generally submit the site file content or author page to the search engine bots. By adding below two lines under robots.txt, this can be achieved. Disallow:  */wp-content/ Disallow:  */author/