Bad Sitemap and Googles Indexing
This week I came across something quite worrying with Sitemaps and Google. This was an effect I have not seen before.
Two WordPress sites I look after, on Friday were deindexed by Google. Alarm bells started ringing loudly and I immediately started looking at algorithm changes and suspected that the supposedly ‘continuing Penguin roll out’ had caught these sites in some way. Outcome the tools to do a quick and dirty link analysis and comparison against the main competitors who were unaffected.
Yes I know I should have taken a step back, got a coffee and just thought about what had happened first and I probably would not have wasted a whole day over-complicating the problem. After hours of analysis and smoking spreadsheets I came up with… nothing! At least nothing that I could see which would have caused a complete deindexing.
So now the coffee finally came out and a pause for thought and back to basics. Testing the domain in Google with a ‘site:…’ search returns nothing, whereas twenty four hours previously the site was fully indexed. Fire up Google Webmaster Tools (GWT) and start looking around and there it was. Google could not read the sitemap. It was returning a 500 error. I quickly checked the other site and it was the same.
To sort this out I regenerated the site maps on the two WordPress sites and then in GWT I removed the existing sitemaps and uploaded the newly generated ones. Within 24 hours the sites were back in Google. Huge relief!
I have no idea what caused the sitemaps to become corrupt. The sites are not connected in anyway and I did not want to delay getting the issue solved by spending time looking for the cause. I have also never seen a corrupted site map causing a complete de-indexing of a site before.
When I get time I will take a further look at this on one of my test sites but for now it is something to take note of and check BEFORE throwing the armoury at a problem!
