<< by on June 14th, 2008
The foundation on an SEO business is search engine rankings. Rankings usually translate into traffic, hopefully if you choose your keywords you want optimized carefully that traffic is targeted and leads to a desired action happening on your site: conversions, KPIs, subscriptions, leads, etc. But the basis of it all is measuring progress of the rankings. To prove that optimization is working regular reporting is necessary.
We have always utilized WebPosition 4 as our rank checking software since our inception. For the most part it has been reliable for the years we have been using it. It has been known to miss the occasional very obvious keyword ranking, usually almost always that ranking is a Yahoo ranking. We export all of our ranking data to a smart reports database and can easily change that occasional missed ranking in that database.
But for about 8 or 9 days at the end of May and beginning of June, WebPosition wasn’t able to track ANY Yahoo rankings. Most of our reporting is measured at the end or beginning of the month and then immediately sent to our clients, so the timing couldn’t have been worse. All of our Yahoo data was wrong and the careful scheduling we had in place so our reporting resources wouldn’t be over taxed was at that point destroyed. It made for a very taxing week of manually checking Yahoo rankings (sometimes up to 100 listings) for numerous keywords.
The problem was universal for all WebPosition users. To my knowledge WebPosition is the number one rank checking software so it affected people worldwide. The issue apparently is not a new one, the same type of problems were noticed last year at almost the same time. Yahoo apparently has been changing out the URL in search results with their own string of gobble-dee-gook code and WebPosition can’t find the URL it’s supposed to be tracking.
Search Engines truly hate rank checking software. I personally heard Google “Search Evangelist” Adam Lasnik say at a conference that Google likened WebPosition Gold to the Devil. Google’s Web Master Guidelines even says:
Don’t use unauthorized computer programs to submit pages, check rankings, etc. Such programs consume computing resources and violate our Terms of Service. Google does not recommend the use of products such as WebPosition Gold™ that send automatic or programmatic queries to Google.
Google’s Terms of Service also state:
5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.
5.4 You agree that you will not engage in any activity that interferes with or disrupts the Services (or the servers and networks which are connected to the Services).
It is completely understandable that all search engines would not want programs like WebPosition constantly pinging its servers. These reports usually search the top 100 rankings for numerous keywords for the URL you are tracking and usually competitor URLs. The search engine data centers that house their servers are the major infrastructure investment for all search engines.
All that being said, WebPosition and its competitors are still out there and being used on a regular basis and are an absolutely necessary tool. SEO is a business, and reports are the work product to our clients that prove we are delivering results. We have to run the queries to see these results, whether through automated means or manually like we had to do last week with Yahoo, either way the searches will have to be done.
I really hope that WebPosition has truly fixed its Yahoo tracking. We have worked extensively to automate our reporting as much as possible, so that we can focus more on writing insightful executive summaries for our SEO reports, rather than doing exhaustive checks on rankings. Let’s just say after last week’s problems we have a renewed appreciation for rank checking software.