Web metrics can mislead you and how to avoid it

Recently, Wall Street ignored comScore’s disclaimer on the limitations of its metrics data, causing people to expect lower Google’s revenue. It turned out that Google had stronger earnings for the first quarter of 2008. Here is the article from NY Times “Web Metrics and Grains of Salt”. Web metrics data could be much helpful if more questions.

The learning from this incident from web analytics strand point, never settle with one main data point and question beyond the given metrics. In this incident, it would be comScore reports showing a slowdown in paid clicks on Google’s sites. If you’re a web analyst, it is important to ask yourself beyond what is given.

I highly doubt those Wall Street analysts only gauged on that one specific data, but when it comes down to analyzing web metrics, there would be far more questions arising out of these data. (I’ll limit it to few key questions since the list can go on forever)

• In this specific scenario, is comScore limiting “paid clicks” to just Google’s SERPs (search engine result pages)? Google’s content targeting ads are placed on so many other websites, and that should automatically tell us that Google’s revenue aren’t just limited to their domain.
• comScore’s projection was based on United States market, how much of that would be accounted in Google’s revenue?
• Do increase/decrease in clicks automatically associate to higher or lower revenue? I don’t think so, if Google says quality of clicks will be enhanced, there is a possibility that each clicks could weigh more in terms of money. In another word, enhancing for qualified click-throughs mean higher conversion, and potentially allowing Google to generate more money (in areas of referrals, high value keywords, competitive keywords, etc.)
• Traffic from other countries beside United States probably offset slower growth, but if you’re not sure what Google’s revenue drivers are then, how can you assure lower revenue from such company?

Although I’m talking about Google in general in this case, the kind of thinking process to ask beyond a data point to generate next questions is an important process in web analytics. So this article by NY Times, states at the beginning “Perhaps Internet metrics should come with a warning label: “handle with care.” is actually not a bad idea. Web analytics are great to analyze “what” is going on with the site, but it always has been a challenge to compare it across different software platforms, different medium, etc.

So when using web metrics to associate data across different platforms or environment, it would probably wiser to count on web analytics specialist rather than wall street analyst, because web metrics data needs to be handled with care.