Red Sheriff has chinks in its armour

By CHRIS BARTON

At last! Independent statistics to tell us how many people visit websites - something New Zealand's net industry has long been waiting for.

The numbers look very good - indicating that website audiences are much bigger than expected. But given a history of unreliable and exaggerated web numbers, are the stats supplied by Red Sheriff accurate?

One of the great myths of the dotcom era was that the internet, unlike any other medium, had the means to count everything accurately - how many people visited a particular site, how long they spent there, how often they "clicked through", where they had surfed in from and where they would surf off to. In theory, all this and more was possible from site server logs of data traffic.

But as is often the case with technology, the gap between hype and reality was a yawning chasm. In the late 90s, website owners took to talking about the number of "hits" their sites took. The big numbers seemed impressive - until people figured out that a hit was a record of any downloaded element on a page.

That meant pages with lots of elements - logos, graphics, and photos - could count multiple hits every time a single page was downloaded. It made a mockery of what was happening, but helped fuel dotcom hysteria where everything operated in multiples of this and that.

The word should have been banned long ago, but today many websites still report this meaningless statistic.

Later, hits gave way to "page impressions", the number of times a page is viewed, and "click throughs" - the number of times the banner ad on a page is accessed. Sites also started to determine the number of "unique visitors" - to get around over-counting those who revisited a site several times a day.

But by this time the damage was done. Website statistics were silly numbers that no one believed. Part of the problem was the lack of standards because of all the different software and methodologies used. It also didn't help that most web statistics were never independently audited. When they were, there were always big arguments about margins of error because of things like caching and dynamic IP address assignment.

One counting technology that survived the nonsense was "the persistent cookie" - a small piece of code placed on your computer's hard drive when you visited a site. There it stayed - unless you knew how to "crunch" it - uniquely identifying your computer and its surfing habits to the traffic counting software.

Red Sheriff employs persistent cookies across the websites it measures along with telephone-based interviews of a representative sample of netizens. So far, only 100 sites are being measured in New Zealand - those which have agreed to pay Red Sheriff a monthly fee. But all participating sites have accepted the Red Sheriff way, which means there is now a standard of sorts.

It also gives competition for Hitwise and Nielsen//NetRatings. The latter uses a sample panel methodology, but has been criticised because the sample users who installed special software on their PCs covered only the home market.

Hitwise gets its information from internet providers and other servers but has been criticised because it doesn't get data from key providers such as Xtra. It also doesn't provide numbers - preferring to rank sites and assign a percentage market share.

So what's wrong with Red Sheriff? Its key weakness is the reliance on cookies, which leaves room for some margin of error.

The first error comes from people like me who routinely crunch their cookies. Why? Because I have an irrational paranoia about people putting a code on my computer when I don't really know what it is doing.

Fortunately, paranoids are in the minority - only about 3 per cent, according to Red Sheriff - so we don't mess up the numbers too much.

But every time we crunch our cookies we automatically get another one when we revisit a Red Sheriff monitored site - so we get over- counted as a unique visitor.

Another problem that inflates Red Sheriff stats is that cookies are PC-dependent - meaning I get counted twice when I look at the Herald site from my work computer and from home.

When the monthly figures come out that will help to explain why xtra.co.nz, for example, will get many more unique visitors than its 400,000 or so subscribers.

But cookies also undercount - not picking up, for example, that at least three people use my home computer.

Other anomalies include xtramsn.co.nz which gets the bulk of its numbers from Xtra's subscriber base, most of whom will have the site as their default home page. But the site also gets a lot of hijacked unique visitors - from people using Microsoft Hotmail accounts who are automatically sent to xtramsn.co.nz when they log out of that service.

So how big is Red Sheriff's margin of error? No one is really saying. But with web numbers, bigger is not always what it seems.

* Email Chris Barton

Red Sheriff

Hitwise

Nielsen//NetRatings