So, some of the biggest internet publishers have gathered together to bully Comscore, the audience measurement service. Sites such as MSNBC.com believe the stats company, which tells advertisers which websites are most popular among the demographic they're targeting, undercounts their traffic by 30%. But it's smaller internet properties with technologically sophisticated audiences — Silicon Valley startups, especially — that suffer the most from Comscore's methodology. And it matters, because internet advertisers rely on Comscore, way more than free traffic charts like Alexa's, in allocating their budgets.

I confess I've never had much sympathy for Silicon Valley complaints about old-school measurement companies. These very same entrepreneurs, hypocrites all, are happy to boast about their soaring Alexa graphs, overlooking the gigantic problems with the free service's numbers. (About which, more, in another post.) Too often, entrepreneurs criticize Comscore simply because they can't accept the fundamental irrelevance of their site, that it has neither revenue, nor traffic meaningful enough to register.

If Comscore merely undercounted all internet sites, it wouldn't matter much, because big advertisers don't use the service to determine how much money to shift from television to the internet, but to divide up their spending among individual web properties. And it is not enough for startups to complain that many of their readers view sites on the Firefox browser. Tech news linkdump, Digg, has a large presence among Firefox-using geeks in San Francisco, but one can't extrapolate from that to come up with a figure for a nationwide audience. What matters is not that Comscore sample be representative of the Digg audience, but that it be representative of US internet users.

All that said, Comscore's methodology has some serious flaws, and they're crimping the advertising revenues of internet startups. Here are five issues.

1. Problems with recruiting over the web. Comscore audience panel consists of users with a special toolbar, which tracks their internet surfing patterns. It is recruited largely over the internet. Comscore uses remnant ad inventory and surveys to persuade users to join the panel. There is a bias towards older people and those with the patience to follow instructions.

2. Problems with recruiting by phone. According to a Marketwatch article about the Comscore controversy, the stats company is supplementing this online method of recruitment by calling individuals at random from the phone directory. If true, this would not solve Comscore's problem. Younger people and college students, whom Comscore undercounts, tend to use cellphones.

3. Erratic results. Although Comscore's US panel is large, with 120,000 members and growing, it still tends to produce erratic results for any site under 1m unique visitors a month. At Gawker Media, we've seen individual sites triple their Comscore audience numbers, and then collapse, without any significant change in the underlying popularity of the site as measured by internal statistics.

4. Weighting. Comscore weights its panel to compensate for its lack of users aged 18 to 24, I'm told, and presumably, for other demographic imbalances in the sample. But the measurement company doesn't say whether it also adjusts for other criteria, such as the fact that it might pay too much attention to users of Internet Explorer.

5. Firefox and Safari. We knew that Comscore's toolbar did not work on Safari, and therefore did not count the typically sophisticated users of the Apple Mac internet browser. But I hadn't realized how much Comscore undercounted usage of Firefox. The Mozilla browser is used by about 15% of the US internet population; Firefox devotees account for only 7% of Comscore's panel, according to a publishing exec who has spoken recently with the firm.

So, what does that mean? Well, a site viewed, hypothetically, only by Safari users, would show as having no audience at all. A site with only Firefox fans would register with about half the audience it should. A normal tech site? The audience for Lifehacker, Gawker Media's software and productivity title, is about 60% Firefox, 5% Safari, and 30% other. Assuming Comscore doesn't take account of its undersampling of geek browsers, that factor alone would cut Lifehacker's measured audience by 33%. And that's compounded by the more general problems with Comscore that MSNBC.com and other big internet publishers are complaining about.