Original URL: http://www.theregister.co.uk/2008/05/12/inside_comscore/
How ComScore can track your mouse clicks
Explores stream of unconsciousness
There's one question no one thought to ask: How did comScore know that all those paid clicks had disappeared from the world's largest search engine?
In late February, the well-known web research outfit unveiled a particularly juicy report claiming that Google's paid-click rate was on the wane - at least in the States. Judging from Google's eventual response, the report wasn't too far from the mark. The search giant acknowledges that paid clicks are down, insisting it's on a mission to improve "the relevance" of its online ads.
Over the next several weeks, tech-happy pundits spent countless columns debating the effect of this click dip on Google finances - which turned out to be no effect at all - but nobody stopped to wonder where comScore's data came from. Or why it hit so close to Google's home.
Heck, the Reston, Virginia-based comScore has churned out such numbers for years, and for years, the press has largely overlooked its behind-the-scenes practices. Which is odd. This is a publicly-traded company with a $600m market cap - an outfit that just posted a record $26.4m in first quarter revenues. Plus, it's a story worth telling. Those behind-the-scenes practices are more interesting than you might imagine.
comScore tracks the online movements of more than two million people in 170 countries, including a million in the US. Thanks to its very own tracking software - sitting on end-user PCs from Japan to the UK - it sees not just Google ad clicks, but every other breed of internet usage, from audio and video streaming to secure web sessions. That's right, secure web sessions. If a user visits an online banking site or a health records site, comScore sees what the user sees - and it sees what the user types.
And it often knows the user's name. Even if multiple people use a machine - Bob, his wife Jane, and their daughter Sue, for instance - comScore can tell one from the other. You see, the company also tracks mouse movements and keystrokes, identifying the telltale habits of each user. Nine times out of ten, it doesn't just record that Christmas Amazon purchase. It records who made it.
There are documented cases where third-party operations have installed comScore's software without consent. And even when users do give the OK, they may not realize what they're consenting to - as is so often the case with web-based user agreements. We can't help but wonder, for instance, if most users realize that comScore knows who they are even they decline to say who they are.
As Chasin explains, the company reserves the right to match a user's internet activity with additional personal data from credit reporting agencies like Experian and Equifax. If the user keys his address into a web form, for instance, the company may take that address to an outside firm, retrieving the user's race, his gender, the size of his household, and more.
There are two primary ways that comScore distributes its software. In some cases, it distributes on its own, through a site called PermissionResearch. In others, it pays third-parties to install the software, tagging it with a separate brand name: Relevant Knowledge.
At PermissionResearch, the game is fairly straightforward. Additional software and services are offered alongside comScore's net monitor, including an online backup tool, a privacy guard, and, believe it or not, screen savers. "PermissionResearch relies on its members to gain valuable insight into Internet trends and behavior," the site home page reads. "In exchange for having their Internet browsing and purchasing activity monitored, members have access to free software downloads and a variety of other benefits."
The PermissionResearch pitch
The question is what happens with third-party distributors - who comScore does not identify. Officially, these third parties distribute much like comScore itself: They piggyback the company's net monitor on other "free" software packages - and they clearly explain the monitoring bit. "It's not like you download an mp3 player and they sneak this comScore thing past you," Chasin said. "They explain, very specifically, that in order to get the mp3 player, you have to join our panel.
But this doesn't always happen. Harvard Business School assistant professor Ben Edelman offers bona fide video proof of a well-known banner ad farm - ExitExchange - installing comScore's software via drive-by download. This was captured almost a year ago, but he witnessed a similar incident as recently as last month. And recent tests from both Edelman and McAfee show that a site called TopDesktop continues to bundle comScore alongside other software without proper notice.
"comScore distributors are supposed to get consent, but in practice, they don't always do that," Edelman told us. "Sometimes, they install the software in tricky and underhanded ways."
comScore vows to jettison partners who distribute without consent. And Edelman has witnessed the company remotely removing software installed via drive-by download. But drive-bys and non-consensual bundles aren't the only problems. There's also that gray area between consensual and non-consensual.
Sears - the venerable American retailer - has distributed comScore software, and as we reported in January, the company wasn't exactly upfront about it.
"[The] extreme level of user tracking is done with little and inconspicuous notice about the true nature of the software," wrote Benjamin Googins, a researcher in the anti-spyware unit at Computer Associates, in his critque of the retailer's My SHC Community service. "In fact, while registering to join the 'community,' very little mention is made of software or tracking." At the time, Sears buried such mention on page 10 of a 54-page user agreement.
Judging from the current user agreement on the My SHC Community service, it appears that Sears no longer distributes comScore software. A company spokesman told us: "We don't talk about our business partners...I don't know that we ever distributed comScore software." But he acknowledged that Sears turned off part of the SHC service earlier this year.
The easiest way
The policy clearly explains that comScore's software tracks all internet usage: "[Our application] monitors all of the Internet behavior that occurs on the computer on which you install the application, including both your normal web browsing and the activity that you undertake during secure sessions, such as filling a shopping basket, completing an application form or checking your online accounts, which may include personal financial or health information."
But it doesn't tell the whole story when it comes to the collection of credit report data - or biometric monitoring. The policy says things like "We may also combine the information that you provide us...with information obtained from other sources (such as consumer preference reporting companies, credit reporting agencies, and prescription benefits managers)" and "This application also tracks the pace and style with which you enter information online." But it doesn't put all the pieces together.
In certain cases, comScore's software asks you for demographic info: your age, your race, who you share a house with, etc. But if you don't fill in all the blanks, comScore nabs your address on its own and uses that to grab more information from another unnamed third party - a firm that consolidates data from Experian, Equifax, and maybe others.
"Here's the easiest way," Chasin says. "If someone on our panel goes to a web form and puts their address into the form, we can then identify the household."
According to Ari Schwartz, the vice president and chief operating officer of the The Center for Democracy and Technology, a well-known privacy watchdog, this sort of "email append" is common.
"Experian, for instance, offers an email append service, so if a company provides your email address, Experian can provide your household information," Schwartz told us. "Companies will say 'Sign up for our mailing list,' and they can go out and get all of your other information."
Experian owns HitWise, a comScore competitor, but it seems that HitWise doesn't collect quite as much personal information that comScore does. HitWise licenses most of its data from ISPs, and the company says this ISP data is kept completely anonymous. But more on that later. We're looking at comScore. The difference with comScore is that it can match Experian data with your internet habits.
Mouse movements and keystrokes
Once comScore knows who's in your house, its software can track the behavior of each individual user. Scrutinizing mouse movements and key stokes, it can easily distinguish between you and your wife and your daughter. "We can attribute over 90 per cent of the behavior to a specific person - though there is some ramp-up time, from a day to a couple of weeks," Chasin said.
Again, this only happens in certain cases. The company tracks individual users for its main "Media Metrix" research service, but not with other services, including qSearch, specifically for search data, and comScore Marketer, for marketing info. These track at the machine level. Each month, 150,000 American users and 45,000 UK users play into the company's Media Metrix data.
But even with Media Metrix, Josh Chasin says, comScore does not share personal data with outside companies. "We report out on cells in aggregate: 'Women 18-34,' for example. Or by geographic location: 'Visitors to Yahoo! from New York.' Person-level data...is used to place behaviors into these buckets. We have to know the age, gender, and zip of panelists to make these classifications (as does Arbitron in radio, Nielsen TV ratings, etc.)"
Well, he does acknowledge that the company may share your name or address with an outside firm in order to nab that Experian and Equifax data. "Again, this is like the Nielsen TV ratings example. They have to know your address to come to your house and put the meter on your set, but there is nothing in their business model that uses your address outside of that."
And there's always the danger that comScore will be hacked - or served with legal papers. "If comScore gets a subpoena," says Edelman, "comScore may be forced to turn over detailed data about you - like all the web sites you visit. And subpoenas happen all the time."
Indeed they do. But Chasin points out that anyone who's running comScore software is free to remove it. "It's important to note that we show up in your 'Add/Remove Programs' and provide an entry under 'Programs' in the Windows Start Menu, so there's nothing covert here," Chasin says. "We are not spyware." ®