Feeds

Google contradicts own counsel in face of antitrust probe

Admits existence of search algorithm whitelists

Security for virtualized datacentres

Google has admitted that it uses whitelists to manually override its search algorithms, more than a year after its European corporate counsel denied the existence of whitelists when defending the company against antitrust complaints in the EU.

According to Search Engine Land, Matt Cutts – the head of Google's webspam team – told a Silicon Valley conference that Google uses "exception lists" that prevent certain algorithms from affecting certain sites. There is no global whitelist, Cutts said, but for some algorithms, Google will make an exception for sites it believes have been wrongly demoted on its search pages.

Last February, when the world learned that EU antitrust regulators were investigating claims that Google unfairly manipulated its search results at the expense of competitors, Google European corporate counsel Julia Holtz told journalists in Brussels that the company doesn't "whitelist or blacklist anyone,” according to The Financial Times. And as late as December, Google spokesman Adam Kovacevich told The Register that the company did not use whitelists.

Whitelists play a big role in the Google antitrust complaint brought by Foundem, a UK-based vertical search engine. Foundem is one of three companies whose complaints are being formally investigated by the European Commission.

In 2006, Foundem essentially vanished from Google, after Mountain View introduced a new algorithm that demoted vertical search engines. But some vertical search engines were not affected by the new algorithms. They were apparently whitelisted – and even though Holtz and other Googlers denied the existence of whitelists, Foundem produced emails from Google's AdWords team indicating that whitelists are used on search-ad algorithms. The company was in discussions with the AdWords team because it had also been demoted in Google's search ad system.

Foundem fought its removal for more than three years, but it didn't return to Google's "organic" search until it went public with its story at the end of 2009. Foundem was granted an audience with Google's Search Quality Team, and the UK outfit has always said that it was then manually whitelisted.

Though Holtz also denied the existence of blacklists, Google clearly has blacklists in place as well – or at least something very similar to blacklists. Last month, Matt Cutts publically admitted that the company will manually demote sites. "We have confirmed that Google's webspam team is willing to take action manually – for example, if we get a spam report, off-topic porn, things like that," he said.

Google took one of these "manual actions" when it demoted JC Penney in its search ranking in mid-February.

On one level, it's hardly surprising that Google manually overrides its algorithms. As even Foundem says, manual interventions are an important part of any search engine. "Whether manual interventions are required to override something that the algorithms have got wrong or to override something that the algorithms have overlooked, such interventions are an essential part of the process," Foundem CEO Shivaun Raff has told us.

The trouble is that Google spent years refusing to acknowledge that these manual interventions exist – and, in some cases, outright denying them. Google spent nearly a decade telling the world that its search engine was completely objective, and it has only recently begun to freely admit that this is not the case, presumably as a result of the EU investigation.

Foundem's EU antitrust complaint takes aim at Google's Universal Search setup, which inserts links from other Google services – such as Google Maps and Google Product Search – into prominent positions on the company's main search-results pages. Foundem says the setup is transforming Google's search engine into an "immensely powerful marketing channel" for its own services. Like Foundem, Google Product Search is a price-comparison service.

But Foundem's complaint also takes issue with the way Google handles manual interventions, including how little information the company has provided about how they work. "The half of our Complaint detailing the anticompetitive power of Google's exclusionary penalties is as much about the systemic failures of Google's manual review process as it is about the legitimacy of the penalty algorithms themselves," Shivaun Raff has told us.

Last month, Cutts also revealed that if a site owner appeals its Google search ranking, making a "reconsideration request", Google won't examine the request unless the site has been manually demoted. Foundem made repeated reconsideration requests, but unbeknownst to Foundem, they were not considered because its site had been algorithmically demoted.

Raff encapsulates Foundem's EU complaint like this: "You have an overwhelmingly dominant search engine. If you add to that that search engine's ability to apply discriminatory penalties - they're discriminatory because some services are manually rendered immune through whitelists - and you add the ability of that search engine to preferentially insert its own services at or near the top of the search results, all of that adds up to an unparalleled and unassailable competitive advantage."

Google has apparently tried to settle the EC's investigation. But the EC has made it clear that the probe will go on.

According to Search Engine Land, Cutts did not use the term whitelist. He called them "exception lists." But the two terms mean the same thing. "It is clear to us that Google no longer likes to use the term 'whitelisting'," Foundem has told us in the past. "But this was their word, not ours. In the context of lifting our AdWord penalty, Google themselves called it “whitelisting', and in leaked quality reviewer guidelines from 2003, Google used the same word 'whitelisting” when talking about immunity from search penalties.

"So Google can’t claim they don’t understand what you mean by the term whitelisting. It also wouldn’t be ok for Google to claim, for example, that 'there is no such thing as whitelisting' simply because they now prefer to call it something else, such as manual removal of a demotion." ®

New hybrid storage solutions

More from The Register

next story
Phones 4u slips into administration after EE cuts ties with Brit mobe retailer
More than 5,500 jobs could be axed if rescue mission fails
Apple CEO Tim Cook: TV is TERRIBLE and stuck in the 1970s
The iKing thinks telly is far too fiddly and ugly – basically, iTunes
Apple Pay is a tidy payday for Apple with 0.15% cut, sources say
Cupertino slurps 15 cents from every $100 purchase
Phones 4u website DIES as wounded mobe retailer struggles to stay above water
Founder blames 'ruthless network partners' for implosion
Huawei ditches new Windows Phone mobe plans, blames poor sales
Giganto mobe firm slams door shut on Microsoft. OH DEAR
Forget silly privacy worries - help biometrics firms make MILLIONS
Beancounter reckons dabs-scanning tech is the next big moneypit
Microsoft's Office Delve wants work to be more like being on Facebook
Office Graph, social features for Office 365 going public
prev story

Whitepapers

Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
Top 5 reasons to deploy VMware with Tegile
Data demand and the rise of virtualization is challenging IT teams to deliver storage performance, scalability and capacity that can keep up, while maximizing efficiency.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.
Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.