Feeds

Google contradicts own counsel in face of antitrust probe

Admits existence of search algorithm whitelists

Designing a Defense for Mobile Applications

Google has admitted that it uses whitelists to manually override its search algorithms, more than a year after its European corporate counsel denied the existence of whitelists when defending the company against antitrust complaints in the EU.

According to Search Engine Land, Matt Cutts – the head of Google's webspam team – told a Silicon Valley conference that Google uses "exception lists" that prevent certain algorithms from affecting certain sites. There is no global whitelist, Cutts said, but for some algorithms, Google will make an exception for sites it believes have been wrongly demoted on its search pages.

Last February, when the world learned that EU antitrust regulators were investigating claims that Google unfairly manipulated its search results at the expense of competitors, Google European corporate counsel Julia Holtz told journalists in Brussels that the company doesn't "whitelist or blacklist anyone,” according to The Financial Times. And as late as December, Google spokesman Adam Kovacevich told The Register that the company did not use whitelists.

Whitelists play a big role in the Google antitrust complaint brought by Foundem, a UK-based vertical search engine. Foundem is one of three companies whose complaints are being formally investigated by the European Commission.

In 2006, Foundem essentially vanished from Google, after Mountain View introduced a new algorithm that demoted vertical search engines. But some vertical search engines were not affected by the new algorithms. They were apparently whitelisted – and even though Holtz and other Googlers denied the existence of whitelists, Foundem produced emails from Google's AdWords team indicating that whitelists are used on search-ad algorithms. The company was in discussions with the AdWords team because it had also been demoted in Google's search ad system.

Foundem fought its removal for more than three years, but it didn't return to Google's "organic" search until it went public with its story at the end of 2009. Foundem was granted an audience with Google's Search Quality Team, and the UK outfit has always said that it was then manually whitelisted.

Though Holtz also denied the existence of blacklists, Google clearly has blacklists in place as well – or at least something very similar to blacklists. Last month, Matt Cutts publically admitted that the company will manually demote sites. "We have confirmed that Google's webspam team is willing to take action manually – for example, if we get a spam report, off-topic porn, things like that," he said.

Google took one of these "manual actions" when it demoted JC Penney in its search ranking in mid-February.

On one level, it's hardly surprising that Google manually overrides its algorithms. As even Foundem says, manual interventions are an important part of any search engine. "Whether manual interventions are required to override something that the algorithms have got wrong or to override something that the algorithms have overlooked, such interventions are an essential part of the process," Foundem CEO Shivaun Raff has told us.

The trouble is that Google spent years refusing to acknowledge that these manual interventions exist – and, in some cases, outright denying them. Google spent nearly a decade telling the world that its search engine was completely objective, and it has only recently begun to freely admit that this is not the case, presumably as a result of the EU investigation.

Foundem's EU antitrust complaint takes aim at Google's Universal Search setup, which inserts links from other Google services – such as Google Maps and Google Product Search – into prominent positions on the company's main search-results pages. Foundem says the setup is transforming Google's search engine into an "immensely powerful marketing channel" for its own services. Like Foundem, Google Product Search is a price-comparison service.

But Foundem's complaint also takes issue with the way Google handles manual interventions, including how little information the company has provided about how they work. "The half of our Complaint detailing the anticompetitive power of Google's exclusionary penalties is as much about the systemic failures of Google's manual review process as it is about the legitimacy of the penalty algorithms themselves," Shivaun Raff has told us.

Last month, Cutts also revealed that if a site owner appeals its Google search ranking, making a "reconsideration request", Google won't examine the request unless the site has been manually demoted. Foundem made repeated reconsideration requests, but unbeknownst to Foundem, they were not considered because its site had been algorithmically demoted.

Raff encapsulates Foundem's EU complaint like this: "You have an overwhelmingly dominant search engine. If you add to that that search engine's ability to apply discriminatory penalties - they're discriminatory because some services are manually rendered immune through whitelists - and you add the ability of that search engine to preferentially insert its own services at or near the top of the search results, all of that adds up to an unparalleled and unassailable competitive advantage."

Google has apparently tried to settle the EC's investigation. But the EC has made it clear that the probe will go on.

According to Search Engine Land, Cutts did not use the term whitelist. He called them "exception lists." But the two terms mean the same thing. "It is clear to us that Google no longer likes to use the term 'whitelisting'," Foundem has told us in the past. "But this was their word, not ours. In the context of lifting our AdWord penalty, Google themselves called it “whitelisting', and in leaked quality reviewer guidelines from 2003, Google used the same word 'whitelisting” when talking about immunity from search penalties.

"So Google can’t claim they don’t understand what you mean by the term whitelisting. It also wouldn’t be ok for Google to claim, for example, that 'there is no such thing as whitelisting' simply because they now prefer to call it something else, such as manual removal of a demotion." ®

Using blade systems to cut costs and sharpen efficiencies

More from The Register

next story
BBC goes offline in MASSIVE COCKUP: Stephen Fry partly muzzled
Auntie tight-lipped as major outage rolls on
iPad? More like iFAD: We reveal why Apple fell into IBM's arms
But never fear fanbois, you're still lapping up iPhones, Macs
Nadella: Apps must run on ALL WINDOWS – PCs, slabs and mobes
Phone egg, meet desktop chicken - your mother
White? Male? You work in tech? Let us guess ... Twitter? We KNEW it!
Grim diversity numbers dumped alongside Facebook earnings
Microsoft: We're making ONE TRUE WINDOWS to rule us all
Enterprise, Windows still power firm's shaky money-maker
HP, Microsoft prove it again: Big Business doesn't create jobs
SMEs get lip service - what they need is dinner at the Club
ITC: Seagate and LSI can infringe Realtek patents because Realtek isn't in the US
Land of the (get off scot) free, when it's a foreign owner
Dude, you're getting a Dell – with BITCOIN: IT giant slurps cryptocash
1. Buy PC with Bitcoin. 2. Mine more coins. 3. Goto step 1
There's NOTHING on TV in Europe – American video DOMINATES
Even France's mega subsidies don't stop US content onslaught
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.