Feeds

When algorithms attack, does Google hear you scream?

Inside Google's search penalties gulag

  • alert
  • submit to reddit

Application security programs and practises

First on Yahoo!, 119th on Google

That simple search, incidentally, might well be what the typical user would hack in if they were looking for cheap air tickets, but for that very reason it doesn't necessarily prove anything in the wonderful world of Search Engine Optimisation - the keywords are extremely competitive.

More specific queries, however, give a cleaner picture of the impact of the penalty. This search for a price comparison of a motorcycle helmet, for example, places Foundem first on Yahoo, sixth on Bing, and one hundred and nineteenth on Google at time of writing.

The Register's Department of Cracked Records feels it should point out here that Google also declined to comment on why apparently similar sites remained unpenalised when others were penalised, and whether the disparity in results between Google and Yahoo! and Bing meant that Google was in some way 'better', or that Google was merely exercising an editorial judgment.

The waters are muddied further by other instances of the Google Big Red Switch in operation. On the same day that that Foundem disappeared from Google search, many other sites, including amazon.com, also vanished. Google later said this was due to a "technical issue", and The Register understands that this was indeed unrelated to the algorithm that hit Foundem. But if you can't get any sensible information out of Google (and just saying "technical issue" hardly counts as sensible information), then you've no idea whether you need to fix something, or whether it was just somebody at Google with a fat finger.

Other sites, however, have certainly been subjected to Google's quality penalties for lengthy periods. Sourcetool.com, for example, was hit with an AdWords penalty in 2006, and filed suit against Google earlier this year.

And there are aspects of Sourcetool's case that are strikingly familiar. Says New York Times writer Joe Nocera: "When I pressed Mr. Fox [Google ad quality product manager] about Sourcetool, he refused to tell me why the algorithm had problems with the site. When I asked him why the business.com site was in the algorithm's good graces but Sourcetool's wasn't, he wouldn't tell me that, either. All I got were platitudes about the user experience.

"It wasn’t long before I was almost as exasperated as Mr. Savage [Sourcetool boss Dan Savage]. How can you adapt your business model to Google's specs if Google won’t tell you what the specs are?

"Google also told me that it never made judgments of what was 'good' and 'bad' because it was all in the hands of the algorithm. But that turns out not to be completely true. Mr. Savage shared with me an e-mail message from a Google account executive to someone at another company who had run into the same kind of landing page problem as Sourcetool.

"The Google account executive wrote back to say that she had looked at the site and found that 'there seems to be a wealth of valuable information on the site.' Consequently, her team overruled the algorithm."

A number of high profile sites, including moneysupermarket.com, confused.com and gocompare.com have also been penalised, suffering disappearances from a couple of days to several weeks, since 2006. Sites are often penalised for perfectly valid reasons - for example, for using paid links and reviews to boost their Google rankings for competitive keywords - and usually when a site's ranking goes into freefall, its operator is widely assumed to be up to no good.

Google won't talk about it, the companies penalised won't talk about it, and it's very seldom that - as in the case of Foundem - the victim goes public and challenges Google to explain and justify its actions. And this tends to reinforce the outside world's view of Google's opinion as perfect - if Google's actions aren't publicly challenged, then Google's actions must have been right, right?

Follow the money

Several Media Corp sites, including gambling.com and www.creditcardexpert.co.uk/, had penalties imposed in October 2007, but were reprieved just last month, two years later. Announcing the lifting of the penalties, Media Corp CEO Justin Drummond commented: "The removal of the Google penalty is a hugely significant event for Media Corp and should sharply increase the revenues derived from these two websites." The effect of the penalty, the company said, had "significantly reduced inbound visitor traffic from Google which, in turn, adversely impacted revenues on these websites".

The Media Corp episode raises some interesting questions about the lack of transparency in Google's processes. Media Corp bought gambling.com for £11m in 2005, and Google's penalty will clearly have damaged the value of the site. In the wake of the announcement, however, Media Corp shares rose 25 per cent.

If it is the case that a level playing field algorithm imposes the penalties, then the scope for insider trading is limited. But as humans are involved in the discussion and implementation of the whitelisting process, one day there will clearly be a need for Google to demonstrate to financial authorities that its systems cannot be used to manipulate stock prices.

We queried this with Google, but were merely referred again to the published "guidelines" for search and for AdWords.

Media Corp did not respond to The Register's questions regarding the steps it took to have the penalties removed. An Evening Standard report, however, had Drummond blaming either a malicious rival or a "bug in the technology" (which it clearly was not), and said that the problem had been sorted out with the help of a consultancy firm appointed a few weeks previously. It's not clear what it was that this firm did to help turn the situation around, but it could not have been SEO, and The Register understands that the firm was not an SEO specialist.

Alongside the long exiles there are numerous cases of swift reinstatements. These, Raff argues, indicate that there is a fast whitelisting process available for high profile sites. As we've seen, there clearly is a manual whitelisting process for search, and - for price comparison sites at least - this appears to involve a certain amount of human value judgement as to what does and does not constitute 'added value'.

One can also reasonably presume that the response from Google support will be markedly different for a huge beast like Amazon and a small startup - this does not require the existence of some controlling intelligence intent on favouring the big boys, it's just the way of the world. The playing field will never be entirely even, but it might at least be a little more transparent.

The Power of One eBook: Top reasons to choose HP BladeSystem

More from The Register

next story
BBC goes offline in MASSIVE COCKUP: Stephen Fry partly muzzled
Auntie tight-lipped as major outage rolls on
There's NOTHING on TV in Europe – American video DOMINATES
Even France's mega subsidies don't stop US content onslaught
You! Pirate! Stop pirating, or we shall admonish you politely. Repeatedly, if necessary
And we shall go about telling people you smell. No, not really
Airbus promises Wi-Fi – yay – and 3D movies (meh) in new A330
If the person in front reclines their seat, this could get interesting
UK Parliament rubber-stamps EMERGENCY data grab 'n' keep bill
Just 49 MPs oppose Drip's rushed timetable
Samsung threatens to cut ties with supplier over child labour allegations
Vows to uphold 'zero tolerance' policy on underage workers
Dude, you're getting a Dell – with BITCOIN: IT giant slurps cryptocash
1. Buy PC with Bitcoin. 2. Mine more coins. 3. Goto step 1
ITC: Seagate and LSI can infringe Realtek patents because Realtek isn't in the US
Land of the (get off scot) free, when it's a foreign owner
prev story

Whitepapers

Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.