Feeds

When algorithms attack, does Google hear you scream?

Inside Google's search penalties gulag

  • alert
  • submit to reddit

The Power of One eBook: Top reasons to choose HP BladeSystem

First on Yahoo!, 119th on Google

That simple search, incidentally, might well be what the typical user would hack in if they were looking for cheap air tickets, but for that very reason it doesn't necessarily prove anything in the wonderful world of Search Engine Optimisation - the keywords are extremely competitive.

More specific queries, however, give a cleaner picture of the impact of the penalty. This search for a price comparison of a motorcycle helmet, for example, places Foundem first on Yahoo, sixth on Bing, and one hundred and nineteenth on Google at time of writing.

The Register's Department of Cracked Records feels it should point out here that Google also declined to comment on why apparently similar sites remained unpenalised when others were penalised, and whether the disparity in results between Google and Yahoo! and Bing meant that Google was in some way 'better', or that Google was merely exercising an editorial judgment.

The waters are muddied further by other instances of the Google Big Red Switch in operation. On the same day that that Foundem disappeared from Google search, many other sites, including amazon.com, also vanished. Google later said this was due to a "technical issue", and The Register understands that this was indeed unrelated to the algorithm that hit Foundem. But if you can't get any sensible information out of Google (and just saying "technical issue" hardly counts as sensible information), then you've no idea whether you need to fix something, or whether it was just somebody at Google with a fat finger.

Other sites, however, have certainly been subjected to Google's quality penalties for lengthy periods. Sourcetool.com, for example, was hit with an AdWords penalty in 2006, and filed suit against Google earlier this year.

And there are aspects of Sourcetool's case that are strikingly familiar. Says New York Times writer Joe Nocera: "When I pressed Mr. Fox [Google ad quality product manager] about Sourcetool, he refused to tell me why the algorithm had problems with the site. When I asked him why the business.com site was in the algorithm's good graces but Sourcetool's wasn't, he wouldn't tell me that, either. All I got were platitudes about the user experience.

"It wasn’t long before I was almost as exasperated as Mr. Savage [Sourcetool boss Dan Savage]. How can you adapt your business model to Google's specs if Google won’t tell you what the specs are?

"Google also told me that it never made judgments of what was 'good' and 'bad' because it was all in the hands of the algorithm. But that turns out not to be completely true. Mr. Savage shared with me an e-mail message from a Google account executive to someone at another company who had run into the same kind of landing page problem as Sourcetool.

"The Google account executive wrote back to say that she had looked at the site and found that 'there seems to be a wealth of valuable information on the site.' Consequently, her team overruled the algorithm."

A number of high profile sites, including moneysupermarket.com, confused.com and gocompare.com have also been penalised, suffering disappearances from a couple of days to several weeks, since 2006. Sites are often penalised for perfectly valid reasons - for example, for using paid links and reviews to boost their Google rankings for competitive keywords - and usually when a site's ranking goes into freefall, its operator is widely assumed to be up to no good.

Google won't talk about it, the companies penalised won't talk about it, and it's very seldom that - as in the case of Foundem - the victim goes public and challenges Google to explain and justify its actions. And this tends to reinforce the outside world's view of Google's opinion as perfect - if Google's actions aren't publicly challenged, then Google's actions must have been right, right?

Follow the money

Several Media Corp sites, including gambling.com and www.creditcardexpert.co.uk/, had penalties imposed in October 2007, but were reprieved just last month, two years later. Announcing the lifting of the penalties, Media Corp CEO Justin Drummond commented: "The removal of the Google penalty is a hugely significant event for Media Corp and should sharply increase the revenues derived from these two websites." The effect of the penalty, the company said, had "significantly reduced inbound visitor traffic from Google which, in turn, adversely impacted revenues on these websites".

The Media Corp episode raises some interesting questions about the lack of transparency in Google's processes. Media Corp bought gambling.com for £11m in 2005, and Google's penalty will clearly have damaged the value of the site. In the wake of the announcement, however, Media Corp shares rose 25 per cent.

If it is the case that a level playing field algorithm imposes the penalties, then the scope for insider trading is limited. But as humans are involved in the discussion and implementation of the whitelisting process, one day there will clearly be a need for Google to demonstrate to financial authorities that its systems cannot be used to manipulate stock prices.

We queried this with Google, but were merely referred again to the published "guidelines" for search and for AdWords.

Media Corp did not respond to The Register's questions regarding the steps it took to have the penalties removed. An Evening Standard report, however, had Drummond blaming either a malicious rival or a "bug in the technology" (which it clearly was not), and said that the problem had been sorted out with the help of a consultancy firm appointed a few weeks previously. It's not clear what it was that this firm did to help turn the situation around, but it could not have been SEO, and The Register understands that the firm was not an SEO specialist.

Alongside the long exiles there are numerous cases of swift reinstatements. These, Raff argues, indicate that there is a fast whitelisting process available for high profile sites. As we've seen, there clearly is a manual whitelisting process for search, and - for price comparison sites at least - this appears to involve a certain amount of human value judgement as to what does and does not constitute 'added value'.

One can also reasonably presume that the response from Google support will be markedly different for a huge beast like Amazon and a small startup - this does not require the existence of some controlling intelligence intent on favouring the big boys, it's just the way of the world. The playing field will never be entirely even, but it might at least be a little more transparent.

Top three mobile application threats

More from The Register

next story
Stick a 4K in them: Super high-res TVs are DONE
4,000 pixels is niche now... Don't say we didn't warn you
BBC goes offline in MASSIVE COCKUP: Stephen Fry partly muzzled
Auntie tight-lipped as major outage rolls on
Philip K Dick 'Nazi alternate reality' story to be made into TV series
Amazon Studios, Ridley Scott firm to produce The Man in the High Castle
iPad? More like iFAD: We reveal why Apple fell into IBM's arms
But never fear fanbois, you're still lapping up iPhones, Macs
Bose says today is F*** With Dre Day: Beats sued in patent battle
Music gear giant seeks some of that sweet, sweet Apple pie
There's NOTHING on TV in Europe – American video DOMINATES
Even France's mega subsidies don't stop US content onslaught
You! Pirate! Stop pirating, or we shall admonish you politely. Repeatedly, if necessary
And we shall go about telling people you smell. No, not really
Too many IT conferences to cover? MICROSOFT to the RESCUE!
Yet more word of cuts emerges from Redmond
Joe Average isn't worth $10 a year to Mark Zuckerberg
The Social Network deflates the PC resurgence with mobile-only usage prediction
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Securing Web Applications Made Simple and Scalable
Learn how automated security testing can provide a simple and scalable way to protect your web applications.