Feeds

When algorithms attack, does Google hear you scream?

Inside Google's search penalties gulag

  • alert
  • submit to reddit

Security for virtualized datacentres

First on Yahoo!, 119th on Google

That simple search, incidentally, might well be what the typical user would hack in if they were looking for cheap air tickets, but for that very reason it doesn't necessarily prove anything in the wonderful world of Search Engine Optimisation - the keywords are extremely competitive.

More specific queries, however, give a cleaner picture of the impact of the penalty. This search for a price comparison of a motorcycle helmet, for example, places Foundem first on Yahoo, sixth on Bing, and one hundred and nineteenth on Google at time of writing.

The Register's Department of Cracked Records feels it should point out here that Google also declined to comment on why apparently similar sites remained unpenalised when others were penalised, and whether the disparity in results between Google and Yahoo! and Bing meant that Google was in some way 'better', or that Google was merely exercising an editorial judgment.

The waters are muddied further by other instances of the Google Big Red Switch in operation. On the same day that that Foundem disappeared from Google search, many other sites, including amazon.com, also vanished. Google later said this was due to a "technical issue", and The Register understands that this was indeed unrelated to the algorithm that hit Foundem. But if you can't get any sensible information out of Google (and just saying "technical issue" hardly counts as sensible information), then you've no idea whether you need to fix something, or whether it was just somebody at Google with a fat finger.

Other sites, however, have certainly been subjected to Google's quality penalties for lengthy periods. Sourcetool.com, for example, was hit with an AdWords penalty in 2006, and filed suit against Google earlier this year.

And there are aspects of Sourcetool's case that are strikingly familiar. Says New York Times writer Joe Nocera: "When I pressed Mr. Fox [Google ad quality product manager] about Sourcetool, he refused to tell me why the algorithm had problems with the site. When I asked him why the business.com site was in the algorithm's good graces but Sourcetool's wasn't, he wouldn't tell me that, either. All I got were platitudes about the user experience.

"It wasn’t long before I was almost as exasperated as Mr. Savage [Sourcetool boss Dan Savage]. How can you adapt your business model to Google's specs if Google won’t tell you what the specs are?

"Google also told me that it never made judgments of what was 'good' and 'bad' because it was all in the hands of the algorithm. But that turns out not to be completely true. Mr. Savage shared with me an e-mail message from a Google account executive to someone at another company who had run into the same kind of landing page problem as Sourcetool.

"The Google account executive wrote back to say that she had looked at the site and found that 'there seems to be a wealth of valuable information on the site.' Consequently, her team overruled the algorithm."

A number of high profile sites, including moneysupermarket.com, confused.com and gocompare.com have also been penalised, suffering disappearances from a couple of days to several weeks, since 2006. Sites are often penalised for perfectly valid reasons - for example, for using paid links and reviews to boost their Google rankings for competitive keywords - and usually when a site's ranking goes into freefall, its operator is widely assumed to be up to no good.

Google won't talk about it, the companies penalised won't talk about it, and it's very seldom that - as in the case of Foundem - the victim goes public and challenges Google to explain and justify its actions. And this tends to reinforce the outside world's view of Google's opinion as perfect - if Google's actions aren't publicly challenged, then Google's actions must have been right, right?

Follow the money

Several Media Corp sites, including gambling.com and www.creditcardexpert.co.uk/, had penalties imposed in October 2007, but were reprieved just last month, two years later. Announcing the lifting of the penalties, Media Corp CEO Justin Drummond commented: "The removal of the Google penalty is a hugely significant event for Media Corp and should sharply increase the revenues derived from these two websites." The effect of the penalty, the company said, had "significantly reduced inbound visitor traffic from Google which, in turn, adversely impacted revenues on these websites".

The Media Corp episode raises some interesting questions about the lack of transparency in Google's processes. Media Corp bought gambling.com for £11m in 2005, and Google's penalty will clearly have damaged the value of the site. In the wake of the announcement, however, Media Corp shares rose 25 per cent.

If it is the case that a level playing field algorithm imposes the penalties, then the scope for insider trading is limited. But as humans are involved in the discussion and implementation of the whitelisting process, one day there will clearly be a need for Google to demonstrate to financial authorities that its systems cannot be used to manipulate stock prices.

We queried this with Google, but were merely referred again to the published "guidelines" for search and for AdWords.

Media Corp did not respond to The Register's questions regarding the steps it took to have the penalties removed. An Evening Standard report, however, had Drummond blaming either a malicious rival or a "bug in the technology" (which it clearly was not), and said that the problem had been sorted out with the help of a consultancy firm appointed a few weeks previously. It's not clear what it was that this firm did to help turn the situation around, but it could not have been SEO, and The Register understands that the firm was not an SEO specialist.

Alongside the long exiles there are numerous cases of swift reinstatements. These, Raff argues, indicate that there is a fast whitelisting process available for high profile sites. As we've seen, there clearly is a manual whitelisting process for search, and - for price comparison sites at least - this appears to involve a certain amount of human value judgement as to what does and does not constitute 'added value'.

One can also reasonably presume that the response from Google support will be markedly different for a huge beast like Amazon and a small startup - this does not require the existence of some controlling intelligence intent on favouring the big boys, it's just the way of the world. The playing field will never be entirely even, but it might at least be a little more transparent.

Beginner's guide to SSL certificates

More from The Register

next story
Bono apologises for iTunes album dump
Megalomania, generosity and FEAR of irrelevance drove group to Apple deal
HBO shocks US pay TV world: We're down with OTT. Netflix says, 'Gee'
This affects every broadcaster, every cable guy
Facebook, Apple: LADIES! Why not FREEZE your EGGS? It's on the company!
No biological clockwatching when you work in Silicon Valley
French 'terror law' declares WAR on the INTERNET itself, say digi-rights folks
Liberté, Égalité, Fraternité: Two out of three ain't bad
SCREW YOU, EU: BBC rolls out Right To Remember as Google deletes links
Not even Google can withstand the power of Auntie
Arab States make play for greater government control of the internet
Nerds told to get lost in last-minute power grab bid at UN meeting
Zippy one-liners, broken promises: Doctor Who on the Orient Express
Series finally hits stride, but Clara's U-turn is baffling
Don't bother telling people if you lose their data, say Euro bods
You read that right – with the proviso that it's encrypted
America's super-secret X-37B plane returns to Earth after nearly TWO YEARS aloft
674 days in space for US Air Force's mystery orbital vehicle
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.