Google explains new page rankings

Pigeons now more bomb resistant

  • alert
  • submit to reddit

The Essential Guide to IT Transformation

Some sanity returned to Google 's page rankings last week, and the only damage incurred appears to be a few bruised egos. So we were surprised when Wired adopted a note of faux outrage (hey! that's our job) in its coverage of the new index.

"Geeks Aghast"! reported Paul Boutin: "The inevitable backlash finally appears to have hit the world's most popular search engine." An anonymous poster cited by the report declares that PigeonRank™" ... is dead."

Is it? Perhaps not: by the end of the first page the report tentatively admits "In fact, it may just be bloggers who are bothered," and on page two, the web developer who on page one describes the new index as "seriously degraded" after his own online journal fell from first to sixth in the Google rankings, admits the old index was "probably not fair." So this appears to be a case of over-enthusiastic sub-editing.

In fact, the new index corrects one of the weirder skews prompted by the explosion of online journals (And surely no one calls them blogs anymore … that's soooo 2001).

The effect was to bump obscure but densely-linked sites high in the rankings. When deployed to a particular effect, it's called "Google bombing", but even determined hyperlinkers could benefit. With the latest redeployment of Google's pigeons, this skew appears to be mitigated.

Not everyone who was flattered by the rankings is taking it hard.

"Amazing what people get upset about," noted Dave Winer, who was no longer the world's number one Dave. "I still think this is ridiculous, I rank higher than The Dave Matthews Band. Hello. They're famous."

"When people say they're taking food out of their family's mouth, I think they should get a real job. Depending on the vagaries of an algorithm programmed by engineers at a VC-backed Silicon Valley dotcom-vestigial company is not a good idea."

Well, quite.

Not that Google itself will shed much light on the specifics of the new algorithms. When we caught up with Google software engineer Matt Cutts (and not Cutz, as we wrote) yesterday, he denied that this month's index was a specific reaction to the Blog Skew:

"We're always trying to improve the quality of the index - this is not a reaction to a particular search," he told us.

"Google loves the weblogging community, because it creates useful content and helps us categorize the web. Webloggers produce great content."

The scoring improvements, said Cutts, are both qualitative and quantitative.

"We're always trying new experiments and this was a natural extension of that; we have human evaluators who test each index. Things have to score higher before we push a new index out."

Cutts reckoned that "fad" of Google bombing had all but died out: "You don't see may people doing it all: a lot of the time people write news stories, and when you search for a particular 'bomb' the stories rank higher."

We wondered if there was any cross-pollination between the Usenet trust metrics and the Web algorithms. Would a high score for Kibo, the humorist who made his reputation on net news, feed into Kibo's web ranking?

Cutts said it was an interesting idea.

"If you were searching for information on an intricate part of the Linux kernel you'd love to see a post by Linus on the term come up first," he said. "We're always looking to find a better trust metric."

And ideally, Google would have one search box:

"Realistically we'd like to have one place to search where you don't have to have tabs for each category: web, Usenet, news."

Cutts shed some light on Google's news service:

"It's a combination of PageRank - how many sources are writing about a story, and how recent they are." Google examines over 4,000 news sources, he said, and is constantly revisiting and rechecking them.

We wondered - purely out of self-interest, of course - how the News search could track a story that develops.

"A much larger portion of the web than anyone could guess is duplicates, and it's not just the 5,000 pages of Linux man page and duplicate sites. We've actually had to come up with a lot of different algorithms: not just hard matches, but soft matches."

Obviously dozens of sources republish the same wire content, and much of that doesn't need to be presented in the search results, he said.

Google is erroneously seen as a truth machine, but there's something it can't do very much about: it only captures public discourse. On private email lists, in IM rooms, in irc channels and on closed CIX conferences, for example, the robots don't venture.

"There's always going to be places we can't go," agrees Cutts. That's only a problem if you view Google as an opinion pollster, though: when the signal to noise ratio in public gets too high, or where's there's too much prattle, we go behind closed doors.

Curiously enough, it was this factor and the Blog Skew that prompted me to call Google a couple of weeks ago. Buzz Aldrin had just been cleared of an assault charge for striking a fruitcake who believes the moon landings had been faked, and I wondered how long it would be before a search for Buzz Aldrin began to return entries from the lunatic fringe.

And one of my favorite b-bl-bl journals (an antidote to the armchair pugilists Governor Ventura calls 'Chickenhawks') had grabbed top spot in Google's page rank for "Demosthenes". Which as the author readily admitted was absurd, and not very helpful if you were doing a school project on ancient Greece.

But Google indicated to world+dog that it was aware of the problem with its new index, last week. For which we ought to thank columnist Jon Udell, perhaps - he says he buttonholed Sergey Brin about the skew recently.

The first reference to the Greek Demosthenes only appears at No.3 in the current index - so Google hasn't entirely fixed the skew yet. But it is improving. ®

Build a business case: developing custom apps

More from The Register

next story
iPad? More like iFAD: We reveal why Apple fell into IBM's arms
But never fear fanbois, you're still lapping up iPhones, Macs
Sonos AXES support for Apple's iOS4 and 5
Want to use your iThing? You can't - it's too old
Amazon says Hachette should lower ebook prices, pay authors more
Oh yeah ... and a 30% cut for Amazon to seal the deal
Philip K Dick 'Nazi alternate reality' story to be made into TV series
Amazon Studios, Ridley Scott firm to produce The Man in the High Castle
Joe Average isn't worth $10 a year to Mark Zuckerberg
The Social Network deflates the PC resurgence with mobile-only usage prediction
Chips are down at Broadcom: Thousands of workers laid off
Cellphone baseband device biz shuttered
Feel free to BONK on the TUBE, says Transport for London
Plus: Almost NOBODY uses pay-by-bonk on buses - Visa
Twitch rich as Google flicks $1bn hitch switch, claims snitch
Gameplay streaming biz and search king refuse to deny fresh gobble rumors
Stick a 4K in them: Super high-res TVs are DONE
4,000 pixels is niche now... Don't say we didn't warn you
prev story


Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.
Maximize storage efficiency across the enterprise
The HP StoreOnce backup solution offers highly flexible, centrally managed, and highly efficient data protection for any enterprise.