Feeds

Google Research head dubs holy PageRank 'over-hyped'

Norvig mum on 'Caffeine' search shot

The Power of One eBook: Top reasons to choose HP BladeSystem

Google research head Peter Norvig says that the search giant's hallowed PageRank link-analysis algorithm is overrated. And always has been.

"One thing that I think is still over-hyped is PageRank," Norvig said this morning during a question and answer keynote at the search-obsessed SMX West conference in Santa Clara, California. "People think we just do this computation on the web graph and order all the pages and that's it. That computation is important, but it's just one thing that we do.

"People [webmasters and SEOs] always said, 'We're stuck if we don't have [a high PageRank].' But we never felt that way. We never felt that it was such a big factor."

PageRank attempts to measure the relative importance of a website based on what other sites it's linked to. Named for Google co-founder Larry Page - who developed the idea while at Stanford University - the patented technology is a central pillar in the Google Mythology, receiving much of the credit for the Mountain View search engine's rise to world dominance. The PageRank patent actually belongs to Stanford, with Google owning exclusive license rights.

Late last year, Google added so-called "real-time search" to its engine - serving up links to fresh Tweets, news, and other recent web posts - and this morning, Norvig was asked if this was a far more difficult undertaking, considering that PageRank doesn't work well with Web2.0rhea. Norvig - the former director of search quality at Google - was quick to say "no," explaining that even with core search, PageRank isn't as important as people think it is. And never was.

"[PageRank] has a catchy name and the name recognition. But we've always looked at all the things that are available [when ranking search results]. We look at where do things come from, what are the words used, how do they interact with each other, how do people interact with them," he said.

"[Real-time search] is more similar [to core search] than dissimilar, in that you're grabbing every available signal and trying to figure out the best way to combine them. The fact that there aren't legacy links from a long time ago - we don't think of that as that much different."

The key to real-time search, Norvig said, is Google's famously distributed back-end infrastructure, which is able to re-build its web index with relatively little delay. When Norvig first joined the company, the Google web index was built once a month. Then the company moved to once a day and then to once an hour. Now, its distributed infrastructure - using proprietary technologies like the Google File System and MapReduce - can update its index in "10 seconds," according to Norvig.

When the hourly index was rolled out, Norvig remembered, Larry Page insisted on calling it the "3600 second" index. "If it was hourly, it was just going to stay like that," Norvig said. "But if you talk about it in seconds, people are going to push it down to 1000 seconds and eventually you get it down to 10. And that's where we are now. His vision has come true."

Google is currently testing a new search indexing system known as "Caffeine," which uses, among other things, a complete rewrite of the Google File System known at least informally as GFS2. In the fall, uber Googler Matt Cutts indicated that Caffeine would begin rolling out across the company's global infrastructure after the Christmas holidays, but that hasn't happened yet.

Norvig would not be drawn on the state of Caffeine, merely confirming that it is still being tested in a single data center. The Google Research team, he said, has very little to do with the company's infrastructure work. "For historical reasons, that kind of systems programming stuff has not been done in Research," he said. ®

Top three mobile application threats

More from The Register

next story
BBC goes offline in MASSIVE COCKUP: Stephen Fry partly muzzled
Auntie tight-lipped as major outage rolls on
iPad? More like iFAD: We reveal why Apple fell into IBM's arms
But never fear fanbois, you're still lapping up iPhones, Macs
Stick a 4K in them: Super high-res TVs are DONE
4,000 pixels is niche now... Don't say we didn't warn you
Philip K Dick 'Nazi alternate reality' story to be made into TV series
Amazon Studios, Ridley Scott firm to produce The Man in the High Castle
Sonos AXES support for Apple's iOS4 and 5
Want to use your iThing? You can't - it's too old
There's NOTHING on TV in Europe – American video DOMINATES
Even France's mega subsidies don't stop US content onslaught
You! Pirate! Stop pirating, or we shall admonish you politely. Repeatedly, if necessary
And we shall go about telling people you smell. No, not really
Too many IT conferences to cover? MICROSOFT to the RESCUE!
Yet more word of cuts emerges from Redmond
Joe Average isn't worth $10 a year to Mark Zuckerberg
The Social Network deflates the PC resurgence with mobile-only usage prediction
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Securing Web Applications Made Simple and Scalable
Learn how automated security testing can provide a simple and scalable way to protect your web applications.