Feeds

Fast Search claims Google's size crown

For Now

  • alert
  • submit to reddit

Top 5 reasons to deploy VMware with Tegile

ComputerWire: IT Industry Intelligence

Fast Search & Transfer ASA has made a play for Google Inc's reputation as the web's largest search engine, in an effort to create a better brand to drive its enterprise business,

Kevin Murphy writes

. Fast said its consumer-facing site, AllTheWeb.com, now searches an index of 2,095,568,809 web pages, just over 20 million more than Google claims.

"What we've been doing for the last couple of years is focusing on the freshness and relevancy of our results, developing our technology," said Fast senior product marketing manager Jami Axelrod. "We decided four months ago we were very comfortable with the index's freshness, and decided to focus on size."

Back in 1999, when Fast and Google first emerged on the scene as formidable players, search firms could have a 200 million-document index and claim the crown of largest. Today, they need over ten times that amount, and they still are not indexing the entire web. Fast's last major milestone was 600 million pages, back in November.

Everyone is in agreement that size is not that important from a practical point of view. People searching for "Britney Spears" surely don't need the 909,000 results Google brings back, or the 3,013,365 hits AllTheWeb claims. But those searching for a more obscure phrase may find their only success comes from searching a large index.

"We definitely welcome competition," said Google spokesperson David Krane. "But size is just one component used in the overall evaluation of quality." Freshness, speed and relevancy are all important factors that consumers and businesses use when making a search engine decision, he said

Fast says a large portion of its index, the popular content, is re-spidered every seven to 11 days, with the remainder being spidered every month or so. Google says its most refreshed pages are spidered every 15 minutes, with the entire index getting a refresh every 28 days.

AllTheWeb.com is not a huge revenue generator for Fast. "It's a showcase, an R&D sandbox," said Axelrod. Google is making inroads selling its technology boxed to web companies and other enterprises, driven somewhat by its strong online brand, and Fast wants to have similar successes.

Being known as the web's largest search engine would be a good start to building a brand, although it seems unlikely Google will take this snub lying down. Spokespeople wouldn't comment on plans to expand the Google index by more than 20 million, to leapfrog Fast, though Fast seems to be expecting it. When WiseNut Inc claimed to beat Google last year, Google quickly updated its claims to remain the largest

In the battle to sell search technology into enterprises, which is where the bulk of the revenue opportunity has been since the portal market collapsed, yesterday Google announced enhancements to its GB-1001 search appliance. The company said that both the 1001 and the GB-8008 have been selling well into enterprises. Recent customers include National Semiconductor, the University of Florida, Cisco, Boeing and PBS.

The GB-1001, targeted at branch offices and smaller businesses, has been upgraded to scale to 300,000 documents, a spokesperson said. A key new feature allows the appliance to index documents addressable via dynamic URLs and cookies, such as those found in personalized or e-commerce content.

Fast has been upgrading its web site, although many of the new features AllTheWeb contains have been used at Google for some time, such as the ability to do full-text searches on PDF documents. One feature Google doesn't have yet, but which has been championed by smaller rivals such as Ask Jeeves Inc's Teoma.com and LookSmart Ltd's WiseNut.com, is the ability to dynamically categorize content.

AllTheWeb now suggests up to four possible categories when a user makes a query. Words associated with the query are suggested based on data scraped from AllTheWeb's query logs. For example, if your keyword is Saturn, AllTheWeb may suggest "Sega Saturn" or "Planet Saturn" or "Saturn Car", depending on what previous searchers have entered.

© ComputerWire.

Beginner's guide to SSL certificates

More from The Register

next story
Facebook pays INFINITELY MORE UK corp tax than in 2012
Thanks for the £3k, Zuck. Doh! you're IN CREDIT. Guess not
Big Content outs piracy hotbeds: São Paulo, Beijing ... TORONTO?
MPAA calls Canadians a bunch of bootlegging movie thieves
Google Glassholes are UNDATEABLE – HP exec
You need an emotional connection, says touchy-feely MD... We can do that
Just don't blame Bono! Apple iTunes music sales PLUMMET
Cupertino revenue hit by cheapo downloads, says report
US court SHUTS DOWN 'scammers posing as Microsoft, Facebook support staff'
Netizens allegedly duped into paying for bogus tech advice
Feds seek potential 'second Snowden' gov doc leaker – report
Hang on, Ed wasn't here when we compiled THIS document
Verizon bankrolls tech news site, bans tech's biggest stories
No agenda here. Just don't ever mention Net neutrality or spying, ok?
prev story

Whitepapers

Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
Simplify SSL certificate management across the enterprise
Simple steps to take control of SSL across the enterprise, and recommendations for a management platform for full visibility and single-point of control for these Certificates.