Feeds

Fast Search claims Google's size crown

For Now

  • alert
  • submit to reddit

Maximizing your infrastructure through virtualization

ComputerWire: IT Industry Intelligence

Fast Search & Transfer ASA has made a play for Google Inc's reputation as the web's largest search engine, in an effort to create a better brand to drive its enterprise business,

Kevin Murphy writes

. Fast said its consumer-facing site, AllTheWeb.com, now searches an index of 2,095,568,809 web pages, just over 20 million more than Google claims.

"What we've been doing for the last couple of years is focusing on the freshness and relevancy of our results, developing our technology," said Fast senior product marketing manager Jami Axelrod. "We decided four months ago we were very comfortable with the index's freshness, and decided to focus on size."

Back in 1999, when Fast and Google first emerged on the scene as formidable players, search firms could have a 200 million-document index and claim the crown of largest. Today, they need over ten times that amount, and they still are not indexing the entire web. Fast's last major milestone was 600 million pages, back in November.

Everyone is in agreement that size is not that important from a practical point of view. People searching for "Britney Spears" surely don't need the 909,000 results Google brings back, or the 3,013,365 hits AllTheWeb claims. But those searching for a more obscure phrase may find their only success comes from searching a large index.

"We definitely welcome competition," said Google spokesperson David Krane. "But size is just one component used in the overall evaluation of quality." Freshness, speed and relevancy are all important factors that consumers and businesses use when making a search engine decision, he said

Fast says a large portion of its index, the popular content, is re-spidered every seven to 11 days, with the remainder being spidered every month or so. Google says its most refreshed pages are spidered every 15 minutes, with the entire index getting a refresh every 28 days.

AllTheWeb.com is not a huge revenue generator for Fast. "It's a showcase, an R&D sandbox," said Axelrod. Google is making inroads selling its technology boxed to web companies and other enterprises, driven somewhat by its strong online brand, and Fast wants to have similar successes.

Being known as the web's largest search engine would be a good start to building a brand, although it seems unlikely Google will take this snub lying down. Spokespeople wouldn't comment on plans to expand the Google index by more than 20 million, to leapfrog Fast, though Fast seems to be expecting it. When WiseNut Inc claimed to beat Google last year, Google quickly updated its claims to remain the largest

In the battle to sell search technology into enterprises, which is where the bulk of the revenue opportunity has been since the portal market collapsed, yesterday Google announced enhancements to its GB-1001 search appliance. The company said that both the 1001 and the GB-8008 have been selling well into enterprises. Recent customers include National Semiconductor, the University of Florida, Cisco, Boeing and PBS.

The GB-1001, targeted at branch offices and smaller businesses, has been upgraded to scale to 300,000 documents, a spokesperson said. A key new feature allows the appliance to index documents addressable via dynamic URLs and cookies, such as those found in personalized or e-commerce content.

Fast has been upgrading its web site, although many of the new features AllTheWeb contains have been used at Google for some time, such as the ability to do full-text searches on PDF documents. One feature Google doesn't have yet, but which has been championed by smaller rivals such as Ask Jeeves Inc's Teoma.com and LookSmart Ltd's WiseNut.com, is the ability to dynamically categorize content.

AllTheWeb now suggests up to four possible categories when a user makes a query. Words associated with the query are suggested based on data scraped from AllTheWeb's query logs. For example, if your keyword is Saturn, AllTheWeb may suggest "Sega Saturn" or "Planet Saturn" or "Saturn Car", depending on what previous searchers have entered.

© ComputerWire.

The Power of One Infographic

More from The Register

next story
BBC goes offline in MASSIVE COCKUP: Stephen Fry partly muzzled
Auntie tight-lipped as major outage rolls on
There's NOTHING on TV in Europe – American video DOMINATES
Even France's mega subsidies don't stop US content onslaught
You! Pirate! Stop pirating, or we shall admonish you politely. Repeatedly, if necessary
And we shall go about telling people you smell. No, not really
Airbus promises Wi-Fi – yay – and 3D movies (meh) in new A330
If the person in front reclines their seat, this could get interesting
UK Parliament rubber-stamps EMERGENCY data grab 'n' keep bill
Just 49 MPs oppose Drip's rushed timetable
Want to beat Verizon's slow Netflix? Get a VPN
Exec finds stream speed climbs when smuggled out
Samsung threatens to cut ties with supplier over child labour allegations
Vows to uphold 'zero tolerance' policy on underage workers
Dude, you're getting a Dell – with BITCOIN: IT giant slurps cryptocash
1. Buy PC with Bitcoin. 2. Mine more coins. 3. Goto step 1
prev story

Whitepapers

Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.