Feeds

Fast Search claims Google's size crown

For Now

  • alert
  • submit to reddit

High performance access to file storage

ComputerWire: IT Industry Intelligence

Fast Search & Transfer ASA has made a play for Google Inc's reputation as the web's largest search engine, in an effort to create a better brand to drive its enterprise business,

Kevin Murphy writes

. Fast said its consumer-facing site, AllTheWeb.com, now searches an index of 2,095,568,809 web pages, just over 20 million more than Google claims.

"What we've been doing for the last couple of years is focusing on the freshness and relevancy of our results, developing our technology," said Fast senior product marketing manager Jami Axelrod. "We decided four months ago we were very comfortable with the index's freshness, and decided to focus on size."

Back in 1999, when Fast and Google first emerged on the scene as formidable players, search firms could have a 200 million-document index and claim the crown of largest. Today, they need over ten times that amount, and they still are not indexing the entire web. Fast's last major milestone was 600 million pages, back in November.

Everyone is in agreement that size is not that important from a practical point of view. People searching for "Britney Spears" surely don't need the 909,000 results Google brings back, or the 3,013,365 hits AllTheWeb claims. But those searching for a more obscure phrase may find their only success comes from searching a large index.

"We definitely welcome competition," said Google spokesperson David Krane. "But size is just one component used in the overall evaluation of quality." Freshness, speed and relevancy are all important factors that consumers and businesses use when making a search engine decision, he said

Fast says a large portion of its index, the popular content, is re-spidered every seven to 11 days, with the remainder being spidered every month or so. Google says its most refreshed pages are spidered every 15 minutes, with the entire index getting a refresh every 28 days.

AllTheWeb.com is not a huge revenue generator for Fast. "It's a showcase, an R&D sandbox," said Axelrod. Google is making inroads selling its technology boxed to web companies and other enterprises, driven somewhat by its strong online brand, and Fast wants to have similar successes.

Being known as the web's largest search engine would be a good start to building a brand, although it seems unlikely Google will take this snub lying down. Spokespeople wouldn't comment on plans to expand the Google index by more than 20 million, to leapfrog Fast, though Fast seems to be expecting it. When WiseNut Inc claimed to beat Google last year, Google quickly updated its claims to remain the largest

In the battle to sell search technology into enterprises, which is where the bulk of the revenue opportunity has been since the portal market collapsed, yesterday Google announced enhancements to its GB-1001 search appliance. The company said that both the 1001 and the GB-8008 have been selling well into enterprises. Recent customers include National Semiconductor, the University of Florida, Cisco, Boeing and PBS.

The GB-1001, targeted at branch offices and smaller businesses, has been upgraded to scale to 300,000 documents, a spokesperson said. A key new feature allows the appliance to index documents addressable via dynamic URLs and cookies, such as those found in personalized or e-commerce content.

Fast has been upgrading its web site, although many of the new features AllTheWeb contains have been used at Google for some time, such as the ability to do full-text searches on PDF documents. One feature Google doesn't have yet, but which has been championed by smaller rivals such as Ask Jeeves Inc's Teoma.com and LookSmart Ltd's WiseNut.com, is the ability to dynamically categorize content.

AllTheWeb now suggests up to four possible categories when a user makes a query. Words associated with the query are suggested based on data scraped from AllTheWeb's query logs. For example, if your keyword is Saturn, AllTheWeb may suggest "Sega Saturn" or "Planet Saturn" or "Saturn Car", depending on what previous searchers have entered.

© ComputerWire.

High performance access to file storage

More from The Register

next story
MtGox chief Karpelès refuses to come to US for g-men's grilling
Bitcoin baron says he needs another lawyer for FinCEN chat
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
Audio fans, prepare yourself for the Second Coming ... of Blu-ray
High Fidelity Pure Audio – is this what your ears have been waiting for?
Did a date calculation bug just cost hard-up Co-op Bank £110m?
And just when Brit banking org needs £400m to stay afloat
Sorry London, Europe's top tech city is Munich
New 'Atlas of ICT Activity' finds innovation isn't happening at Silicon Roundabout
Zucker punched: Google gobbles Facebook-wooed Titan Aerospace
Up, up and away in my beautiful balloon flying broadband-bot
Apple DOMINATES the Valley, rakes in more profit than Google, HP, Intel, Cisco COMBINED
Cook & Co. also pay more taxes than those four worthies PLUS eBay and Oracle
It may be ILLEGAL to run Heartbleed health checks – IT lawyer
Do the right thing, earn up to 10 years in clink
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
HP ArcSight ESM solution helps Finansbank
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mobile application security study
Download this report to see the alarming realities regarding the sheer number of applications vulnerable to attack, as well as the most common and easily addressable vulnerability errors.