Feeds

Fast Search claims Google's size crown

For Now

  • alert
  • submit to reddit

Protecting users from Firesheep and other Sidejacking attacks with SSL

ComputerWire: IT Industry Intelligence

Fast Search & Transfer ASA has made a play for Google Inc's reputation as the web's largest search engine, in an effort to create a better brand to drive its enterprise business,

Kevin Murphy writes

. Fast said its consumer-facing site, AllTheWeb.com, now searches an index of 2,095,568,809 web pages, just over 20 million more than Google claims.

"What we've been doing for the last couple of years is focusing on the freshness and relevancy of our results, developing our technology," said Fast senior product marketing manager Jami Axelrod. "We decided four months ago we were very comfortable with the index's freshness, and decided to focus on size."

Back in 1999, when Fast and Google first emerged on the scene as formidable players, search firms could have a 200 million-document index and claim the crown of largest. Today, they need over ten times that amount, and they still are not indexing the entire web. Fast's last major milestone was 600 million pages, back in November.

Everyone is in agreement that size is not that important from a practical point of view. People searching for "Britney Spears" surely don't need the 909,000 results Google brings back, or the 3,013,365 hits AllTheWeb claims. But those searching for a more obscure phrase may find their only success comes from searching a large index.

"We definitely welcome competition," said Google spokesperson David Krane. "But size is just one component used in the overall evaluation of quality." Freshness, speed and relevancy are all important factors that consumers and businesses use when making a search engine decision, he said

Fast says a large portion of its index, the popular content, is re-spidered every seven to 11 days, with the remainder being spidered every month or so. Google says its most refreshed pages are spidered every 15 minutes, with the entire index getting a refresh every 28 days.

AllTheWeb.com is not a huge revenue generator for Fast. "It's a showcase, an R&D sandbox," said Axelrod. Google is making inroads selling its technology boxed to web companies and other enterprises, driven somewhat by its strong online brand, and Fast wants to have similar successes.

Being known as the web's largest search engine would be a good start to building a brand, although it seems unlikely Google will take this snub lying down. Spokespeople wouldn't comment on plans to expand the Google index by more than 20 million, to leapfrog Fast, though Fast seems to be expecting it. When WiseNut Inc claimed to beat Google last year, Google quickly updated its claims to remain the largest

In the battle to sell search technology into enterprises, which is where the bulk of the revenue opportunity has been since the portal market collapsed, yesterday Google announced enhancements to its GB-1001 search appliance. The company said that both the 1001 and the GB-8008 have been selling well into enterprises. Recent customers include National Semiconductor, the University of Florida, Cisco, Boeing and PBS.

The GB-1001, targeted at branch offices and smaller businesses, has been upgraded to scale to 300,000 documents, a spokesperson said. A key new feature allows the appliance to index documents addressable via dynamic URLs and cookies, such as those found in personalized or e-commerce content.

Fast has been upgrading its web site, although many of the new features AllTheWeb contains have been used at Google for some time, such as the ability to do full-text searches on PDF documents. One feature Google doesn't have yet, but which has been championed by smaller rivals such as Ask Jeeves Inc's Teoma.com and LookSmart Ltd's WiseNut.com, is the ability to dynamically categorize content.

AllTheWeb now suggests up to four possible categories when a user makes a query. Words associated with the query are suggested based on data scraped from AllTheWeb's query logs. For example, if your keyword is Saturn, AllTheWeb may suggest "Sega Saturn" or "Planet Saturn" or "Saturn Car", depending on what previous searchers have entered.

© ComputerWire.

Website security in corporate America

More from The Register

next story
Hey, Scots. Microsoft's Bing thinks you'll vote NO to independence
World's top Google-finding website calls it for the UK
Phones 4u slips into administration after EE cuts ties with Brit mobe retailer
More than 5,500 jobs could be axed if rescue mission fails
Apple CEO Tim Cook: TV is TERRIBLE and stuck in the 1970s
The iKing thinks telly is far too fiddly and ugly – basically, iTunes
Israeli spies rebel over mass-snooping on innocent Palestinians
'Disciplinary treatment will be sharp and clear' vow spy-chiefs
Huawei ditches new Windows Phone mobe plans, blames poor sales
Giganto mobe firm slams door shut on Microsoft. OH DEAR
Phones 4u website DIES as wounded mobe retailer struggles to stay above water
Founder blames 'ruthless network partners' for implosion
Found inside ISIS terror chap's laptop: CELINE DION tunes
REPORT: Stash of terrorist material found in Syria Dell box
OECD lashes out at tax avoiding globocorps' location-flipping antics
You hear that, Amazon, Google, Microsoft et al?
prev story

Whitepapers

Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.
Security and trust: The backbone of doing business over the internet
Explores the current state of website security and the contributions Symantec is making to help organizations protect critical data and build trust with customers.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.