Feeds

Fast Search claims Google's size crown

For Now

  • alert
  • submit to reddit

Internet Security Threat Report 2014

ComputerWire: IT Industry Intelligence

Fast Search & Transfer ASA has made a play for Google Inc's reputation as the web's largest search engine, in an effort to create a better brand to drive its enterprise business,

Kevin Murphy writes

. Fast said its consumer-facing site, AllTheWeb.com, now searches an index of 2,095,568,809 web pages, just over 20 million more than Google claims.

"What we've been doing for the last couple of years is focusing on the freshness and relevancy of our results, developing our technology," said Fast senior product marketing manager Jami Axelrod. "We decided four months ago we were very comfortable with the index's freshness, and decided to focus on size."

Back in 1999, when Fast and Google first emerged on the scene as formidable players, search firms could have a 200 million-document index and claim the crown of largest. Today, they need over ten times that amount, and they still are not indexing the entire web. Fast's last major milestone was 600 million pages, back in November.

Everyone is in agreement that size is not that important from a practical point of view. People searching for "Britney Spears" surely don't need the 909,000 results Google brings back, or the 3,013,365 hits AllTheWeb claims. But those searching for a more obscure phrase may find their only success comes from searching a large index.

"We definitely welcome competition," said Google spokesperson David Krane. "But size is just one component used in the overall evaluation of quality." Freshness, speed and relevancy are all important factors that consumers and businesses use when making a search engine decision, he said

Fast says a large portion of its index, the popular content, is re-spidered every seven to 11 days, with the remainder being spidered every month or so. Google says its most refreshed pages are spidered every 15 minutes, with the entire index getting a refresh every 28 days.

AllTheWeb.com is not a huge revenue generator for Fast. "It's a showcase, an R&D sandbox," said Axelrod. Google is making inroads selling its technology boxed to web companies and other enterprises, driven somewhat by its strong online brand, and Fast wants to have similar successes.

Being known as the web's largest search engine would be a good start to building a brand, although it seems unlikely Google will take this snub lying down. Spokespeople wouldn't comment on plans to expand the Google index by more than 20 million, to leapfrog Fast, though Fast seems to be expecting it. When WiseNut Inc claimed to beat Google last year, Google quickly updated its claims to remain the largest

In the battle to sell search technology into enterprises, which is where the bulk of the revenue opportunity has been since the portal market collapsed, yesterday Google announced enhancements to its GB-1001 search appliance. The company said that both the 1001 and the GB-8008 have been selling well into enterprises. Recent customers include National Semiconductor, the University of Florida, Cisco, Boeing and PBS.

The GB-1001, targeted at branch offices and smaller businesses, has been upgraded to scale to 300,000 documents, a spokesperson said. A key new feature allows the appliance to index documents addressable via dynamic URLs and cookies, such as those found in personalized or e-commerce content.

Fast has been upgrading its web site, although many of the new features AllTheWeb contains have been used at Google for some time, such as the ability to do full-text searches on PDF documents. One feature Google doesn't have yet, but which has been championed by smaller rivals such as Ask Jeeves Inc's Teoma.com and LookSmart Ltd's WiseNut.com, is the ability to dynamically categorize content.

AllTheWeb now suggests up to four possible categories when a user makes a query. Words associated with the query are suggested based on data scraped from AllTheWeb's query logs. For example, if your keyword is Saturn, AllTheWeb may suggest "Sega Saturn" or "Planet Saturn" or "Saturn Car", depending on what previous searchers have entered.

© ComputerWire.

Providing a secure and efficient Helpdesk

More from The Register

next story
Doctor Who's Flatline: Cool monsters, yes, but utterly limp subplots
We know what the Doctor does, stop going on about it already
Facebook, Apple: LADIES! Why not FREEZE your EGGS? It's on the company!
No biological clockwatching when you work in Silicon Valley
'Cowardly, venomous trolls' threatened with TWO-YEAR sentences for menacing posts
UK government: 'Taking a stand against a baying cyber-mob'
Happiness economics is bollocks. Oh, UK.gov just adopted it? Er ...
Opportunity doesn't knock; it costs us instead
The 'fun-nification' of computer education – good idea?
Compulsory code schools, luvvies love it, but what about Maths and Physics?
Ex-US Navy fighter pilot MIT prof: Drones beat humans - I should know
'Missy' Cummings on UAVs, smartcars and dying from boredom
Sysadmin with EBOLA? Gartner's issued advice to debug your biz
Start hoarding cleaning supplies, analyst firm says, and assume your team will scatter
Don't bother telling people if you lose their data, say Euro bods
You read that right – with the proviso that it's encrypted
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Win a year’s supply of chocolate
There is no techie angle to this competition so we're not going to pretend there is, but everyone loves chocolate so who cares.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.