Feeds

'The most ambitious project at eBay for a long, long time'

Inside the auction leviathan's search megaproject

The essential guide to IT transformation

"It's one of the most ambitious projects that's been undertaken at eBay for a very, very long time," vice president of search and experience technology Hugh Williams tells us on the phone. "I'd argue it's on the scale of any major search engine re-write a company has done."

That's a meaty claim. eBay is one of the biggest names in tech, a survivor of the late 1990s dot-com bubble which has come to dominate the market for online auctions.

Not only that, but this isn't the first mega-search project Williams has worked on. Two years ago Williams was an actor in possibly the internet's greatest comeback story – the jury's still out - building Microsoft's Bing. Williams was a development manager working on Microsoft's multi-year, multi-million-dollar effort to steal Google's crown as web number 1.

And you don't need to be a search algorithm nerd to appreciate Williams' Bing work: it's right in front of your eyes. He introduced endless scrolling and playing video thumbnails, breaking new ground and winning a patent with others in the process.

Williams' eBay project is a complete rewrite of the piece of software that's been hardwired into eBay's annual $9.2bn business for nearly 10 years: its search engine.

Williams' vision for Cassini is a 'world class' search engine. He wants something more intuitive for eBay's 97 million active users, something that is itself smarter

eBay claims nearly 100 million active users with global search in 41 markets; more than half of all purchases made on eBay by users in those 41 markets start with the simple act of the customer conducting a search query. The rest are prompted to bid through things like email alerts. There are "thousands" of search queries conducted every second on more than 200 million live listings in eBay's inventory – 10 million are added or deleted daily.

Announced in October last year, the search-engine project – codenamed Cassini – is due to enter A/B testing in late summer with plans for full launch in 2012, Williams told me during a recent interview. I spoke to Williams before eBay pressed the button on testing.

Cassini will replace eBay's current search engine, called Voyager – yes eBay's techies have a slight NASA fixation – that's knocking on for 10 years old, and which Williams believes has been surpassed by many advances in the world of search technology in recent years.

Williams' vision for Cassini is a "world class" search engine. He wants something more intuitive for eBay's nearly 97 million active users, something that is itself smarter: something capable of understanding precisely what it is that users want and doesn't shower them with heaps of tenuous results.

eBay also wants something which that is easier and cheaper for its IT department to run and which can be easily extended without the need for another massive reset in 10 years' time.

SPARC-ing out

So big is the Cassini project, it has sucked in a quarter of the 400 eBay engineers under Williams from three of his teams: search back-end, search science, and search front-end. Williams says these are three of eBay's "largest and deepest" technology teams. They are building Cassini using thousands of lines of C++ and Java code, and massively expanding eBay's existing use of the open-source HBase and Hadoop to crunch terabytes of data.

Search currently runs on "thousands" of SPARC servers in eBay distributed data centres around the world but, for added excitement, eBay plans to uproot from SPARC and head for "commodity servers" – industry shorthand for cheaper Intel or AMD x86 hardware.

The enormity of the task is made all the more remarkable when you consider Williams could be considered to be relatively new to leading such big, battle-ready, commercial software projects.

Boost IT visibility and business value

More from The Register

next story
Pay to play: The hidden cost of software defined everything
Enter credit card details if you want that system you bought to actually be useful
Shoot-em-up: Sony Online Entertainment hit by 'large scale DDoS attack'
Games disrupted as firm struggles to control network
HP busts out new ProLiant Gen9 servers
Think those are cool? Wait till you get a load of our racks
Silicon Valley jolted by magnitude 6.1 quake – its biggest in 25 years
Did the earth move for you at VMworld – oh, OK. It just did. A lot
VMware's high-wire balancing act: EVO might drag us ALL down
Get it right, EMC, or there'll be STORAGE CIVIL WAR. Mark my words
Forrester says it's time to give up on physical storage arrays
The physical/virtual storage tipping point may just have arrived
prev story

Whitepapers

Top 10 endpoint backup mistakes
Avoid the ten endpoint backup mistakes to ensure that your critical corporate data is protected and end user productivity is improved.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Backing up distributed data
Eliminating the redundant use of bandwidth and storage capacity and application consolidation in the modern data center.
The essential guide to IT transformation
ServiceNow discusses three IT transformations that can help CIOs automate IT services to transform IT and the enterprise
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.