Feeds

'The most ambitious project at eBay for a long, long time'

Inside the auction leviathan's search megaproject

Next gen security for virtualised datacentres

"It's one of the most ambitious projects that's been undertaken at eBay for a very, very long time," vice president of search and experience technology Hugh Williams tells us on the phone. "I'd argue it's on the scale of any major search engine re-write a company has done."

That's a meaty claim. eBay is one of the biggest names in tech, a survivor of the late 1990s dot-com bubble which has come to dominate the market for online auctions.

Not only that, but this isn't the first mega-search project Williams has worked on. Two years ago Williams was an actor in possibly the internet's greatest comeback story – the jury's still out - building Microsoft's Bing. Williams was a development manager working on Microsoft's multi-year, multi-million-dollar effort to steal Google's crown as web number 1.

And you don't need to be a search algorithm nerd to appreciate Williams' Bing work: it's right in front of your eyes. He introduced endless scrolling and playing video thumbnails, breaking new ground and winning a patent with others in the process.

Williams' eBay project is a complete rewrite of the piece of software that's been hardwired into eBay's annual $9.2bn business for nearly 10 years: its search engine.

Williams' vision for Cassini is a 'world class' search engine. He wants something more intuitive for eBay's 97 million active users, something that is itself smarter

eBay claims nearly 100 million active users with global search in 41 markets; more than half of all purchases made on eBay by users in those 41 markets start with the simple act of the customer conducting a search query. The rest are prompted to bid through things like email alerts. There are "thousands" of search queries conducted every second on more than 200 million live listings in eBay's inventory – 10 million are added or deleted daily.

Announced in October last year, the search-engine project – codenamed Cassini – is due to enter A/B testing in late summer with plans for full launch in 2012, Williams told me during a recent interview. I spoke to Williams before eBay pressed the button on testing.

Cassini will replace eBay's current search engine, called Voyager – yes eBay's techies have a slight NASA fixation – that's knocking on for 10 years old, and which Williams believes has been surpassed by many advances in the world of search technology in recent years.

Williams' vision for Cassini is a "world class" search engine. He wants something more intuitive for eBay's nearly 97 million active users, something that is itself smarter: something capable of understanding precisely what it is that users want and doesn't shower them with heaps of tenuous results.

eBay also wants something which that is easier and cheaper for its IT department to run and which can be easily extended without the need for another massive reset in 10 years' time.

SPARC-ing out

So big is the Cassini project, it has sucked in a quarter of the 400 eBay engineers under Williams from three of his teams: search back-end, search science, and search front-end. Williams says these are three of eBay's "largest and deepest" technology teams. They are building Cassini using thousands of lines of C++ and Java code, and massively expanding eBay's existing use of the open-source HBase and Hadoop to crunch terabytes of data.

Search currently runs on "thousands" of SPARC servers in eBay distributed data centres around the world but, for added excitement, eBay plans to uproot from SPARC and head for "commodity servers" – industry shorthand for cheaper Intel or AMD x86 hardware.

The enormity of the task is made all the more remarkable when you consider Williams could be considered to be relatively new to leading such big, battle-ready, commercial software projects.

Gartner critical capabilities for enterprise endpoint backup

More from The Register

next story
The Return of BSOD: Does ANYONE trust Microsoft patches?
Sysadmins, you're either fighting fires or seen as incompetents now
Microsoft: Azure isn't ready for biz-critical apps … yet
Microsoft will move its own IT to the cloud to avoid $200m server bill
Shoot-em-up: Sony Online Entertainment hit by 'large scale DDoS attack'
Games disrupted as firm struggles to control network
Cutting cancer rates: Data, models and a happy ending?
How surgery might be making cancer prognoses worse
Silicon Valley jolted by magnitude 6.1 quake – its biggest in 25 years
Did the earth move for you at VMworld – oh, OK. It just did. A lot
Forrester says it's time to give up on physical storage arrays
The physical/virtual storage tipping point may just have arrived
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
5 things you didn’t know about cloud backup
IT departments are embracing cloud backup, but there’s a lot you need to know before choosing a service provider. Learn all the critical things you need to know.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?