Feeds

'The most ambitious project at eBay for a long, long time'

Inside the auction leviathan's search megaproject

Maximizing your infrastructure through virtualization

"It's one of the most ambitious projects that's been undertaken at eBay for a very, very long time," vice president of search and experience technology Hugh Williams tells us on the phone. "I'd argue it's on the scale of any major search engine re-write a company has done."

That's a meaty claim. eBay is one of the biggest names in tech, a survivor of the late 1990s dot-com bubble which has come to dominate the market for online auctions.

Not only that, but this isn't the first mega-search project Williams has worked on. Two years ago Williams was an actor in possibly the internet's greatest comeback story – the jury's still out - building Microsoft's Bing. Williams was a development manager working on Microsoft's multi-year, multi-million-dollar effort to steal Google's crown as web number 1.

And you don't need to be a search algorithm nerd to appreciate Williams' Bing work: it's right in front of your eyes. He introduced endless scrolling and playing video thumbnails, breaking new ground and winning a patent with others in the process.

Williams' eBay project is a complete rewrite of the piece of software that's been hardwired into eBay's annual $9.2bn business for nearly 10 years: its search engine.

Williams' vision for Cassini is a 'world class' search engine. He wants something more intuitive for eBay's 97 million active users, something that is itself smarter

eBay claims nearly 100 million active users with global search in 41 markets; more than half of all purchases made on eBay by users in those 41 markets start with the simple act of the customer conducting a search query. The rest are prompted to bid through things like email alerts. There are "thousands" of search queries conducted every second on more than 200 million live listings in eBay's inventory – 10 million are added or deleted daily.

Announced in October last year, the search-engine project – codenamed Cassini – is due to enter A/B testing in late summer with plans for full launch in 2012, Williams told me during a recent interview. I spoke to Williams before eBay pressed the button on testing.

Cassini will replace eBay's current search engine, called Voyager – yes eBay's techies have a slight NASA fixation – that's knocking on for 10 years old, and which Williams believes has been surpassed by many advances in the world of search technology in recent years.

Williams' vision for Cassini is a "world class" search engine. He wants something more intuitive for eBay's nearly 97 million active users, something that is itself smarter: something capable of understanding precisely what it is that users want and doesn't shower them with heaps of tenuous results.

eBay also wants something which that is easier and cheaper for its IT department to run and which can be easily extended without the need for another massive reset in 10 years' time.

SPARC-ing out

So big is the Cassini project, it has sucked in a quarter of the 400 eBay engineers under Williams from three of his teams: search back-end, search science, and search front-end. Williams says these are three of eBay's "largest and deepest" technology teams. They are building Cassini using thousands of lines of C++ and Java code, and massively expanding eBay's existing use of the open-source HBase and Hadoop to crunch terabytes of data.

Search currently runs on "thousands" of SPARC servers in eBay distributed data centres around the world but, for added excitement, eBay plans to uproot from SPARC and head for "commodity servers" – industry shorthand for cheaper Intel or AMD x86 hardware.

The enormity of the task is made all the more remarkable when you consider Williams could be considered to be relatively new to leading such big, battle-ready, commercial software projects.

The Power of One eBook: Top reasons to choose HP BladeSystem

More from The Register

next story
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
SHOCK and AWS: The fall of Amazon's deflationary cloud
Just as Jeff Bezos did to books and CDs, Amazon's rivals are now doing to it
BlackBerry: Toss the server, mate... BES is in the CLOUD now
BlackBerry Enterprise Services takes aim at SMEs - but there's a catch
The triumph of VVOL: Everyone's jumping into bed with VMware
'Bandwagon'? Yes, we're on it and so what, say big dogs
Carbon tax repeal won't see data centre operators cut prices
Rackspace says electricity isn't a major cost, Equinix promises 'no levy'
Disaster Recovery upstart joins DR 'as a service' gang
Quorum joins the aaS crowd with DRaaS offering
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Securing Web Applications Made Simple and Scalable
Learn how automated security testing can provide a simple and scalable way to protect your web applications.