Ensure Ease of Recovery with Asigra’s Agentless Software

"It's one of the most ambitious projects that's been undertaken at eBay for a very, very long time," vice president of search and experience technology Hugh Williams tells us on the phone. "I'd argue it's on the scale of any major search engine re-write a company has done."

That's a meaty claim. eBay is one of the biggest names in tech, a survivor of the late 1990s dot-com bubble which has come to dominate the market for online auctions.

And you don't need to be a search algorithm nerd to appreciate Williams' Bing work: it's right in front of your eyes. He introduced endless scrolling and playing video thumbnails, breaking new ground and winning a patent with others in the process.

Williams' eBay project is a complete rewrite of the piece of software that's been hardwired into eBay's annual $9.2bn business for nearly 10 years: its search engine.

Williams' vision for Cassini is a 'world class' search engine. He wants something more intuitive for eBay's 97 million active users, something that is itself smarter

eBay claims nearly 100 million active users with global search in 41 markets; more than half of all purchases made on eBay by users in those 41 markets start with the simple act of the customer conducting a search query. The rest are prompted to bid through things like email alerts. There are "thousands" of search queries conducted every second on more than 200 million live listings in eBay's inventory – 10 million are added or deleted daily.

Announced in October last year, the search-engine project – codenamed Cassini – is due to enter A/B testing in late summer with plans for full launch in 2012, Williams told me during a recent interview. I spoke to Williams before eBay pressed the button on testing.

Cassini will replace eBay's current search engine, called Voyager – yes eBay's techies have a slight NASA fixation – that's knocking on for 10 years old, and which Williams believes has been surpassed by many advances in the world of search technology in recent years.

Williams' vision for Cassini is a "world class" search engine. He wants something more intuitive for eBay's nearly 97 million active users, something that is itself smarter: something capable of understanding precisely what it is that users want and doesn't shower them with heaps of tenuous results.

eBay also wants something which that is easier and cheaper for its IT department to run and which can be easily extended without the need for another massive reset in 10 years' time.

SPARC-ing out

So big is the Cassini project, it has sucked in a quarter of the 400 eBay engineers under Williams from three of his teams: search back-end, search science, and search front-end. Williams says these are three of eBay's "largest and deepest" technology teams. They are building Cassini using thousands of lines of C++ and Java code, and massively expanding eBay's existing use of the open-source HBase and Hadoop to crunch terabytes of data.

Search currently runs on "thousands" of SPARC servers in eBay distributed data centres around the world but, for added excitement, eBay plans to uproot from SPARC and head for "commodity servers" – industry shorthand for cheaper Intel or AMD x86 hardware.

The enormity of the task is made all the more remarkable when you consider Williams could be considered to be relatively new to leading such big, battle-ready, commercial software projects.

SaaS data loss: The problem you didn’t know you had

Next page: 'A pretty applied guy'