Sun packs 150 billion web pages into meat locker

Getting your arms around the internet

By Cade Metz in Santa Clara • Get more from this author

Posted in Servers, 25th March 2009 23:59 GMT

Free whitepaper – Reliability analysis of the APC Symmetra MW Power System

In the beginning...

By all accounts, the notion of a containerized data center originated with Kahle. "The idea of a shipping container came in 2001, out of the absolute frustration of trying to build a data center for the Archive," he told reporters over lunch this afternoon.

Wayback Machine - Jud Cooley, lead engineer

The Project Blackbox director of engineering, Jud Cooley, spies a Thumper

The original thought was that the Internet Archive would ship these containerized copies of its Wayback Machine to strategic spots across the globe - so that Kahle's online database would never suffer the fate of the Library of Alexandria.

"If we could have four or five copies in places around the world and keep them in sync, then as upheavals go up and down and earthquakes happen, we might be able to survive and maintain an insight into our past. If Egypt had made copies of the Library of Alexandria in China or India, we would still have the other works of Aristotle. As it is, we don't."

In theory, you could manufacturer these standardized data centers from a central location and ship them across the planet via trains, planes, and automobiles. And all this would be cheaper - and quicker - than building centers from scratch.

Rack in reverse

By 2003, Kahle colleague Bruce Baumgart was pitching the idea to commercial operations, including IBM, which eventually rejected it. At one point, Kahle remembers, Baumgart delivered his stock presentation - still available here (PDF) - at some sort of west coast hacker's conference. And, yes, Google co-founder Larry Page was in the audience.

Next page: Sun follows Google