Sun packs 150 billion web pages into meat locker
Getting your arms around the internet
Sun follows Google
But as Page reached for a patent on the idea, Kahle's brainstorm sparked two other minds over at Sun Microsystems. In the past, Sun has said that its Modular Datacenter - originally code-named Project Blackbox - grew out of a discussion between Sun chief technology officer Greg Papadopoulos and Danny Hills, now co-chairman and chief technology officer of a California consulting operation called Applied Minds. But this morning, Papadopoulus acknowledged that project sprung from Kahle, whom he had worked with at the Cambridge supercomputer maker Thinking Machines.
Blackbox racks are tracked. And cables too
"Danny Hills and I developed the original concept for Project Blackbox, but our inspiration was Brewster - the first person we know of to say 'Hey, we should put a bunch of circuit boards in a shipping container and blow cold air over them,'" Padadopoulus told the gathered digerati this morning in the heart of California's Silicon Valley.
Padadopoulus and company officially announced the Sun Modular Datacenter, or Sun MD, in January 2008. And according to Jud Cooley, the project's director of engineering, Sun has shipped its shipping containers "in the low double digits" to operations as far flung as the Radboud University Nijmegen Medical Centre in the Netherlands and the Belgian wind turbine outfit Hansen Transmissions.
Blackbox fire suppression
And now it's hosting the Wayback Machine in a container tucked between the Spanish tile roofs of its Santa Clara campus, just down the road from Google. Measuring 20 feet longer by 8 feet deep by 8 feet high, the modular net history holds two petabytes of data - with space for another two.
Sun's cramped container includes eight server racks on sliding tracks, each racking nine Sun Fire x4500 "Thumper" servers running Solaris 10 and Sun's ZFS file system. And the necessary networking, power, cooling, and fire-fighting hardware is packed in as well. All it needs from the outside world is a power source (25kW per rack) and a cooling-fluid hook-up (ordinary tap water).
As you walk into the container, with the fans whirring and the racks tight on either side, you feel as if you've walked into a meat locker. Though it's slightly warmer. And it smells better. And you know it's crunching data. Holding 2 quadrillion characters of information, the Wayback Machine processes 500 queries per second, and it's growing at a rate of four billion data rows per month.
In case of earthquake, spring-mounts - but snug the bolts, lads
The rub is that this particular shipping container won't be shipped. The Wayback Machine will live at Sun forever - or least until IBM buys the company and pulls the plug. But the 20-foot container is another step towards Kahle's dream of a digital Alexandria capable of surviving a Caesarean fire - and most any other earthly disaster.
"Even if this first data center never moves, it encapsulates engineering efforts in a building that's reproducible," Kahle told us. "It's something that's centrally manufacturable and shippable."
Meanwhile, Google has built an internet archive of its own. "They're storing more than they let on," Kahle says. But the aims of Google's modular data center project are, shall we say, more commercial. ®
Photos and additional reporting by Rik Myslewski
Sponsored: Flash storage buyer's guide