Inside Sun Labs - the best and the 'bots

Sun showcased a selection of its research efforts at the Computer Science Museum in Mountain View last week. Established by Ivan and Bert Sutherland and Bob Sproull in 1990, Sun Labs has an enviable reputation for pursuing difficult problems, and investing in long-term research.

Serious computer research has been under pressure in recent years from two areas, finance and the media. In Wall Street's short-sighted view, systems companies have no business doing systems research, which is adequately covered by Intel, Microsoft and favored start-ups. But research has also been under pressure from what we might call 'Disneyfication", which reflects the media's obsession with novelty and a belief that technology works well enough already and can only get better.

This approach has been exemplified by the privately-funded MIT Media Lab, which only recently showcased Clocky - a shagpile alarm clock that hides. Deep problems will be solved by advances in hardware or bandwidth capacity, which really passes the buck over to the materials scientists responsible for the annual advances in microprocessor speed and bandwidth, but not much more. Underpinning this is a belief that technology, which already works fantastically well, is unveiling deep epistemological truths to us, if we are only prepared to stare at the patterns long enough. Meanwhile, the hard business involved in making systems work reliably is wished away.

Then there are researchers who don't flinch from the hard stuff. Computer science is very young, and very many difficult problems remain to be solved. We can barely trust our systems on a daily basis, let alone trust them to preserve our culture over a decade, or for several generations. From an ecological standpoint too, today's technology is grossly inefficient. Even techno utopians such as Buckminster Fuller were concerned by the scarcity of materials, and designed accordingly, but this has yet to become a factor in computer science discussions. However this school of researchers believe that for technology to become widely accepted by the public it must be trusted, and such problems won't solve themselves, or be solved by ever-faster hardware. If you believe that technology can play a part in solving our problems, then only the second approach has any enduring validity.

Over 15 years, Sun Labs has been a terrific example of the latter, and only now is the first school in the ascendent, as we'll discover.

For your reporter, the highlight of the dozens of projects showcased was Celeste. When the trade press is marveling at Google's "global operating system", where data is available from any client system on the planet, it's worth a moment to discuss how difficult this is to implement. Now make it several orders of magnitude harder. Google has an easy task, as it manages its own private network - but what if the public had to maintain such a system, using the computers and bandwidth available to us? How could we trust this to deliver our data securely? Google's services are largely read only - and yet a 'global' file system must be able to support read write access.

Celeste's researchers tackle these problems head on by assuming that bandwidth cannot be guaranteed and nodes are likely to be disconnected. The small team builds on earlier explorations with storage on distributed P2P networks [PDF] and like the P2P networks (it builds on the researcher's earlier work, Celeste uses distributed hash tables to create a global identifier. From 16 nodes, we discovered, Celeste can reconstruct your original file.

It's something the newer generation of web startups such as the aptly-named Flickr photo service forget. (Flickr only flickrs into life when a goat has been sacrificed).

Sun's HPCS work, partially sponsored by DARPA, is an umbrella for over 20 individual related research projects [PDF presentation] [ MP3, is intended to lead to real high-performance petascale systems - a NUMA machine that looks like an SMP with hundreds of thousands of processors - by around 2010.

Several groups are exploring processors connected without wires, using capacitive coupling, separated only by a 10 nanometer gap. A system of this scale and density raises new management problems, which are also being explored in several ways. Sun stands in good stead here. Work on the Continuous System Telemetry Harness, which gathers feedback from hardware - the temperature, currents and voltages of the systems - to anticipate problems using statistical analysis techniques only previously applied to nuclear power plants. Some of this has already been folded into Solaris.