Sun showcased a selection of its research efforts at the Computer Science Museum in Mountain View last week. Established by Ivan and Bert Sutherland and Bob Sproull in 1990, Sun Labs has an enviable reputation for pursuing difficult problems, and investing in long-term research.

Serious computer research has been under pressure in recent years from two areas, finance and the media. In Wall Street's short-sighted view, systems companies have no business doing systems research, which is adequately covered by Intel, Microsoft and favored start-ups. But research has also been under pressure from what we might call 'Disneyfication", which reflects the media's obsession with novelty and a belief that technology works well enough already and can only get better.

This approach has been exemplified by the privately-funded MIT Media Lab, which only recently showcased Clocky - a shagpile alarm clock that hides. Deep problems will be solved by advances in hardware or bandwidth capacity, which really passes the buck over to the materials scientists responsible for the annual advances in microprocessor speed and bandwidth, but not much more. Underpinning this is a belief that technology, which already works fantastically well, is unveiling deep epistemological truths to us, if we are only prepared to stare at the patterns long enough. Meanwhile, the hard business involved in making systems work reliably is wished away.

Then there are researchers who don't flinch from the hard stuff. Computer science is very young, and very many difficult problems remain to be solved. We can barely trust our systems on a daily basis, let alone trust them to preserve our culture over a decade, or for several generations. From an ecological standpoint too, today's technology is grossly inefficient. Even techno utopians such as Buckminster Fuller were concerned by the scarcity of materials, and designed accordingly, but this has yet to become a factor in computer science discussions. However this school of researchers believe that for technology to become widely accepted by the public it must be trusted, and such problems won't solve themselves, or be solved by ever-faster hardware. If you believe that technology can play a part in solving our problems, then only the second approach has any enduring validity.

Over 15 years, Sun Labs has been a terrific example of the latter, and only now is the first school in the ascendent, as we'll discover.

For your reporter, the highlight of the dozens of projects showcased was Celeste. When the trade press is marveling at Google's "global operating system", where data is available from any client system on the planet, it's worth a moment to discuss how difficult this is to implement. Now make it several orders of magnitude harder. Google has an easy task, as it manages its own private network - but what if the public had to maintain such a system, using the computers and bandwidth available to us? How could we trust this to deliver our data securely? Google's services are largely read only - and yet a 'global' file system must be able to support read write access.

Celeste's researchers tackle these problems head on by assuming that bandwidth cannot be guaranteed and nodes are likely to be disconnected. The small team builds on earlier explorations with storage on distributed P2P networks [PDF] and like the P2P networks (it builds on the researcher's earlier work, Celeste uses distributed hash tables to create a global identifier. From 16 nodes, we discovered, Celeste can reconstruct your original file.

It's something the newer generation of web startups such as the aptly-named Flickr photo service forget. (Flickr only flickrs into life when a goat has been sacrificed).

Sun's HPCS work, partially sponsored by DARPA, is an umbrella for over 20 individual related research projects [PDF presentation] [ MP3, is intended to lead to real high-performance petascale systems - a NUMA machine that looks like an SMP with hundreds of thousands of processors - by around 2010.

Several groups are exploring processors connected without wires, using capacitive coupling, separated only by a 10 nanometer gap. A system of this scale and density raises new management problems, which are also being explored in several ways. Sun stands in good stead here. Work on the Continuous System Telemetry Harness, which gathers feedback from hardware - the temperature, currents and voltages of the systems - to anticipate problems using statistical analysis techniques only previously applied to nuclear power plants. Some of this has already been folded into Solaris.

Floating point flops

Intervals is another pure research project that addresses a serious shortcoming in computer systems: floating point arithmetic is accurate only to a few decimal places. Whether the programmer is calculating 1/2 or 22/7, today's systems return the same amount of accuracy information: none at all. Researcher Bill Walster cites the explosion of the Ariane 5 rocket, and Patriot Missile failures as costly examples of floating point maths errors. The Ariane blew up because the software threw an exception converting a 64 point floating point number to a 16 bit integer. Interval arithmetic is only possible because of the computation we now have available, as it calculates all possible values between two bounds. For it to be adopted, it must become an intrinsic data type, and Bill Walster told us, it already is in Sun's Fortran. Perhaps it will take a new language such as Guy Steele's Fortress to make this accepted practice. Another part of the DARPA HPCS work, Fortress is designed for large scientific applications.

Sun's Virsona identity project has morphed into both a larger prototype application OfficeCentral, and a new project Venice. OfficeCentral is intended to provide a virtual water cooler for organizations (such as Sun itself) where hot desking has resulted in a dispersed workforce and soulless office environment. The Virsona principle - that you own all the data about you - is undoubtedly the correct one. Unfortunately, the identity debate has been driven largely by the banks and financial interests, who are foisting a "federated" model onto their customers, and governments (such as the Blair administration), who are trying to drive an open-ended ID card onto the public. But Venice also looks at issues that social networks duck, like security, and puts issues like "trust" and "identity" into context. Without "valuation", Seth Proctor told us, "trust" and "identity" are meaningless.

Sun's content retrieval research these days is focussed on machine classification. Taxonomies are all the rage, but it's unreasonable to expect anyone other than anal retentives to tag their work as they go along, so machines must play a part if it is to be useful. Sun is looking to improve on the abysmal state of machine classification in which computers can't tell that VW and Volkswagen, for example, are the same thing.

The team tells us that bad feature selection is to blame. Working from a basic taxonomy (a dog is a mammal, for example), the software reduces a 90,000 dimensional space to a more manageable 300 dimensional space. The software can autoclassify, assign multiple classifications to a document and most refreshingly, we noticed, simply refuse to classify a document at all if it fails the criteria.

Researcher Paul Lamere has done some impressive pattern matching with his Search Inside The Music project, which aims to improve on iTunes' lamentable Party Shuffle and create a listenable playlist. Party Shuffle is notoriously bad - nothing clears the room faster than its inappropriate ham-fisted choice of music selections. But like DRM for media, SitM struck your reporter as a solution looking for a problem. The more machines make choices for us, the more we see how inadequate these choices are, and contrary to what pessimists believe, the past decade has seen subjectivity and expertise become more valued. The psychological insight necessary to produce a divine playlist doesn't come from an algorithm, or even an encyclopedic knowledge of music, but from experience and understanding.

But when the fashionable western view of the human mind these days is as a kind of inconsequential device driver layer, research like this is sure to be steered into such dead ends.

Thwarting the RoTM - with a boiled egg

For the first time at a Sun Labs day, the floor was invaded by robots. Sensors are the hot new research area at Sun, and with an explosion of cheap, connected devices predicted, it's a good bet. But the work seemed haphazard, and singularly failed to impress the visitors. (For the Japanese, simple robots are not exactly new). One American visitor scathingly told your reporter, "this is the kind of thing I was doing when I was five." And we even managed to flummox a Sun sensor-enabled robot using only a hard boiled egg.

Unfortunately, technical difficulties prevented us from recording this temporary setback for the robot - but here is the egg:

Seeking what deep philosophical thinking informed the sensor work, we rushed to one talk that offered to explain the "magic" at work, only to catch the very end of the presentation. The speaker talked of reinventing TV by putting the web, the chat room and email into a high definition TV, with "... Sun sensors which are wearable, that track body telematics that allow Magic Wands, that have Bots that have presence, and connection into the virtual. Television at that point becomes much more like a renaissance fair!"

And then we noticed that a member of the audience, who subsequently turned out to be from a "Cyber clothing" company, was dressed in a medieval jousting suit - as if he'd stepped out from a game of Dungeons and Dragons. Clearly the Rise of the Machines™ is more pervasive than we thought, and we must investigate further. Even at Sun Labs these days, the serious work wrestles for attention with the fatuous. ®

Related stories

RoTM™ - The Rise of the Machines
MIT invents computer that runs away
Hoax paper fools cybernetic boffins
Vaulting into a Rapturous techno-future with Jaron Lanier
Email destroys the mind faster than marijuana - study
Berkeley to lead $19m cybersecurity research gig
Rise of the man-eating cyberloo
Rise of the botnets