Original URL: https://www.theregister.com/2007/06/08/surface_computing_mailbag/

Surface computers: debunking Microsoft and Han

More on the 3D, tabletop UI

By Andrew Fentem

Posted in Personal Tech, 8th June 2007 22:22 GMT

Letters My recent article 'Why Microsoft innovation is only Surface deep' prompted a number of Reg readers to pick up their keyboards in anger. I had anticipated knee-jerk claims of gratuitous "MS bashing", but these aside, the feedback has mostly been supportive.

Gary Herbert perhaps best encapsulated the indignant mood of Reg readers, when he wrote, "I can't believe MS has the balls to call it innovation".

Many readers were also surprised that so many news sources such as The Guardian, Observer, BBC Online News, and many blogs have refused to veer from the mantra of the Microsoft press release, despite so much prior art being only a Google search away.

Microsoft - having its cake, eating its cake

The concluding paragraph did however cause a certain amount of controversy. I wrote:

..."Microsoft said it aimed to produce cheaper versions for homes within three to five years". And despite the sterling work of the likes of Philips et al, Microsoft have also claimed to be "the first major technology company to bring surface computing to market in a commercially ready product". These conflicting statements seem to raise question marks over quite how far Microsoft have actually got...."

Ian Wellock asked, "How on Earth is that conflicting? They raise no question marks about anything, they support each other 100%, and actually seem to be quite truthful regarding timelines, for once." His sentiment was echoed by reader Andy Iles.

Let me explain.

A detailed exegesis of Microsoft press releases does not make particularly good copy, so for the sake of brevity, it all comes down to what Microsoft mean by the word "market". If they mean tiny niche markets like casinos, then their second statement is untrue - there are plenty of companies with similar systems in the pipeline. If they are, as I am sure their shareholders and the media believe, a major technology company producing mass-market products, then the first statement suggests that what they actually have at the moment is a beta prototype that they intend to market test at some point in the future. Again making the second statement problematic.

Remember, the reason why the international media lap up Microsoft press releases is because it is assumed that Microsoft innovations will one day affect every home. Not just the domiciles of gambling addicts.

A reader also pointed out that rear-projection interfaces are not necessarily enormous, and that in fact Sony's new generation of rear-projection TV screens are actually "tiny". Unfortunately, the "tiny" Sony KDF-E42A11E 42-inch rear-projection TV is more than a foot deep and is 25Kg of glass, metal, and electronics. And there's still no room inside for the system of infrared cameras and lenses that you would need to turn the TV into a Surface.

The hard boundaries of physics and geometry prevented the cathode ray tube from evolving into a completely flat and portable technology, and it has been superseded by the LCD. For the same reasons rear-projection devices will suffer a similar fate, and will not "simply mature for use on the desktop" as reader Stuart Smith hopes.

Mr Smith is correct that an important contribution that Microsoft and other developers can make is to adapt their operating systems to enable and make use of multi-touch interaction. These developments are to be very much welcomed as some of us move beyond the constraints of keyboards and mice.

But here, too, there lies another prior art problem for Microsoft.

Around seven years ago the confluence of projector, webcam, and Macromedia Director plugin technologies enabled interaction design students across the world to start building experimental interactive tables. And they haven't stopped. The ubiquity of these interactive tables has led Regine Debatty, the editor of the multi-Webby-winning technology/design/art blog We Make Money Not Art, once a champion of such applications, to all but ban them from her site. Before Microsoft stake claims over software innovations in this particular domain, they may want to sift Regine's archive.

Implementation is not Innovation

The most pervasive myth in the 'multi-touch' arena, a myth ingrained in blogs and the writings of technology pundits who use blogs as their primary sources, is that Jeff Han's work at New York University is the real thing, as far as hardware innovation goes.

"Much of what the Gates coffee table does was being done years ago by Jeff Kan, a researcher at New York University," wrote John Naughton in the Observer.

Mr Han is clearly a talented engineer, but these claims are simply not true.

In the February 2007 edition of Fast Company magazine , Mr Han describes in great detail the process of invention of his system, and how, "Inspiration came in the form of an ordinary glass of water."

Well, this author's 'inspiration' came rather more prosaically - after less than 20 seconds of searching on Espacenet this morning - when I found patent US6061177. In this patent, filed in the US in 1996, a Mr Kenneth Fujimoto describes a multi-touch system consisting of a camera and a projector, mounted behind a panel which exhibits frustrated internal total reflection when touched.

Meanwhile, Mr Han has sat on his hands rather than fess up, or give credit where it's due. On his website he notes that frustrated total internal reflection (FTIR) is "familiar to the biometrics community".

In 2004, during an R&D collaboration with Novation EMS Ltd, I myself created a simple experimental prototype based on the FTIR effect. This involved building a custom digital camera around an LM9630 Image Sensor to enable low resolution but very high speed multi-touch detection, primarily for musical applications.

Microsoft and Han have most likely used higher resolution but off-the-shelf technologies. In the YouTube demo of the Surface, Bill Gates moves his hands fairly slowly and the response is quite sluggish, particularly towards the end of the video. It is more difficult to tell from the Jeff Han videos on YouTube whether or not the latencies within his system would frustrate certain high-speed real-time game and musical applications.

Ben Shneiderman, the colossus of human-computer interaction research, and reportedly not a Han fan, has praised him as a "great showman". And everyone seems to agree that Han is indeed a great implementer and illustrator of ideas, so hats off to him. But unfortunately the US Patent Office doesn't (yet) consider YouTube hits in the patent examination process. (Just give it time).

There is more to innovation than simple implementation. It is easy to throw money at slick demos, but valuable original ideas are harder to come by, and hard to recognise when they do.

Bootnote

Thanks to everyone who took the time to write to me. I was particularly intrigued by the DigitalDash - a tactile car dashboard multitouch interface.

Andrew Fentem has worked in and human-computer interaction research and hardware development for 15 years. His Spaceman Technologies is exhibiting a reconfigurable tactile multi-touch and multi-object tracking interface at the Kinetica Museum, the UK's electronic art museum, in London until 29 June 2007.