Feeds

Only Kinect: Microsoft boffins build Minority Report-style tools

Big, expensive goggles no longer compulsory for AR

Combat fraud and increase customer satisfaction

In the two years since Microsoft launched Kinect, there’s been huge interest among enthusiasts in adapting the hands-free controller to new settings beyond gaming.

Techies and geeks of open- and closed-source shades have hacked the hardware, used the tools or adapted Kinect to steer computerised, wheeled armchairs.

Among those doing the fiddling is Microsoft Research in Cambridge, the Microsoft division whose work on real-time human pose recognition and depth-sensing algorithms (here and here) helped make Kinect a reality in the first place.

These engineers have been using Kinect in different experiments in an attempt to construct Star Trek-style holodecks and Minority Report-style 3D touch-displays.

They have combined feeds from multiple Kinect cameras positioned around a room, stitching together a 3D map of the room and then combining the map with moving pico projectors. The result has been to combine Kinect’s depth sensing with the pico data so you can see and touch the computer-generated pixels without the need for special glasses, headsets or gloves to operate. Among the achievements so far is a pixel-generated car driving around the floor.

The Kinect sensor has also been combined with mirror and web-camera tracking to produce a holodesk and Kinect sensors used as hand-held units to scan objects in 3D.

Microsoft Research 3D car

Augmented reality car using Kinect, from Microsoft Research

Shahram Izadi, who jointly leads Microsoft Research Cambridge’s interactive 3D technologies group, reckons augmented reality – or what he also calls “looking at pixels in the air without a screen” - will be the “next big thing” thanks to Kinect and other industry efforts. On Izadi’s team is Andrew Fitzgibbon who helped author that original paper on human skeletal 3D recognition for the Kinect.

“We are starting to move towards this notion of augment reality. It’s about augmentation with the instrumentation of the environment, instead of augmentation of the user,” he said on a recent visit to Microsoft in Cambridge.

“We have been working a lot with the Kinect and know the potential,” Izadi told The Reg.

Izadi points to a number of innovations in a relatively short period of time, including Kinect. We've also had multi-touch motion gaming and augmented reality in smartphones from Apple and Google.

“Companies like Microsoft will be looking to exploit it,” Izadi predicted.

Izadi reckons on reason augmented reality is about to step from science fiction to fact is the relatively ubiquity of the Kinect. The Kinect sold so many units during its first 60 says (eight million) that it set a Guinness World record. The Kinect's companion Xbox 360 game unit – it has since been made available for the PC, too – is also now the biggest selling games console in the US.

Another factor on the Kinect’s side is its price point: £129 from the Microsoft online store in the UK. This makes getting your hands on the Kinect doable - which is not the case with most augmented-realty kit. “Kinect is just a starting point,” Izadi says.

“The reason I’m so excited is two years ago there was nothing like this on the market and it cost tends of thousands to buy [the equipment]. Now we are talking about natural language user interface and augmenter reality as if they are almost upon us, and two years ago we were far away form that vision.”

Izadi’s team has been capitalising on Kinect’s depth-sensing technology to meet the vision. The depth-sensing camera can tell the system how far an object is from the sensor. The Kinect is a rich source of 3D data and it is mostly this information that is being interpreted. An image is smaller than one you’ll find in most digital cameras, 640 x 480 pixels, but the Kinect manages to track 300,000 points at a rate of 30 frames per second.

But there are niggles, hence the work’s still at the prototype stage.

One problem is that the various Kinects interfere with each other when set up and running at the same time, a fact that creates visual noise on the display. This is something Microsoft Research realised could be solved for now with the addition of a little vibration to each unit; the vibes are supplied by a small motor fixed to the top of each Kinect in the holodeck idea (here and here).

Also, sensor readings can be incorrect, with 3m being read as 2.9 meters for example, or they can fluctuate. In other scenarios depth data can’t be scanned, for example in the case of the Dell logo on a Dell PC box.

Izadi also reckons the next challenge is not input - although issues still remain over the input received via the Kinect – but rather the lag in output technologies, for the moment filled by pico projectors.

When will holodecks and holodesks become a reality?

Despite his enthusiasm, Izadi points to the fact the work at Microsoft Research tends doesn’t pay off immediately – it’s generally a good decade before something gets turned into a product. Kinect was quicker, of course. It was first discussed in 2007 and officially unveiled by Microsoft at E3 in June 2009 under the codename Project Natal.

In the meantime, the pace is being pushed on tablets and smartphones thanks to companies like Layar and Qualcomm, which lets you point your phone at a real-world object and receive information, with the phone acting as a head-up-display.

“Many companies are now developing augmented reality or immersive technologies,” Izadi says, "and that will accelerate a lot of what we are seeing. But to add the disclaimer: we are a search group and the work we are doing is five to 10 years out on what could feasibly be a product... [however] there is this real acceleration effect in we are seeing." ®

SANS - Survey on application security programs

More from The Register

next story
Next Windows obsolescence panic is 450 days from … NOW!
The clock is ticking louder for Windows Server 2003 R2 users
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Ditch the sync, paddle in the Streem: Upstart offers syncless sharing
Upload, delete and carry on sharing afterwards?
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
Batten down the hatches, Ubuntu 14.04 LTS due in TWO DAYS
Admins dab straining server brows in advance of Trusty Tahr's long-term support landing
Red Hat to ship RHEL 7 release candidate with a taste of container tech
Grab 'near-final' version of next Enterprise Linux next week
Windows 8.1, which you probably haven't upgraded to yet, ALREADY OBSOLETE
Pre-Update versions of new Windows version will no longer support patches
Inside the Hekaton: SQL Server 2014's database engine deconstructed
Nadella's database sqares the circle of cheap memory vs speed
prev story

Whitepapers

Mainstay ROI - Does application security pay?
In this whitepaper learn how you and your enterprise might benefit from better software security.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.