Free ESG report : Seamless data management with Avere FXT

Microsoft researchers are working with Chinese boffins to tap the XBox 360 Kinect motion sensor's body-tracking capabilities to provide sign language recognition capabilities for the deaf and hard-of-hearing.

Members of the Microsoft Research Asia team have produced a new paper (PDF, 2 pages, abstract) - Sign Language Recognition and Translation with Kinect - co-authored by researchers at the Institute of Computing Technology at the Chinese Academy of Sciences (CAS).

It explains that although sign language recognition has been researched for many years, it has been difficult to achieve accurately in non-control conditions, where background and lighting can vary considerably.

Special “data gloves” and web cams have also been required in previous attempts to make the technology work, but are “too expensive to popularise”, according to the paper.

Luckily for them, Kinect came along, as explained here:

Fortunately, Kinect is able to provide depth and colour data simultaneously, based on which the hand and body action can be tracked more accurate and easier. Therefore, 3D motion trajectory of each sign language vocabulary is aligned and matched between probe and gallery to get the recognised result.

First, the 3D trajectory of the signer’s hands is mapped thanks to a Kinect Windows SDK, then a resampling is done to get a normalised trajectory, taking account of differences in hand speed.

The result is then scored and matched with the closest result from a list of “gallery trajectories”.

The boffins have used their 3D trajectory matching algorithm to build a sign language recognition and translation system with two functions: translating sign language into text or speech, and translating text into sign language through a computer-generated avatar.

The Redmond PR machine has not been the best in publicising the firm’s cutting edge research and development in recent years, allowing Google and Apple to steal Microsoft’s thunder as the most innovative technology giants in Silicon Valley.

However, work like this with the potential for transforming people’s lives in a meaningful way, not just by making their smartphone apps easier to find, could help redress the balance.

For the record, the project was shown off at Microsoft’s Faculty Summit 2013 – an annual meeting bringing together 400 academics with Microsoft bods to discuss research insights.

®

5 ways to prepare your advertising infrastructure for disaster