Feeds

MS, Chinese boffins team up on Kinect-powered sign language recog

Big step forward for deaf and hard-of-hearing gamers

Remote control for virtualized desktops

Microsoft researchers are working with Chinese boffins to tap the XBox 360 Kinect motion sensor's body-tracking capabilities to provide sign language recognition capabilities for the deaf and hard-of-hearing.

Members of the Microsoft Research Asia team have produced a new paper (PDF, 2 pages, abstract) - Sign Language Recognition and Translation with Kinect - co-authored by researchers at the Institute of Computing Technology at the Chinese Academy of Sciences (CAS).

It explains that although sign language recognition has been researched for many years, it has been difficult to achieve accurately in non-control conditions, where background and lighting can vary considerably.

Special “data gloves” and web cams have also been required in previous attempts to make the technology work, but are “too expensive to popularise”, according to the paper.

Luckily for them, Kinect came along, as explained here:

Fortunately, Kinect is able to provide depth and colour data simultaneously, based on which the hand and body action can be tracked more accurate and easier. Therefore, 3D motion trajectory of each sign language vocabulary is aligned and matched between probe and gallery to get the recognised result.

First, the 3D trajectory of the signer’s hands is mapped thanks to a Kinect Windows SDK, then a resampling is done to get a normalised trajectory, taking account of differences in hand speed.

The result is then scored and matched with the closest result from a list of “gallery trajectories”.

The boffins have used their 3D trajectory matching algorithm to build a sign language recognition and translation system with two functions: translating sign language into text or speech, and translating text into sign language through a computer-generated avatar.

The Redmond PR machine has not been the best in publicising the firm’s cutting edge research and development in recent years, allowing Google and Apple to steal Microsoft’s thunder as the most innovative technology giants in Silicon Valley.

However, work like this with the potential for transforming people’s lives in a meaningful way, not just by making their smartphone apps easier to find, could help redress the balance.

For the record, the project was shown off at Microsoft’s Faculty Summit 2013 – an annual meeting bringing together 400 academics with Microsoft bods to discuss research insights.

®

Remote control for virtualized desktops

Whitepapers

Free virtual appliance for wire data analytics
The ExtraHop Discovery Edition is a free virtual appliance will help you to discover the performance of your applications across the network, web, VDI, database, and storage tiers.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
How to determine if cloud backup is right for your servers
Two key factors, technical feasibility and TCO economics, that backup and IT operations managers should consider when assessing cloud backup.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Business security measures using SSL
Examines the major types of threats to information security that businesses face today and the techniques for mitigating those threats.