Feeds

MS, Chinese boffins team up on Kinect-powered sign language recog

Big step forward for deaf and hard-of-hearing gamers

Application security programs and practises

Microsoft researchers are working with Chinese boffins to tap the XBox 360 Kinect motion sensor's body-tracking capabilities to provide sign language recognition capabilities for the deaf and hard-of-hearing.

Members of the Microsoft Research Asia team have produced a new paper (PDF, 2 pages, abstract) - Sign Language Recognition and Translation with Kinect - co-authored by researchers at the Institute of Computing Technology at the Chinese Academy of Sciences (CAS).

It explains that although sign language recognition has been researched for many years, it has been difficult to achieve accurately in non-control conditions, where background and lighting can vary considerably.

Special “data gloves” and web cams have also been required in previous attempts to make the technology work, but are “too expensive to popularise”, according to the paper.

Luckily for them, Kinect came along, as explained here:

Fortunately, Kinect is able to provide depth and colour data simultaneously, based on which the hand and body action can be tracked more accurate and easier. Therefore, 3D motion trajectory of each sign language vocabulary is aligned and matched between probe and gallery to get the recognised result.

First, the 3D trajectory of the signer’s hands is mapped thanks to a Kinect Windows SDK, then a resampling is done to get a normalised trajectory, taking account of differences in hand speed.

The result is then scored and matched with the closest result from a list of “gallery trajectories”.

The boffins have used their 3D trajectory matching algorithm to build a sign language recognition and translation system with two functions: translating sign language into text or speech, and translating text into sign language through a computer-generated avatar.

The Redmond PR machine has not been the best in publicising the firm’s cutting edge research and development in recent years, allowing Google and Apple to steal Microsoft’s thunder as the most innovative technology giants in Silicon Valley.

However, work like this with the potential for transforming people’s lives in a meaningful way, not just by making their smartphone apps easier to find, could help redress the balance.

For the record, the project was shown off at Microsoft’s Faculty Summit 2013 – an annual meeting bringing together 400 academics with Microsoft bods to discuss research insights.

®

The Power of One Infographic

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Securing Web Applications Made Simple and Scalable
Learn how automated security testing can provide a simple and scalable way to protect your web applications.