Feeds

MS, Chinese boffins team up on Kinect-powered sign language recog

Big step forward for deaf and hard-of-hearing gamers

Boost IT visibility and business value

Microsoft researchers are working with Chinese boffins to tap the XBox 360 Kinect motion sensor's body-tracking capabilities to provide sign language recognition capabilities for the deaf and hard-of-hearing.

Members of the Microsoft Research Asia team have produced a new paper (PDF, 2 pages, abstract) - Sign Language Recognition and Translation with Kinect - co-authored by researchers at the Institute of Computing Technology at the Chinese Academy of Sciences (CAS).

It explains that although sign language recognition has been researched for many years, it has been difficult to achieve accurately in non-control conditions, where background and lighting can vary considerably.

Special “data gloves” and web cams have also been required in previous attempts to make the technology work, but are “too expensive to popularise”, according to the paper.

Luckily for them, Kinect came along, as explained here:

Fortunately, Kinect is able to provide depth and colour data simultaneously, based on which the hand and body action can be tracked more accurate and easier. Therefore, 3D motion trajectory of each sign language vocabulary is aligned and matched between probe and gallery to get the recognised result.

First, the 3D trajectory of the signer’s hands is mapped thanks to a Kinect Windows SDK, then a resampling is done to get a normalised trajectory, taking account of differences in hand speed.

The result is then scored and matched with the closest result from a list of “gallery trajectories”.

The boffins have used their 3D trajectory matching algorithm to build a sign language recognition and translation system with two functions: translating sign language into text or speech, and translating text into sign language through a computer-generated avatar.

The Redmond PR machine has not been the best in publicising the firm’s cutting edge research and development in recent years, allowing Google and Apple to steal Microsoft’s thunder as the most innovative technology giants in Silicon Valley.

However, work like this with the potential for transforming people’s lives in a meaningful way, not just by making their smartphone apps easier to find, could help redress the balance.

For the record, the project was shown off at Microsoft’s Faculty Summit 2013 – an annual meeting bringing together 400 academics with Microsoft bods to discuss research insights.

®

The essential guide to IT transformation

More from The Register

next story
Top Gun display for your CAR: Heads-up fighter pilot tech
Sadly Navdy kit doesn't include Sidewinder missile to blast traffic
FEAST YOUR EYES: Samsung's Galaxy Alpha has an 'entirely new appearance'
Wow, it looks like nothing else on the market, for sure
iPhone 6 flip tip slips in Aussie's clip: Apple's 'reversible USB' leaks
New plug not compatible with official Type-C, according to fresh rumors
YES YES YES! Apple patents mousy, pressure-sensing iVibrator
Fanbois prepare to experience the great Cupertin-O
TV transport tech, part 1: From server to sofa at the touch of a button
You won't believe how much goes into today's telly tech
NVIDIA claims first 64-bit ARMv8 SoC for Androids
Mile-High 'Denver' Tegra K1 successor said to rival PC performance
XBOX One will learn to play media from USB and DLNA sources
Hang on? Aren't those file formats you hardly ever see outside torrents?
Giving your old Tesco Hudl to Auntie June? READ THIS FIRST
You can never wipe supermarket slab clean enough
prev story

Whitepapers

Endpoint data privacy in the cloud is easier than you think
Innovations in encryption and storage resolve issues of data privacy and key requirements for companies to look for in a solution.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Solving today's distributed Big Data backup challenges
Enable IT efficiency and allow a firm to access and reuse corporate information for competitive advantage, ultimately changing business outcomes.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.