Nvidia unveils Minority Report-style face-bot tech

Keeping it real takes two trillion calculations a second

Build a business case: developing custom apps

GTC 2013 Accurately modeling and rendering the ocean is highly complex from a computational perspective – but it’s considerably harder to portray accurately an artificial simulation of the human face, according to Nvidia CEO Jen-Hsun Huang in his opening keynote address at the GTC 2013 conference in San Jose.

Human faces contain a subtlety that’s incredibly difficult to capture. Consider the wide range of skin textures, pore sizes, bumps, humps, colors and other features that all have to interact in a completely natural manner in order for the illusion to hold. And that’s before we even think about modelling beards, goatees, and the like.

Anyone attempting to build a believable human facsimile also has to beware of the “uncanny valley”. This is a school of thought that says human copies which are almost – but not quite – perfectly human can cause big-time revulsion in us real humans.

(I would submit a picture of Carrot Top as proof positive that this is true. I’m sure you can come up with your own examples of real people who have gone over the uncanny cliff into the valley of revulsion. Spend a couple of hours on it today; it’s important, and sort of scientific too.)

Modelling the perfect human face is something that Nvidia has worked on for years. Their best attempt (until recently) was Dawn, the fairy-like gal in the picture at left. This picture doesn’t do her justice; she truly does look realistic. The skin tone and texture is incredibly lifelike and attractive. Her hair moved in exactly the right way as the wind gently blew across her delicate brow. Sure, the bat ears detract a bit from the overall “realness”, but they certainly weren’t a deal-breaker for me.

The illusion fell apart when Dawn started to move. Her bodily motions were somewhat jerky and unnatural, despite all of the technology rendering her and the sophisticated programming that dictated her movements. When her face reflected emotions they were abrupt and, again, just not quite right. She was in that uncanny valley for sure, but still pretty cute.

Ira - a little menacing, but at least he's realistic

The best way to imbue an artificial human with real human movements is, of course, to apply some technology in the form of hardware and advanced software. How is this done? On a light stage, 156 cameras roll video as a genuine human moves around, extracting complete 3D geometry of every move. Then you run the real human through 30 or so different facial expressions, recording every detail from nearly every angle. (Are there really 30 facial expressions? I only have six or so… maybe seven on a good day.)

This exercise yields around 32GB of data, which Faceworks compresses and distills into 400MB that can be used programmatically to generate facial expressions just like those of a a real human. Combining the power of Nvidia's new Titan graphics card and their Faceworks software, their latest attempt – Digital Ira – is orders of magnitude better than Digital Dawn on her best day.

Ira: is it just us or does he look a little ...tetchy?

Pictured here with Huang in the foreground, Ira is damned realistic. Every detail of his face looked natural and accurate to me and everyone else in the room.

When Ira worked through a range of emotions and facial expressions, the movements were completely natural, if a little hurried (which was due to the speed of the demo more than anything else, I think). The details were "just right", and the transitions from emotion to emotion were smooth and, again, natural.

But Ira really jumped the chasm from digital Vin Diesel to real deal when he started talking. His speech was fast-paced yet completely realistic. He was a little upset that his morning fruit yoghurt had too much fruit and not enough yoghurt, but that’s something all of us have raged about at some point or another.

It takes some power to make Ira so damned believable. He’s rendered on Nvidia's newest Titan graphics card, a 2,600-core behemoth sporting 7.1 billion transistors which is, according to Huang, the most complex semiconductor device ever manufactured. On the software side, there’s an 8,000-instruction program that articulates Ira’s geometry.

So how much compute power does it take for Ira to keep it real? Quite a bit. Here’s some of the math. First, if you figure that each instruction uses five floating point operations, we’re looking at 40,000 operations for each pixel that undergoes a change.

Assuming that he’s changing half the screen at a time, and then multiplying by the refresh rate of 60Hz, you come to the conclusion that Ira needs around 2 Teraflops to sustain his amazingly lifelike façade.

And the effect is truly amazing. He was animated, but not overly so; he was real. So how is this going to be used in the real world? I bet that Hollywood will definitely be interested. Huang discussed an idea about using this technology for virtual meetings, but I’m not so sure I see that as an early use.

On second thought, if Ira were to video-call me and demand I send him some money, there’s a pretty good chance I’d pay up. ®

Build a business case: developing custom apps

More from The Register

next story
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
KDE releases ice-cream coloured Plasma 5 just in time for summer
Melty but refreshing - popular rival to Mint's Cinnamon's still a work in progress
Leaked Windows Phone 8.1 Update specs tease details of Nokia's next mobes
New screen sizes, dual SIMs, voice over LTE, and more
Secure microkernel that uses maths to be 'bug free' goes open source
Hacker-repelling, drone-protecting code will soon be yours to tweak as you see fit
Mozilla keeps its Beard, hopes anti-gay marriage troubles are now over
Plenty on new CEO's todo list – starting with Firefox's slipping grasp
Apple: We'll unleash OS X Yosemite beta on the MASSES on 24 July
Starting today, regular fanbois will be guinea pigs, it tells Reg
HIDDEN packet sniffer spy tech in MILLIONS of iPhones, iPads – expert
Don't panic though – Apple's backdoor is not wide open to all, guru tells us
prev story


Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Backing up Big Data
Solving backup challenges and “protect everything from everywhere,” as we move into the era of big data management and the adoption of BYOD.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.