Experts agree: Your next car will be smarter than you

Heads up!

Safer merging into traffic is also one of the goals of Victor Ng-Thow-Hing and his team at the Honda Research Institute USA in Mountain View, California. Their system's recommendations, however, are presented on an interface that's at the core of their research: an augmented-reality head-up display viewed through a vehicle's windshield.

Like Tippelhofer and his Audi team, Ng-Thow-Hing and his three fellow researchers at Honda are focused on providing a vehicle's driver with information that will make driving safer and less stressful, despite the fact that cars are being loaded up with more and more distracting technology.

Changing demographics are one spur to their research. At the same time that cars and their multiple displays are becoming more complex, Ng-Thow-Hing said, more and more older drivers are on the road – and they have slower reaction times and problems with perception of such critical driving factors as speed and distance.

"And we also have a lot of young hipsters," he said. "A lot of people have fallen in love with their smartphone devices, and they want to have that connectivity, social media, and all that stuff in their cars" and use them safely. Both populations need help, he said.

What Honda is working on is augmented reality in the car. "And I want to differentiate that from just a simple head-up display. What I mean is that we're looking at a real-world scene, and we're actually augmenting that scene with computer-graphic elements mixed into that."

As with Audi's system, Honda's system – which they call HI-CAR, for Honda Interactive Contextual Augmented Reality – uses information both from sensors and from the internet. As Ng-Thow-Hing explained, however, augmented reality in a car is far different from the augmented reality apps many of us have checked out on our smartphones and tablets.

Smartphones are great for augmented reality, he said, because they're mobile, portable, and all have cameras. Tablets can be even better because they have a larger display with which to interact. "But in both of these cases the augmented-reality experience is not very immersive," he said, "because you're looking at everything through a video overlay – you're not actually seeing it directly in the scene."

Not so with automobiles. "The car is the ultimate mobile device," Ng-Thow-Hing said, because it's connected, portable, loaded with sensors, can have a larger amount of compute power to work with, and the windshield offers a larger field of view. "And you're actually looking through and seeing the real world through your wiindshield – it's not like a video overlay."

One of the problems that Ng-Thow-Hing sees in current augmented-reality displays is that their designers overlay flat, 2D labels on objects in a 3D world – a method that he argues creates a "cognitive dissonance." His system instead annotates real-world images with labels that have been transformed using perspective – a label identifying a builing, for example, is displayed as if it were a sign on the building itself, in the same perspective as the building.

Overlaying computer-generated imagery onto a real-world view through a car's windshield also presents two other challenges that overlaying imagery onto a flat video on a smartphone or tablet doesn't have: first, your eyes change focal length when moving from close-in to far-off objects; second, parallax shifts occur and misalign the augmented reality images when you shift your point of view inside the car. According to Ng-Thow-Hing, however, his team is tackling both those problems.

But even when such technical problems have been overcome, a head-up augmented reality must be carefully designed so as not to cause what Ng-Thow-Hing defined as "inattentional blindness," meaning distraction when interface-design elements such as blinking red lights cause the driver to look only at them and not at the scene as a whole. Also to be shunned are elements on the display that aren't related to driving – "album covers and things like that" – which can distract the driver.

Such lousy augmented-reality interface design can be disastrous. "You're no longer driving very carefully," he said. "When that happens, it can lead to big accidents and crashes. Dying is a bad user experience."

Quite.

One of the non-intrusive user-interface elements that the Honda team has come up with is a proposed answer to the merging and lane-changing challenge mentioned above. A grid is projected onto the head-up display with the active car in the middle, and other cars around it – front, back, and sides – are indicated by red rectangles. With this information, the driver can know what's approaching from the car's blind spot without having to glance backward and take his or her eyes off the road.

Another user interface that the team has developed aids left turns. The speed of oncoming cars is estimated by sensors, and their projected path is indicated by a yellow on-road overlay in the augmented-reality head-up display. The path of the driver's car is also indicated, turning left. When the oncoming car's path would intersect with the driver's care, that path turns red; when the turn would be safe, the driver's path turns green.

Before they test their system in an actual vehicle, the Honda team is using a driving simulator. Ng-Thow-Hing emphasized that much care and iteration must be undertaken to ensure that an augmented-reality display adds to the drivers's safety, not distracts as noted above. When the team moves from simulators to an actual vehicle, he said, "Most likely I'll be the test subject for the first car, and I don't want to die while doing it."