Remember the Uber self-driving car that killed a woman crossing the street? The AI had no clue about jaywalkers

The self-driving Uber car that hit and killed a woman walking her bike across a street wasn’t designed to detect “jaywalking pedestrians.”

That's according to an official dossier published by the US National Safety Transportation Board (NTSB) on Tuesday.

The March 2018 accident was the first recorded death by a fully autonomous vehicle. On-board video footage showed the victim, 49-year-old Elaine Herzberg, pushing her bike at night across a road in Tempe, Arizona, moments before she was struck by the AI-powered SUV at 39 MPH.

Now, an investigation by the NTSB into the crash has pinpointed a likely major contributing factor: the code couldn't recognize her as a pedestrian, because she was not at an obvious designated crossing. Rather than correctly anticipating her movements as a person moving across the road, it ended up running right into her.

“The system design did not include a consideration for jaywalking pedestrians,” the watchdog stated [PDF] in its write-up. “Instead, the system had initially classified her as an 'other' object which are not assigned goals.”

The computer-vision systems in self-driving cars are trained to identify things, such as other vehicles, trees, sign posts, bicycles, and so on, and make decisions on what to do next using that information. It appears Uber’s software wasn’t able to identify Herzberg since there was no classification label for a person not using a proper crossing point, and it wasn't able to make the right decisions.

Countdown to impact

Some 5.6 seconds before hitting her, the car's radar detected Herzberg, and at 5.2 seconds, she was picked out by the Lidar. However, the machine-learning system more or less ignored her, figuring her to be a non-moving object not in the vehicle's way.

As the robo-vehicle drew nearer, it categorized her variously as a vehicle, a bike, or some other thing that was not, or was only partially, in its way.

Just 1.2 seconds before hitting her, it identified her not only as a bicycle but also clearly in the path of its travel, by which point it was far too late to change course.

Crucially, the software was "unable to correctly predict the path" of Herzberg, the report noted. Perhaps if it had correctly identified her early on as a person crossing the street, it could have dramatically slowed down or otherwise tried to avoid her. But it did not anticipate the collision because, for the most part, it considered her to be some unknown object not in the way.

Also, don't forget: the SUV's emergency braking system was deliberately disabled because when it was switched on, the vehicle would act erratically, according to Uber. The software biz previously said “the vehicle operator is relied on to intervene and take action," in an emergency.

The self-driving car was fully autonomous at the time at the accident, though it had a human driver at the wheel. An internal camera caught the Uber worker looking down and away from the road moments before the accident, unaware of Herzberg’s presence before it was too late.

Below is a timetable, produced by the NTSB, detailing the car's decision-making and speed in the seconds before the accident:

The report comes just two weeks before the board is due to hold a public meeting to “determine the probable cause of a crash involving a pedestrian and an Uber test vehicle,” on 19 November.

The safety board also released more than 40 documents totaling at least 430 pages with various bits of supporting evidence and comprehensive accounts into things like the vehicle’s properties and the internal safety culture at Uber.

The massive data dump also revealed that the ride-hailing biz's self-driving cars were involved in 37 smashes between September 2016 and March 2018, prior to the deadly accident. In these 37 incidents, all of the robo-vehicles were driving in autonomous mode, and in 33 of these cases, other vehicles crashed into the self-driving cars.

'Tragedy'

On Tuesday evening, a spokesperson for Uber told us: “We regret the March 2018 crash involving one of our self-driving vehicles that took Elaine Herzberg’s life. In the wake of this tragedy, the team at Uber Advanced Technologies Group has adopted critical program improvements to further prioritize safety.

"We deeply value the thoroughness of the NTSB’s investigation into the crash and look forward to reviewing their recommendations once issued after the NTSB’s board meeting later this month.”

El Reg asked Uber how it planned to train its autonomous vehicles to detect jaywalkers in future, and the biz told us: "We conduct hundreds of tests in simulation and on our test track to help us improve the overall safety of our system. These tests include taking our software and having it go through scenarios that involve complex situations like jaywalking, hard braking, and unprotected turns." ®