Emissions cheating detection shines light on black box code
But boffins say better tools are needed to nab scofflaws
Researchers analyzing the emissions defeat devices found in automobiles made by the Volkswagen Group and Chrysler Fiat Automobiles have developed a way to test software for misbehavior, but they caution that lack of visibility into programming code could pose a challenge for regulators.
In 2015, the US Environmental Protection Agency charged Volkswagen AG and its subsidiaries, Audi AG and Volkswagen Group of America, with violating the Clean Air Act. The agency said the company had used a "defeat device" to alter diesel engine emissions on vehicles over the span of several years during regulatory testing.
Volkswagen has acknowledged that over 11 million vehicles worldwide carried its emissions cheating software.
The scandal claimed the company's CEO at the time, Martin Winterkorn, and is expected to cost the company at least $18.3 billion. In April, a US judge hit the company with a $2.8 billion criminal fine.
An academic paper published in 2015 projected that 59 people will die prematurely as a result of the unlawful automobile emissions.
In January, the EPA issued a similar notice of violation to Fiat Chrysler over 104,000 Dodge Ram 1500 and Jeep Grand Cherokee vehicles (Model Years 2014-2016). According to USA Today, the US Department of Justice is preparing to sue the automaker if terms can be reached with regulators.
In a paper presented this week at the 38th IEEE Symposium on Security and Privacy, "How They Did It: An Analysis of Emission Defeat Devices in Modern Automobiles," researchers from Ruhr University Bochum in Germany and University of California, San Diego analyzed the defeat devices used by Volkswagen and Chrysler Fiat with an eye toward developing reliable detection methods through static (source code) analysis.
"Electronic engine control has also made it easier to circumvent emissions testing by implementing a defeat device in software," the paper explains. "The black box nature of emissions testing makes it nearly impossible to discover such a software-based defeat device during a test, forcing regulators to rely on heavy fines to discourage cheating. Unfortunately, as the Volkswagen case illustrates, it can take many years to discover such a defeat device."
In affected Volkswagen Group vehicles, the Engine Control Unit (ECU) governs the operation of the engine. The ECUs in question relied on a signal referred to as the "acoustic condition" to trigger emissions cheating. Activating this signal, stored in the variable
stNsCharCor, involves assessing various engine parameters, such as coolant temperature, oil temperature, fuel temperature, atmospheric pressure, and whether the engine has recently been started.
stNsCharCor = 0 means that the ECU considers itself to be in normal driving mode, while
stNsCharCor = 1 indicates testing (emmissions-compliant) mode," the paper explains.
The researchers developed software called CURVEDIFF that reliably detects ECU firmware images configured to cheat. But the technique is not foolproof because a carmaker could find ways to evade detection.
New and better tools needed
"Regulators and the research community will need to rethink how we make sure that vehicles, and other devices subject to regulatory oversight, are compliant," said Kirill Levchenko, assistant research scientist at University of California, San Diego, and co-author of the IEEE paper, in an email to The Register.
"This is something we are working on right now, but the bottom line is that we need to give regulators new tools to deal with complex software-controlled systems. What our work highlights is that defeat devices can be arbitrarily complex and that a single fixed test is simply not enough to ensure a vehicle is in compliance."
The paper also points out that passive techniques to cheat on emissions tests are not easily detected.
"For example, emission tests are comparably short, which opens up the possibility to simply stay in a compliant mode for as long as the average emission test is carried out and switch to a more harmful emissions policy afterwards," the paper says. "The Fiat defeat device we discussed earlier belongs to this category."
Levchenko said additional work needs to be done to enable regulators to catch deliberate attempts to deceive. "Making sure software works correctly is already hard, even in the best conditions," he said.
"If you now have the software developer trying to conceal some behavior and evade detection, it becomes much, much harder. We will need a new breed of techniques for this, because it's not something we are well-equipped to handle today." ®