Lies, damned lies and government statistics

Apparently statistics are all down to personal opinion. Except, of course, when you are a government spokesperson, in which case they are the gospel truth. Despite expert evidence – and public exposure - that the “official estimate” of casualty reduction due to speed cameras is seriously overestimated, the Department for Transport continues to feed us the same old unreliable figures.

The story begins with the Tories stirring matters on the thorny issue of speed cameras. (And before El Reg commenters reach for the flaming green ink, please note: this is not about the effects of speed. Nor is it about the effects of speed cameras, precisely. Rather, it is about the way in which the government treats evidence when it doesn’t like the way experts interpret it.)

Research by Tory police reform spokesman, David Ruffley, shows that the take from speed cameras has quadrupled over the last ten years. Drivers now fork out well in excess of £100m a year in speeding fines, and Mr Ruffley would like to know what the money is spent on.

The Department for Transport (DfT) passed on that one. Instead, they referred Mr Ruffley to the pretty impressive figure of 1,745 fewer deaths and serious injuries (KSIs) at camera sites each year. This figure first surfaced in 2005 in a four-year evaluation report on the speed camera programme, carried out by “independent” experts PA Consulting.

In fact, the report is a statistician’s goldmine, with about a fifth of the whole (including a dedicated appendix) given over to the thorny issue of something called “regression to the mean” (rtm).

In layman’s terms: cameras are sited on spots where the accident rate is usually above average. Therefore, some part of the reduction in KSIs attributed to a camera will actually be due to a reduction that would have happened anyway.

If you really want to understand the effect of speed cameras, you absolutely have to have a decent estimate for rtm. And lo! Academics Linda Mountain (Liverpool) and Mike Maher, now Professor of the Mathematical Analysis of Transport Systems (Leeds), provided the DfT with such an estimate.

Once they had done that, then within the data they looked at – and they were only able to study a small subset of the whole – the actual effect of speed cameras was significantly less than the top-line figure. In fact, it was about half what the DfT claim.

If that effect were applied to the nation as a whole, the real effect of speed cameras could be as low as just 873 saved KSIs per year. However, being statisticians, they did not stick their neck out quite so far. (It took the Times to make that leap!)

Did you see that

So where are we now? A much better estimate of the effect of speed cameras could be obtained if extra data were available or more research carried out. This has not been done and, an informed source tells us, is unlikely to be - it might be too embarrassing.

Since the original report, the occasional parliamentary question has been used to prod the DfT on the rtm issue – to little effect. Official documents continue to quote the 1,745 headline figure, with an obligatory footnote to the effect that this does not take account of rtm effects. A bit like diagnosing a patient as perfectly healthy, apart from the fact that they are currently dead.

However, when it comes to the general public, the DfT appears to prefer the much easier, simpler – and probably wrong - line that speed cams do save 1,745 KSIs a year.

As for the effects of rtm? When El Reg asked them about that, they described it as “merely a matter of opinion”.

Not quite the view of Professor Maher, who, as one of the leading UK experts on this topic, suggested politely that “there remains strong evidence that the headline figure is an over-estimate”.

Does it matter? After all, a reduction in casualties of even 870-odd per year is still worth having. The rtm analysis does not necessarily undermine the case for speed cameras. But coyness about the topic – espcially from a government that claims to rely on the evidence – not only debases public debate, but bodes ill for other more difficult topics.

Does government fix statistics? One might as well inquire after the defecatory habits of the arboreal ursine. But it comes to something when they so blatantly spin against what their own experts tell them. ®