Lies, damn lies and election polls: Why GE2015 pundits fluffed the numbers so badly

Systemic bias? Perhaps - but it's not intentional

As several commenters have already observed, this consistency of error suggests that something “systemic” was going on. That is, there was something about the way the polls were conducted that skewed the forecast.

Over the years, a range of factors have been identified as having the potential to skew results, from the age/gender/class profile of those polled, to the way they are polled (face-to-face or by phone) and even the order in which questions are asked.

Pollsters therefore apply correction factors. Young people tend to vote less than older people: so they upweight the influence of the latter and downplay the former. Those from a lower social class also tend to vote less so there is correction applied there, too.

Over the years, polling organisations have identified what they consider to be the most significant sources of skew and they have developed a range of corrections to apply to their raw data. The problem? First, the nature and direction of the correction required can only be fully understood after the event: so every election forecast is built with one eye on the rear-view mirror.

Worse, as the outcome from your predictive tool is increasingly based not the data going into it, but the transformations applied to that data – the “fudge factors” – the integrity of the tools themselves begins to be called into account.

In this case, one of the largest fudge factors was that of the “shy Tory”: first identified in the 1992 general election, this is the voter unwilling to own to voting conservative. Not only are they unprepared to admit this publically: they also fib to the pollsters.

Attempts are regularly made to compensate for this effect. Yet even this appears to have been insufficient on this occasion.

Telling the truth

Perhaps that gets us closer to the heart of the problem, which is simply that a chunk of voters – perhaps a growing chunk – just aren’t telling the truth any more.

Some may simply not know how they are going to vote in advance. Anecdotal evidence suggests that there was a lot of last minute agonising in the polling booth. Most polls recognise this and do their best to compensate, allocating “don’t knows” according to previously tried and trusted formulae.

And then there is that other thing, the social embarrassment factor, or a very British reluctance to own that one is about to do something rather disreputable. Like have a second slice of pie.

Is it possible that as well as the shy Tory, this last election saw the emergence of the “let’s give Clegg a good kicking” factor?

For while shy Toryness would just about explain the discrepancy between forecast and exit poll, it does not explain the discrepancy between Lib Dem support and seats. The analysis is yet to be done, but it seems plausible that far from having local strengths, it may turn out that where the Lib Dems had sitting MPs, the public were more inclined to vote against them to “teach them a lesson.”

The fact that many may now feel that they took this a bit far is evidenced by the fact that just days after the election there are signs of a Lib Dem bounceback: many, many electors reportedly returning to the fold (too late!) and a rise in party membership of more than 4,000 since the weekend.

Behind all of the above, of course, is the endearing fallacy that voters vote for rational and knowable reasons. Perhaps many do: but in a system where the “noise” – a shift in votes of just one or two percentage points either way - can have a major impact on the outcome, perhaps the truth is that we can never have the level of accuracy that newspapers and assorted columnists pretend to offer.

Does it matter?

In one sense, no. We get the government we vote for.

But in another very real sense, polls do matter. They feed on themselves, influencing the campaign and, critically, influencing the things that politicians promise and therefore nail themselves to during the campaign. The last-minute panic in the Scottish referendum over the possibility of a Yes vote led to promises that are now directly impacting the future of Scotland.

We cannot now know what promises might have been made, or not made, had the polls provided a truer reflection in advance of how the voters were going to vote.

Polls also have a direct and potentially toxic impact upon our finances. In the days before the election UK share prices tumbled and the pound took a major hit against other currencies. All this, righted itself the day after, as the markets, in typical fashion, overshot, allowing Tory voters to celebrate not just an election victory, but some very bankable gains as well.

Still, that may have cost some pension funds – and therefore individual people – some very real money.

The answer, in the end, is that the answer is probably never going to be precisely knowable. Polling organisations will improve their fudge factors and they won’t make the same mistakes next time. They’ll just make different ones instead.

Whereas we, the electorate, will make exactly the same error we always do: we will believe that for once, the polls have got it right – and we will be cross when we discover they haven’t. ®