The Great Brain Scan Scandal: It isn’t just boffins who should be ashamed

Special Report If the fMRI brain-scanning fad is well and truly over, then many fashionable intellectual ideas look like collateral damage, too.

What might generously be called the “British intelligentsia” – our chattering classes – fell particularly hard for the promise that “new discoveries in brain science” had revealed a new understanding of human behaviour, which shed new light on emotions, personality and decision making. But all they were looking at was statistical quirks. There was no science to speak of, the results of the experiments were effectively meaningless, and couldn’t support the (often contradictory) conclusions being touted. The fMRI machine was a very expensive way of legitimising an anecdote.

This is an academic scandal that’s been waiting to explode for years, for plenty of warning signs were there.

Voodoo Correlations

In 2005, Ed Vul, now a psychology professor at UCSD, and Hal Pashler – then and now at UCSD – were puzzled by a claim being made in a talk by a neuroscience researcher. He was explaining a study that purported to report a high correlation between a test subject’s brain activity and the speed with which they left the room after the study.

“It seemed unbelievable to us that activity in this specific brain area could account for so much of the variance in walking speed,” explained Vul. “Especially so, because the fMRI activity was measured some two hours before the walking happened. So either activity in this area directly controlled motor action with a delay of two hours — something we found hard to believe — or there was something fishy going on.”

Vul and Pashler sounded the alarm in a paper originally titled Voodoo correlations in social neuroscience. Their methodological worry was that researchers pick and choose which tiny areas, or voxels, of the brain are seen to “light up” during an experiment, and then attribute a high significance to these areas. This is circular logic, Vul explained: “[The procedure] chooses voxels that have a high correlation, and then estimates a high average correlation. This practice inflates the correlation measurement because it selects those voxels that have benefited from chance, as well as any real underlying correlation, pushing up the numbers.”

He adds: “If researchers select only highly correlated voxels, they select voxels that ‘got lucky’, as well as having some underlying correlation. So if you take the correlations you used to pick out the voxels as a measure of the true correlation for these voxels, you will get a very misleading overestimate.”

A theory of everything, for anything

Voodoo Correlations made an impact on the popular science press, but the concerns it raised were soon swept aside. The pictures the fMRI scanners were producing were dazzling, and “your brain on…” stories made it into the papers almost every day. The brain scanning boffins were feeding an insatiable demand. The conclusions the neuroscientists drew were eagerly devoured by social scientists and policy makers. The “breakthroughs” offered up a very tempting picture for some onlookers: a much more simplified, predictable and malleable "human" than previous generations of psychologists had drawn. Most attractive of all, particularly to those of an authoritarian disposition, was the idea that questions hitherto requiring an individual to have independent moral judgement could be cast in a new light:

“Brain images … convey the erroneous message (and here the press is deeply culpable for reporting the studies so uncritically) that thoughts, feelings and reactions are hard-wired and unchangeable,” Sharon Begley noted in Newsweek.

“That profoundly shapes how we view ourselves and others — in particular how we think of free will and individual responsibility. And the day is coming when brain images will be used to make predictions about behaviors, attitudes and aptitudes.”

The New Brain Science adventure had the veneer of science, since the fMRI scanner was a new kind of machine, and unimpeachable men in white lab coats were conducting the experiments. Luvvies do tend to put scientists – of any discipline – on a pedestal, and it’s something the scientists do little to discourage.

“Philosophy is dead,” boasted Professor Stephen Hawking at the start of his bestseller The Grand Design. "Philosophy has not kept up with modern developments in science, particularly physics. Scientists have become the bearers of the torch of discovery in our quest for knowledge.

Will Davies has the best account of the rise of pop behaviourism in his book, The Happiness Industry. Davies was inspired to write it when he heard neuroscience and behavioural economics being advanced as reasons for the great financial crash of 2008. (Such explainery continues, here and here.) Curiously, such accounts caught the attention of funding bodies. Since the 1980s the “decade of the brain” has been launched very often. The European Commission launched one in 1992. But with the fMRI fad, a firehose of public money was splashed at it. Davies writes that EU spent €2bn on neuroscience projects between 2007 and 2013. President Obama then trumped that with a $3bn BRAIN initiative. Then it caught on. Looking at blood flows in an individual's brain became a theory capable of explaining anything and everything.

Stanford sprouted a NeuroLaw department. The behavioural economics movement seized upon it as scientific proof. Every fashionable salon needed to show it was abreast of these great breakthroughs. Under Matthew Taylor, the RSA spewed forth projects such as Neuroscience and Social Progress or The Social Brain.

Pop behaviourism found a champion in Malcolm Gladwell, and later Jonah Lehrer, the feted author of books with the titles Proust Was a Neuroscientist, How we Decide and even more ambitiously, Imagine: How Creativity Works. The shelves groaned under their imitators. TED talks became a vehicle for obscure psychologists or economists to become minor celebrities, or quasi public intellectuals.

And the neuroscientists’ claims grew ever more extravagant.

“Gallant, a researcher at the University of California, Berkeley, has a brain decoding machine – a device that uses brain scanning to peer into people’s minds and reconstruct what they’re seeing,” the BBC proclaimed, in an uncritical puff piece that also highlighted a Japanese “dream reader”. Both projects simply use fMRI machines, with fancy statistical post-processing.

Still, nobody wanted to hear the bad news: that the new brain science might not actually be science at all.