That Public Health study? No, it didn't say 'don't do chemo'
When big media goes clueless about big data
“Chemotherapy kills” was bound to pique our interest, especially since in the best traditions of modern research, its source was a badly-reported scientific study.
The screaming headlines were just as you'd expect: “Chemotherapy warning as hundreds die from cancer fighting drugs” from Blighty's The Telegraph; almost the same – “Chemotherapy WARNING: Hundreds die from cancer drug treatment” – from The Express; the more measured “Nineteen NHS trusts to review chemotherapy over high death rates” from The Daily Mail, and so on.
The mistreatment of the study has sent the “natural cure” blogs into a veritable frenzy of anti-medicine frothing which The Register will ignore.
Having a high regard for the boffins that create science, The Register decided to walk through the study with a scientist – Dr Darren Saunders, cancer researcher at the University of New South Wales.
Also, it's a study that can be legitimately put under the heading of “big data” (tens of thousands of records plucked from millions for analysis) – and it's instructive not just about what the data can tell us, but also (and in terms of media reporting), what it can't tell us.
Don't generalise to populations from subsets
The first thing to understand about the study, Dr Saunders told Vulture South, is that it sweeps up all cases of breast and lung cancer in the UK in 2014. The study looked at patient outcomes – particularly deaths – in the first 30 days after starting chemotherapy, across a total of 32,862 patients for that year (23,228 breast cancers and 9,634 lung cancers).
That covers a very broad spectrum of individuals: from the young to the old; from early-detection to cases that aren't spotted until after metastasis has planted cancers all over the body.
The study itself – and the Public Health England media release – were careful to identify exactly which subgroups had the highest death rates: those already receiving palliative care (1,289 patients) and those treated “with the intention to cure” (94 patients in total – 0.4 per cent of the full sample).
“Of course some subgroups have a higher increase in mortality,” Dr Saunders said – he added that anybody already frail at the commencement of chemotherapy is more likely to die from it than the healthy.
Who chose? The data doesn't say
Dr Saunders also identified two distinct late-diagnosis groups to help El Reg interpret the data.
The group already under palliative care are identified in the study. Dr Saunders explained that in such cases, chemotherapy is given in the hope of improving quality of life – for example, to shrink an inoperable secondary tumour that would otherwise cause intolerable and intractable pain.
However, there's a possible (or probable) subgroup not captured in the raw data, and therefore invisible to the study: someone whose symptoms and therefore diagnosis come so late that they – the patients, not the clinicians – choose a dangerous course of treatment in hopes that it works.
“Patient choice is very poorly represented in the data,” Dr Saunders told The Register. That means we don't know how many patients (if any) were advised against chemotherapy, but responded: “the cancer's going to kill me anyway, let's try it.”