This article is more than 1 year old
Health quango: Booze 'evidence' not Puritan enough, do us another
FoI request reveals researchers pressured to tweak base case
Updated Academics at the UK's leading alcohol research centre tweaked their model to help the government introduce more Puritanical booze advice.
The tweaks emerged after FoI requests uncovered correspondence between the Sheffield Alcohol Research Group (SARG) and the quango Public Health England, which had commissioned the researchers for impartial advice.
This culminated in new advice issued by the Chief Medical Officer Sally Davies that there was "no safe level" of drinking. Davis also lowered the recommended weekly alcohol intake guidelines for adults.
Backing up the new advice was "evidence" from SARG in a report published the same day.
The advice surprised many, and was questioned by the Royal Statistical Society and others, who pointed out the epidemiological evidence of risk followed a J-curve: abstinence was riskier than a low consumption of alcohol. The BMJ weighed in earlier this year, confirming that non-drinking increased the risk of a bunch of conditions. If official government health advice should exist at all, it should recommend moderate drinking, not total abstinence (e.g. "Have your had your two units a day?").
However, we now know that there was an earlier draft. And the version of Sheffield's model in this version supported the original higher level advice. In a version sent to PHE in December 2014, it looked like this:
What had changed between the earlier model and the one supporting the more Puritanical guidelines? PHE had suggested that Sheffield tweak its model to "estimate risk urges without threshold effects for wholly alcohol-attributable chronic conditions". This had the effect of inflating risk for which alcohol can at worst only be partly to blame.
Sheffield initially pushed back. Why do you want this? it asked in 2015.
Christopher Snowdon, whose FoI requests unearthed the story, wrote:
This idea evidently concerned the Sheffield team. Writing back the following day, they said that "the first four items on the list are not a problem and can be done before 11th March for £7,800 including VAT". But with regards to Point 5 they were "unclear exactly what was being requested here and why it was requested in addition to item 4 (a sensitivity analysis on threshold effects)".
Our view remains that it does not seem right to assign people drinking at very low levels a risk of acquiring alcoholic liver disease and similar conditions. Unless there are strong opposing views, we think it better to keep the threshold in the base case.
Removing thresholds did indeed get the "headline" the study's sole sponsor, PHE, desired:
The headline message from the new base case analysis is removing all of the threshold effects and remedying the problems with the model leads to implied guideline thresholds which are around 30%-50% lower than those in the previous base case.
It's now in the public record that the academics were unhappy with advice that contradicted the empirical evidence base. Minutes of a meeting on April 8, 2015, find Sheffield's John Holmes signalling as much.
The new risk curves were "not precisely consistent with the literature" and added other caveats. And in the final report was a remarkable warning, not noticed at the time:
These results suggest the base case should not be accepted uncritically as the implied guideline thresholds are sensitive to alternative assumptions and baseline data and there are not strong arguments for preferring the base case specifications over those used in the sensitivity analyses.
"At various stages in the report, the reader gets the impression that the authors are trying to distance themselves from their work. On more than one occasion, they stress that they would not normally program the Sheffield Alcohol Policy Model (SAPM) in the way they had," explains lifestyle regulation researcher Snowdon.
Sheffield stated that the threshold effect had been removed at the request of the sponsors, and was more or less telling readers to ignore the new, scarier base case analysis that PHE wanted.
We asked Sheffield for a statement.
Dr John Holmes, from the Sheffield Alcohol Research Group (SARG), said:
The report by the Sheffield Alcohol Research Group was one of many pieces of evidence considered to inform the UK Chief Medical Officers' revisions to the drinking guidelines – no single piece of evidence or modelling decision was used in isolation to determine the final guidelines.
As is normal practice with decision-making committees, additional analyses were requested by the Guideline Development Group in relation to uncertainties in the scientific evidence. All analyses were included within the published report and discussed in detail with the Guideline Development Group.
SARG doesn't dispute the quotes unearthed by FoI, but claims several were taken out of context:
The quote from a Sheffield Alcohol Research Group email on 14 January 2015 stating: "We do not expect to make any further changes to the numbers" is taken out of context.
The following sentence of the original email states: "We are also aware the GDG may make further comments which we would want to address."
Minutes from the subsequent GDG meeting on 21 January 2015 state that, after hearing Sheffield's presentation of their work, the GDG concluded: "A holistic, expert judgement on guideline levels would be needed, taking account of uncertainties and issues not fully modelled".
This demonstrates that the group recognised there was considerable scientific uncertainty present and that no single piece of evidence or modelling decision used in isolation to would determine the final guideline.
As noted in the Royal Statistical Society's consultation response: "This is a contested area of science with considerable uncertainties" (paragraph 1.1).
The change to the base case analyses related to a point of scientific uncertainty. The Sheffield Alcohol Research Group were happy with the decision taken whereby the base case analysis was revised but the original modelling assumptions were retained as one of a series of sensitivity analyses.
Those analyses explored major areas of uncertainty within the underlying evidence and their implications for the Guideline Development Group's work. The group considered those sensitivity analyses in detail and took them into account in their decision-making.
"When it failed to produce the results needed to justify a change in the guidelines, PHE told SARG to program it in a way that it had never been programmed before, using an assumption that had no scientific basis and about which the Sheffield team had obvious, well-founded reservations," Snowdon concludes. ®
Updated to add
A PHE spokesperson told The Register: "The UK Chief Medical Officers' alcohol guidelines were based on a comprehensive analysis of the evidence and advice from the Guidelines Development Group of independent experts.
"As part of the secretariat to the group, we commissioned the analysis, as requested by the Guidelines Development Group, from Sheffield University. We categorically refute the claim that PHE in any way attempted to influence or pressure Sheffield University on their research work to inform the alcohol guidelines."
Biting the hand that feeds IT
