It looks like I’m not the only one upset by the severely biased report from the Brookings Institute:
An article published December 7 by the Brookings Institution, a prominent Washington, DC think tank, is under fire for promoting the idea that harm reduction approaches—like syringe exchange and naloxone—may encourage “riskier opioid use” and increase rates of opioid-related deaths.
The focus of the criticism is the choice of which articles to cite—and which not to cite—by the authors of the Institution’s “research roundup.”
The paper was written by
- Jennifer L. Doleac, an associate professor of Economics at Texas A&M University,
- Anita Mukherjee, an assistant professor of Risk and insurance at the University of Wisconsin-Madison, and
- Molly Schnell, a postdoctoral Fellow at the Stanford Institute for Economic Policy Research.
These are all economists! No wonder this supposed “research” was so cherry-picked with a view to “Risk and Insurance” and economic factors. These folks know nothing about the difference between prescription opioids for pain versus the illicit opioids involved in addiction.
“New work by Packham and Wells (2018) suggests that syringe exchange programs—a staple of harm-reduction efforts—reduce HIV rates as intended but unintentionally increase opioid-related mortality by making it easier, cheaper, and safer to use heroin” the authors state.
“Urge @BrookingsEcon to RESCIND this piece and have it undergo peer review by subject matter experts,” tweeted Leo Beletsky, associate professor of Law and Health Sciences at Northeastern University.
Luckily we have Leo Beletsky, who is a real authority on public health, making a public call for a retraction.
He is far more knowledgable about the overdose crisis than those three economists from the Brookings Institute.
“It ignores much of the empirical evidence base on the topics it purports to analyze. This is not a balanced, informed analysis and will do public health harm if left uncorrected.”
That article was also confronted by a barrage of methodological criticism when it was published.
For example, as Gertner pointed out at the time, “the authors find that naloxone access laws lead to more opioid-related emergency department visits, the premise being that naloxone access laws increase opioid overdoses.
But there’s a far more likely explanation:People are generally instructed to seek medical care for overdose after receiving naloxone.”
Filter reached out to Jennifer L. Doleac, the lead author of the paper, through Brookings, and was instructed to email questions. Shannon Meraw, Brookings’ media relations manager, responded to questions with the following:
So a “media relations manager” is now answering questions about a paper written by know-nothing economists. The authors are hiding behind their institute’s public relations machine.
“Worth noting—we’ve received several questions related to your numbers 1 and 2. The authors will be updating the piece to clarify why some of the findings cited are different from those supported by many public health advocates (economists tend to think about these issues differently than folks in other disciplines). I’m not sure when that will be finalized, but we will include an editor’s note when it is.”
Doleac has also tweeted:
“Sincere request: If you think our post on opioid has missed relevant studies from public health or any other discipline, please email them to me. Vague allusions to decades of evidence aren’t persuasive but I do like reading good research, & am willing/able to change my priors!”
Doleac seemed to be referring to a tweet thread of Gertner’s that referenced “decades of evidence.” But in the same thread, Gertner provided a link to a literature review of studies on this exact topic.
Plus, as many pointed out, looking into previous research on a topic is generally done before a lit review is written, not after.
Meraw added in her email:
“Brookings of course doesn’t take a position on any issue, but as we provide a platform for individuals to do so, we do want to make sure the debate is properly contextualized.”
Brookings may not officially “take a position,” but many policymakers look to the the think tank to form their views. The New York Times called Brookings “the most prestigious think tank in the world.” So by choosing who to give a platform to (and who not), Brookings effectively does take positions.
On December 11, the Brookings Institution’s article was also updated “to contextualize and clarify its discussion of harm reduction policies.”
We recognize that these findings strongly contrast with previous work by public health researchers that suggested few, if any, trade-offs that result from harm reduction policies. We believe that the research described above credibly establishes a causal effect, but given the stakes at-hand—and the impact drug abuse can have on individuals, their families, and society at-large—this is clearly an area that needs further study. In the meantime, policymakers should acknowledge and work to mitigate any potential unintended consequences when implementing harm-reduction policies.”
The new paragraphs do not link to any of the referenced “previous work by public health researchers.”
And here’s the excellent PubMed abstract of a real review of evidence:
This first international review of the evidence that needle syringe programs reduce HIV infection among injecting drug users found that conservative interpretation of the published data fulfills six of the nine Bradford Hill criteria
- (strength of association,
- replication of findings,
- temporal sequence,
- biological plausibility,
- coherence of evidence, and
- reasoning by analogy)
and all six additional criteria [for public health -zyp]
- absence of negative consequences,
- feasibility of implementation,
- expansion and coverage,
- unanticipated benefits, and
- application to special populations).
The Bradford Hill criteria are often used to evaluate public health interventions.
I find it reassuring that there are such excellent criteria with which to evaluate all the “studies” being done on opioids and overdoses.
By these measures, I’m pretty sure all the nonsensical opioid studies would be considered invalid (see Opioids Blamed for Side-Effects of Chronic Pain).
The principal finding of this review was that there is compelling evidence of effectiveness, safety, and cost-effectiveness, consistent with seven previous reviews conducted by or on behalf of U.S. government agencies. Authorities in countries affected or threatened by HIV infection among injecting drug users should carefully consider this convincing evidence now available for needle syringe programs with a view to establishing or expanding needle syringe programs to scale.
Here is a more complete explanation of the Bradford Hill criteria, from Wikipedia:
The Bradford Hill criteria, otherwise known as Hill’s criteria for causation, are a group of 9 principles, established in 1965 by the English epidemiologist Sir Austin Bradford Hill. They can be useful in establishing epidemiologic evidence of a causal relationship between a presumed cause and an observed effect and have been widely used in public health research.
In 1965, the English statistician Sir Austin Bradford Hill proposed a set of nine criteria to provide epidemiologic evidence of a causal relationship between a presumed cause and an observed effect. (For example, he demonstrated the connection between cigarette smoking and lung cancer.)
The list of the criteria is as follows:
- Strength (effect size): A small association does not mean that there is not a causal effect, though the larger the association, the more likely that it is causal.
- Consistency (reproducibility): Consistent findings observed by different persons in different places with different samples strengthens the likelihood of an effect.
- Specificity: Causation is likely if there is a very specific population at a specific site and disease with no other likely explanation. The more specific an association between a factor and an effect is, the bigger the probability of a causal relationship.
- Temporality: The effect has to occur after the cause (and if there is an expected delay between the cause and expected effect, then the effect must occur after that delay).
- Biological gradient: Greater exposure should generally lead to greater incidence of the effect. However, in some cases, the mere presence of the factor can trigger the effect. In other cases, an inverse proportion is observed: greater exposure leads to lower incidence.
- Plausibility: A plausible mechanism between cause and effect is helpful (but Hill noted that knowledge of the mechanism is limited by current knowledge).
- Coherence: Coherence between epidemiological and laboratory findings increases the likelihood of an effect. However, Hill noted that “… lack of such [laboratory] evidence cannot nullify the epidemiological effect on associations”.
- Experiment: “Occasionally it is possible to appeal to experimental evidence”.
- Analogy: The effect of similar factors may be considered.
…the Bradford Hill criteria applied to complex systems such as health sciences are useful in prediction models where a consequence is sought; explanation models as to why causation occurred are deduced less easily from Bradford Hill criteria because the instigation of causation, rather than the consequence, is needed for these models.
Here is a review of the criteria:
The evolution of evidence hierarchies: what can Bradford Hill’s ‘guidelines for causation’ contribute? Free full text PMC article