An honest analysis of anti-opioid data

Are doctors bribed by pharma? An analysis of data – By Rafael Fonseca MD & John A Tucker MBA, PhD – Jul 21, 2018
A Critical Analysis of a Recent Study by Hadland and colleagues

Here’s an astute analysis of the statistics used to demonize doctors’ “prescribing behaviors”, especially when they prescribe opioids.

Association studies that draw correlations between drug company-provided meals and physician prescribing behavior have become a favorite genre among advocates of greater separation between drug manufacturers and physicians.

Recent studies have demonstrated correlations between acceptance of drug manufacturer payments and undesirable physician behaviors, such as increased prescription of promoted drugs.  

The authors of such articles are usually careful to avoid making direct claims of a cause-effect relationship since their observations are based on correlation alone. Nonetheless, such a relationship is often implied by conjecture.

Further, the large number of publications in high profile journals on this subject can only be justified by concerns that such a cause-and-effect relationship exists and is widespread and nefarious.

In this article, we will examine a recent paper by Hadland et al. which explores correlational data relating opioid prescribing to opioid manufacturer payments and in which the authors imply the existence of a cause-and-effect relationship.

Hadland et al.: Opioid Prescriptions and Manufacturer Payments to Physicians

The authors of this paper linked physician-level data from the 2014 CMS Open Payments database to 2015 opioid prescribing behavior described in the Medicare Opioid Prescribing Database.

They explored the hypothesis that meals and other payments increase physician opioid prescribing by examining the association between receipt of meals and other financial benefits with the number of opioid prescriptions written. Specifically, they found the following:

A nearly linear relationship between the number of opioid manufacturer-provided meals accepted by a prescriber and the number of opioid prescriptions written.

The relevant data is provided in Figure 1 below. Prescribers who received nine meals from opioid manufacturers in 2014 prescribed opioid analgesics at slightly more than 3x the rate of those who accepted only one meal.

When broken down by physician specialty, those who accepted any payment from opioid manufacturers wrote between 1.2% more and 11% more opioid prescriptions as those who did not accept any such payments (Table 1).

Figure 1.

Hadland et al. conclude thatour findings suggest that manufacturers should consider a voluntary decrease or complete cessation of marketing to physicians. Federal and state governments should also consider legal limits on the number and amount of payments.

While no cause-and-effect relationship between payments and prescribing habits has been demonstrated by this correlative study,the implication that one exists is made clear in the authors’ recommendations.

Our View: It is More Complicated than That….

To better understand the issues presented by the Hadland’s correlative study, we undertook an independent analysis of the same data. We repeated the Hadland data extraction from the CMS sources cited in the paper. We associated payments with prescribing behavior using physician name and geographical information as described by Hadland

How Large is the Association Between Manufacturer Payments and Prescribing Volume?

Our first criticism of the Hadland analysis is directed at the non-standard presentation of the data in Figure 1. The most widely accepted way to present the relationship between two continuous variables such as payments and the prescription count is a correlation diagram. We present the data in this manner in Figure 2 (Note the logarithmic Y axis). Doctors who accepted no free meals from opioid manufacturers wrote between 0 and 1000 opioid prescriptions in 2015. As did those who accepted 50 or more.

Figure 2. Correlation Diagram Relating Number of Opioid Prescriptions Written to Number of Drug Maker Meals Accepted

This graph gives a very different impression than the presentation of the same data in Figure 1. Why is that?

Here we have shown every data point, though some are hard to see because there are so many of them (345K to be exact)

In Hadland’s presentation of the data, they grouped the prescribers into categories based on the number of meals that they accepted.

They calculated the mean for each group, which hides the tremendous variation in prescribing behavior within each group.

The error bars are shown in Hadland’s figure are not standard deviations (a measure of within-group variation) but standard errors (A measure of how precisely the mean has been estimated)

This qualifies as deliberate obfuscation, using various statistical models to find one that makes the data seem to support the original hypothesis.

The latter value is derived from the former by dividing by the square root of the number of data points, which ranges as high as 8468 for some of the categories in Hadland’s figure. So a clear representation of within-group variation would show error bars as much as 92-fold larger than those shown.

A similar criticism can be directed at the presentation of the data in Table 1.

Comparing mean prescribing rates between those who accepted any payment and those who accepted none gives a non-representative picture because the distributions are highly skewed.

Imagine a cancer trial in which 5 patients live 2, 3, 3, 4, or 20 months. Reporting that the average survival was 7.5 months and the standard deviation was 8.3 months really doesn’t give a very meaningful picture of what happened in the trial

This is the same point I’ve made for statistics hiding wide variation, with NO values close to the “mean” or “average” (like the average human having one breast and one testicle).

Similarly, Hadland et al. report that physicians who accepted payments in 2014 wrote 539 +/- 945 prescriptions in 2015, while those who did not wrote 134 +/- 281.

This is clearly nonsense when the variation (+ or -) is greater than the “mean”.

Who are the physicians who wrote less than zero prescriptions in 2015, and what does a negative prescription look like?

This type of bizarre result arises from applying statistical methods appropriate to a normal distribution of values to a data set that is decidedly non-normal.

The problems become even more apparent when we compare these numbers to the authors’ statement in the text that

  • those who accepted payments in 2014 increased their prescription count in 2015 by 1.6, while
  • those who did not accept payments in 2014 reduced their prescription count by 0.8.

How is the difference (2.4 prescriptions) equal to 9.3% of 134 prescriptions (Table 1)?

And does a relative increase of 2.4 prescriptions per year from a base of 539 prescriptions merit publication in JAMA Internal Medicine and a call for legislation?

Are Drug Companies Paying Doctors to Write Prescriptions?

let’s analyze whether the relationship is causative or merely correlative.

Hadland’s implicit hypothesis is that doctors are writing opioid prescriptions in “exchange for pizza.”

An alternative explanation might be that attending manufacturer informational sessions at which meals are served and prescribing opioids might both be driven by having a practice that involves treating many pain patients.

Let’s look at the data and see if we can distinguish between these possibilities.

If Doctors are writing prescriptions in exchange for payments, one would expect that the number of prescriptions would rise predictably with the payment amount.

In practice, we find this is not the case.

only 1% of the total variation amongst prescribers is associated with variation in the amount of payment received

Figure 3. Relationship Between the Number of Opioid Prescriptions Written and Total Payments Received

if both attendance at educational sessions at which meals are served and opioid prescribing are driven by having a practice that involves treating many pain patients, one might expect a very modest or no correlation of prescribing with non-meal payments.

In practice, we see the latter (Figure 4).

Figure 4 was drawn using Hadland’s categorical style of presentation to allow direct comparison to Figure 1.

While Hadland found that opioid prescribing tripled as the number of industry-sponsored meals increased from one to nine, we find no trend in toward increased prescribing among those who received between $0.01 and $65,536 in non-meal payments from opioid manfacturers.

For the 58 physicians who received more than $65,536, the rate of prescribing was increased by nearly twofold relative to those receiving less than a dollar, but due to large within group differences, this difference was not statistically significant.

The fact that opioid prescribing correlates with the number of meals accepted but not with the total amount of non-meal payments received suggests that attendance at educational events at which meals are served and opioid prescribing are both driven by practice characteristics

In practice, we find that the association of increased opioid prescribing with attendance at informational lunches offered by the manufacturers of pain therapeutics is independent of whether the pain product is an opioid!

Conclusion

Correlation is not causation.

While many advocates of reduced interactions between “commercial” interests and physicians have implied or directly suggested a quid pro quo between industry meals and other financial interactions and prescribing habits, correlation alone does not prove a quid pro quo relationship.

In the case of opioid prescribing, we believe that we have presented a strong case that

1) the relationship between industry payments and prescribing is much weaker than has been presented in the literature, and

2) that prescribing and attendance at manufacturer-sponsored informational lunches are both driven by practice characteristics, rather than the meals themselves driving prescriptions

We believe that much of what has been published regarding the correlation of prescribing with industry payments and sponsored meals suffers from the shortcomings described in this short note.

In particular, many of these papers conflate causation with correlation.

This is also how opioid studies are mangled to make it seem like ill effects are associated with higher opioid doses, yet they ignore the fact that opioid doses are determined by pain level. (see Opioids Blamed for Side-Effects of Chronic Pain)

In cases where fairly simple and obvious analyses would serve to differentiate between the authors’ preconceptions and alternative interpretations of the data, these analyses have not been performed.

We urge all with an interest in this area to approach these data with the highest possible level of objectivity, as is our responsibility as scientists. We have done our best to do so here, and commit to doing so in our planned analyses of other papers in this area.

Other thoughts?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.