Trials of mindfulness to improve mental health selectively report positive results.
I’m sure this isn’t a big surprise to pain patients who have been told to meditate when their unmedicated pain becomes unmanageable.
There’s a little too much wishful thinking about mindfulness, and it is skewing how researchers report their studies of the technique.
These days, any treatment that doesn’t involve opioids is being lauded and recommended, often without scientific evidence. But we pain patients have said all along that most of these treatments just don’t work.
Researchers at McGill University in Montreal, Canada, analysed 124 published trials of mindfulness as a mental-health treatment, and found that scientists reported positive findings 60% more often than is statistically likely. The team also examined another 21 trials that were registered with databases such as ClinicalTrials.gov; of these, 62% were unpublished 30 months after they finished.
The findings—reported in PLoS ONE on April 8— hint that negative results are going unpublished.
A bias toward publishing studies that find the technique to be effective withholds important information from mental-health clinicians and patients, says Christopher Ferguson, a psychologist at Stetson University in Florida, who was not involved in the study.
The scientists’ calculations suggested that 66 of 124 trials would have positive results. Instead, 108 trials had positive results. And none of the 21 registered trials adequately specified which of the variables they tracked would be the main one used to evaluate success.
This doesn’t necessarily suggest that none of the mindfulness treatments work, says study co-author Brett Thombs, a psychologist at McGill. “I have no doubt that mindfulness helps a lot of people,” he says.
“I’m not against mindfulness. I think that we need to have honestly and completely reported evidence to figure out for whom it works and how much.”
Trials with larger sample sizes—and thus more statistical power—would be an improvement. In the McGill team’s analysis, the 30 trials with the most statistical power showed no over-reporting of positive results.
The bias towards reporting positive results is pervasive across many types of mental health, psychology and medical research, says Ferguson. For example, the widely popularized theory of ego depletion—that people have limited self-control for decisions—recently failed to hold up in a large replication trial.
“A lot of these things are reported to be true, they’re in a TEDx talk,” he says. “Now we’re seeing, when we look at things much more closely, we’ve kind of been bullshitting people [for] a decade.”
He advocates pre-registering studies, in which a journal reviews and accepts a study—including the outcomes that it will measure—before data collection begins. This way, the journal publishes the trial results regardless of whether they are negative or positive.
Without this kind of agreement, journals are more likely to publish only positive results, and scientists need published papers to get funding and tenure.
This creates a perverse incentive that does not make sense from a care standpoint. “For the health-care system,” says Thombs, “it’s just as important to know what doesn’t work.”
I’m tremendously relieved that this “quick fix” of using a tiny part of an ancient Asian lifestyle practice is being debunked.
I’ve always been suspicious of these claims because of how hard I’ve had to work at achieving even the smallest, though still significant, benefits of meditation.
It took me months of regular daily practice to make the slightest dent in my anxiety, which is a troubling outcome of many chronic diseases.
These days, a superficial form of meditation (“Meditation Light”), which barely scratches the surface of true meditation practice, is being promoted and marketed as a “quick fix” that anyone can access.
Meditation has been portrayed as a panacea, but the reality is that it can help you manage your feelings about pain, but not the pain itself (unless you are a yogi with decades of intense practice).
Meditation is not a cure for pain.
It can ease distress and suffering,
but cannot change physical issues that cause pain.
Any serious practitioner can tell you that to getting the real benefits of mindfulness meditation requires years of study and practice, not just a few weeks of once-a-week lessons.
Having researched this subject for many years, I strongly believe in the benefits of meditation. However, the
However, the current “fast food” approach to this practice cannot so quickly and so thoroughly ease our pain, especially for those of us suffering from serious physically-induced pain.
I will continue to practice, practice, practice, but I don’t expect a quick return on my significant investment of time and effort over the years. I know the benefits are subtle but they can facilitate deeply meaningful changes.
Diligent, difficult, and devoted practice is the only way to achieve the true benefits of meditation.