Here’s a typical abstract for a study of a drug that everyone knows works:

We study the efficacy of the drug Azadravapine on pain in a population of chronic pain sufferers. Azadravapine works by downregulating the CHNDA-2 pathway in the calcium channel gateways in the human brain. The target population was a patients in the United States who had been classified as suffering from chronic pain for at least three months. Patients were assessed on the clinical standard 100 point Blob-Allen pain scale before and after intervention. Compared to the placebo group, patients treated with Azadravapine saw a reduction in pain by an average of 6 points on the scale, representing a 0.4 ± 0.3 standard deviation decrease in pain level, with no evidence of treatment effect heterogeneity. The results reach statistical significance at conventional levels but the clinical significance of the effect size is uncertain.

This is surprising because Azadravapine is a drug that everyone knows works. Clinicians know it when they prescribe it and see immediate, miraculous reductions in pain. Patients know it when it allows them to function after years of trying to get through a day without considering suicide. And yet the gold standard RCTs are saying it has a treatment effect of just 0.4 standard deviations, the difference between a moderate annoyance and a slightly less moderate annoyance.

Furthermore, it doesn’t look like the drugs work strongly for some people and not others, which would be easier to believe. If that were true, we would expect to see the variance of pain in the treatment group to be greater than in the placebo group, but the variance looks about equal. I want to highlight this because it’s the most commonly given explanation for a low average treatment effect-that the drug works really well for some people but not others, which is all averaged together. Studies generally don't find evidence of this, however (this is an example for antidepressants).^[1]

I made up the drug Azadravapine, but as Scott Alexander has noted, this is a common occurrence for real drugs, including those whose effects are nearly impossible to doubt like benzodiazepines. What might be happening here? Could clinicians and patients really be totally deluded by the placebo effect, attributing miraculous changes to the drug when really all you need is a placebo?

Maybe that’s true in some cases, and mayb...

Posts

Wikitag Contributions

Comments