Thanks for the links, I'll take a look in particular at the Benedetti et al. stuff.

The IBS study was already the topic of the previous, linked discussion so I don't expect to get any more out of it - I've already explained why it (and the excerpt you bolded above) are not sufficient grounds for the claim "there is such a thing as the placebo effect".

Alice: Yeah, I had that problem for years

Has Alice also had two back surgeries? Otherwise, I'd assign more probability to the hypothesis that their differential response to pain meds has something to do with their different medical histories.

A definition closer to my intended meaning would be something like "the effect caused by the perceptions and expectations of the patient, rather than the particular substance being administered in the treatment".

I find this confusing: in the relevant experimental setups, nobody is directly manipulating (intervening on) "the perceptions and expectations of the patient". These perceptions and expectations are themselves an effect, usually not measured except indirectly through the patient's self-reports and voluntarily controlled responses.

What is being manipulated in the studies I've seen (e.g. the IBS study) is a specific experimental condition, namely the type of treatment being administered: either a particular substance or a particular therapeutic intervention or gesture.

When people talk about placebo effect, I take them to mean something like "the mysterious healing power of the mind"; that is, they predict that there is a reliable causal effect of holding particular beliefs, other than voluntarily selecting behavioural responses consistent with the beliefs, in particular an amelioration of some pathology or a mitigation of some symptoms. In the IBS studies and in the Dougherty case, the effects observed are all voluntarily selected responses - there is nothing more mysterious at work than the desire to behave or respond in ways consistent with a professed belief.

Ingesting an inert pill (vs a similar pill with paracetamol) doesn't make my headache or back ache go away, it just makes me less likely to verbally report pain than I would otherwise have. What really takes care of the pain is the paracetamol, and that's what I happen to be interested in.

Playing games with my own expectations, so as to elicit behaviour that is already under my control, is of limited value - if I want to shut up about the pain, I can just decide to shut up about the pain. (I would predict that a study comparing spontaneous pain reports under differential treatments of "Verbal instruction to suppress pain by not reporting it" and "Administration of inert pill" would favor verbal instruction in a wide range of pain-causing injuries or pathologies.)

Knowledge value = knowledge quality × domain importance

by John_Maxwell 1 min read16th Apr 201241 comments


Months ago, my roommate and I were discussing someone who had tried to replicate Seth Roberts' butter mind self-experiment. My roommate seemed to be making almost no inference from the person's self-reports, because they weren't part of a scientific study.

But knowledge does not come in two grades, "scientific" and "useless". Anecdotes do count as evidence, they are just weak evidence. And well designed scientific studies constitute stronger evidence then poorly designed studies. There's a continuum for knowledge quality.

Knowing that humans are biased should make us take their stories and ad hoc inferences less seriously, but not discard them altogether.

There exists some domains where most of our knowledge is fairly low-quality. But that doesn't mean they're not worth study, if the value of information in the domain is high.

For example, a friend of mine read a bunch of books on negotiation and says this is the best one. Flipping through my copy, it looks like the author is mostly just enumerating his own thoughts, stories, and theories. So one might be tempted to discard the book entirely because it isn't very scientific.

But that would be a mistake. If a smart person thinks about something for a while and comes to a conclusion, that's decent-quality evidence that the conclusion is correct. (If you disagree with me on this point, why do you think about things?)

And the value of information in the domain of negotiation can be very high: If you're a professional, being able to negotiate your salary better can net you hundreds of thousands over the course of a career. (Anchoring means your salary next year will probably just be an incremental raise from your salary last year, so starting salary is very important.)

Similarly, this self-help book is about as dopey and unscientific as they come. But doing one of the exercises from it years ago destroyed a large insecurity of mine that I was only peripherally aware of. So I probably got more out of it in instrumental terms than I would've gotten out of a chemistry textbook.

In general, self-improvement seems like a domain of really high importance that's unfortunately flooded with low-quality knowledge. If you invest two hours implementing some self-improvement scheme and find yourself operating 10% more effectively, you'll double your investment in just a week, assuming a 40 hour work week. (ALERT: this seems like a really important point! I'd write an entire post about it, but I'm not sure what else there is to say.)

Here are some free self-improvement resources where the knowledge quality seems at least middling: For people who feel like failuresFor students. For mathematiciansProductivity and general ass kicking (web implementation for that last idea). Even more ass kicking ideas that you might have seen already.