The following happened to me in an IRC chatroom, long enough ago that I was still hanging around in IRC chatrooms. Time has fuzzed the memory and my report may be imprecise.
So there I was, in an IRC chatroom, when someone reports that a friend of his needs medical advice. His friend says that he’s been having sudden chest pains, so he called an ambulance, and the ambulance showed up, but the paramedics told him it was nothing, and left, and now the chest pains are getting worse. What should his friend do?
I was confused by this story. I remembered reading about homeless people in New York who would call ambulances just to be taken someplace warm, and how the paramedics always had to take them to the emergency room, even on the 27th iteration. Because if they didn’t, the ambulance company could be sued for lots and lots of money. Likewise, emergency rooms are legally obligated to treat anyone, regardless of ability to pay.1 So I didn’t quite understand how the described events could have happened. Anyone reporting sudden chest pains should have been hauled off by an ambulance instantly.
And this is where I fell down as a rationalist. I remembered several occasions where my doctor would completely fail to panic at the report of symptoms that seemed, to me, very alarming. And the Medical Establishment was always right. Every single time. I had chest pains myself, at one point, and the doctor patiently explained to me that I was describing chest muscle pain, not a heart attack. So I said into the IRC channel, “Well, if the paramedics told your friend it was nothing, it must really be nothing—they’d have hauled him off if there was the tiniest chance of serious trouble.”
Thus I managed to explain the story within my existing model, though the fit still felt a little forced . . .
Later on, the fellow comes back into the IRC chatroom and says his friend made the whole thing up. Evidently this was not one of his more reliable friends.
I should have realized, perhaps, that an unknown acquaintance of an acquaintance in an IRC channel might be less reliable than a published journal article. Alas, belief is easier than disbelief; we believe instinctively, but disbelief requires a conscious effort.2
So instead, by dint of mighty straining, I forced my model of reality to explain an anomaly that never actually happened. And I knew how embarrassing this was. I knew that the usefulness of a model is not what it can explain, but what it can’t. A hypothesis that forbids nothing, permits everything, and thereby fails to constrain anticipation.
Your strength as a rationalist is your ability to be more confused by fiction than by reality. If you are equally good at explaining any outcome, you have zero knowledge.
We are all weak, from time to time; the sad part is that I could have been stronger. I had all the information I needed to arrive at the correct answer, I even noticed the problem, and then I ignored it. My feeling of confusion was a Clue, and I threw my Clue away.
I should have paid more attention to that sensation of still feels a little forced. It’s one of the most important feelings a truthseeker can have, a part of your strength as a rationalist. It is a design flaw in human cognition that this sensation manifests as a quiet strain in the back of your mind, instead of a wailing alarm siren and a glowing neon sign reading:
Either Your Model Is False Or This Story Is Wrong.
1 And the hospital absorbs the costs, which are enormous, so hospitals are closing their emergency rooms . . . It makes you wonder what’s the point of having economists if we’re just going to ignore them.
2 From McCluskey (2007), “Truth Bias”: “[P]eople are more likely to correctly judge that a truthful statement is true than that a lie is false. This appears to be a fairly robust result that is not just a function of truth being the correct guess where the evidence is weak—it shows up in controlled experiments where subjects have good reason not to assume truth[.]” http://www.overcomingbias.com/2007/08/truth-bias.html .
And from Gilbert et al. (1993), “You Can’t Not Believe Everything You Read”: “Can people comprehend assertions without believing them? [...] Three experiments support the hypothesis that comprehension includes an initial belief in the information comprehended.”
Reminds me of a family dinner where the topic of the credit union my grandparents had started came up.
According to my grandmother, the state auditor was a horribly sexist fellow. He came and audited their books every single month, telling everyone who would listen that it was because he "didn't think a woman could be a successful credit union manager."
This, of course, got my new-agey aunts and cousins all up-in-arms about how horrible it was that that kind of sexism was allowed back in the 60s and 70s. They really wanted to make sure everyone knew they didn't approve, so the conversation dragged on and on...
And about the time everyone was all thoroughly riled up and angry from the stories of the mean, vindictive things this auditor had done because the credit union was run by a woman my grandfather decided to get in on the ruckus and told his story about the auditor...
Seems like the very first time the auditor had come through, the auditor spent several hours going over the books and couldn't make it all balance correctly. He was all-fired sure this brand new credit union was up to something shady. Finally, my grandfather (who was the credit union accountant) leaned over his shoulder and pointed out the rookie math mistake the auditor had been making... repeatedly... until an hour past closing time and "could we please go home now?"
The auditor was horribly embarrassed, and stormed out in a huff. And then proceeded to come back every single month for over twenty years trying to catch them in a mistake somewhere.
I don't know if my cousins learned anything from that story. My grandfather's a quiet fellow. They might not even have heard his side of it. But I sure did. See, in the 60s and 70s, the auditor coming out and saying, "I'm harassing you because you humiliated me and I want revenge" would have been totally unacceptable and likely would have gotten him dismissed. But saying it was because he didn't trust a female manager? That was a lie, but it was a socially acceptable reason for doing what he wanted to do for personal reasons anyway.
Makes me wonder just how much historic racism and sexism was simply people looking for a socially acceptable excuse to be jerks. And since I don't think people's overall level of desire to be spiteful has changed much, I wonder what the excuses are today now that the "traditional" ones are no longer acceptable.