A Case Study of Motivated Continuation

by Eliezer Yudkowsky2 min read31st Oct 200736 comments


Personal Blog

I am not wholly unsympathetic to the many commenters in Torture vs. Dust Specks who argued that it is preferable to inflict dust specks upon the eyes of 3^^^3 (amazingly huge but finite number of) people, rather than torture one person for 50 years.  If you think that a dust speck is simply of no account unless it has other side effects - if you literally do not prefer zero dust specks to one dust speck - then your position is consistent.  (Though I suspect that many speckers would have expressed a preference if they hadn't known about the dilemma's sting.)

So I'm on board with the commenters who chose TORTURE, and I can understand the commenters who chose SPECKS.

But some of you said the question was meaningless; or that all morality was arbitrary and subjective; or that you needed more information before you could decide; or you talked about some other confusing aspect of the problem; and then you didn't go on to state a preference.

Sorry.  I can't back you on that one.

If you actually answer the dilemma, then no matter which option you choose, you're giving something up.  If you say SPECKS, you're giving up your claim on a certain kind of utilitarianism; you may worry that you're not being rational enough, or that others will accuse you of failing to comprehend large numbers.  If you say TORTURE, you're accepting an outcome that has torture in it.

I falsifiably predict that of the commenters who dodged, most of them saw some specific answer - either TORTURE or SPECKS - that they flinched away from giving.  Maybe for just a fraction of a second before the question-confusing operation took over, but I predict the flinch was there.  (To be specific:  I'm not predicting that you knew, and selected, and have in mind right now, some particular answer you're deliberately not giving.  I'm predicting that your thinking trended toward a particular uncomfortable answer, for at least one fraction of a second before you started finding reasons to question the dilemma itself.)

In "bioethics" debates, you very often see experts on bioethics discussing what they see as the pros and cons of, say, stem-cell research; and then, at the conclusion of their talk, they gravely declare that more debate is urgently needed, with participation from all stakeholders.  If you actually come to a conclusion, if you actually argue for banning stem cells, then people with relatives dying of Parkinson's will scream at you.  If you come to a conclusion and actually endorse stem cells, religious fundamentalists will scream at you.  But who can argue with a call to debate?

Uncomfortable with the way the evidence is trending on Darwinism versus creationism?  Consider the issue soberly, and decide that you need more evidence; you want archaeologists to dig up another billion fossils before you come to a conclusion.  That way you neither say something sacrilegious, nor relinquish your self-image as a rationalist.  Keep on doing this with all issues that look like they might be trending in an uncomfortable direction, and you can maintain a whole religion in your mind.

Real life is often confusing, and we have to choose anyway, because refusing to choose is also a choice.  The null plan is still a plan.  We always do something, even if it's nothing.  As Russell and Norvig put it, "Refusing to choose is like refusing to allow time to pass."

Ducking uncomfortable choices is a dangerous habit of mind.  There are certain times when it's wise to suspend judgment (for an hour, not a year).  When you're facing a dilemma all of whose answers seem uncomfortable, is not one of those times!  Pick one of the uncomfortable answers as the best of an unsatisfactory lot.  If there's missing information, fill in the blanks with plausible assumptions or probability distributions.  Whatever it takes to overcome the basic flinch away from discomfort.  Then you can search for an escape route.

Until you pick one interim best guess, the discomfort will consume your attention, distract you from the search, tempt you to confuse the issue whenever your analysis seems to trend in a particular direction.

In real life, when people flinch away from uncomfortable choices, they often hurt others as well as themselves.  Refusing to choose is often one of the worst choices you can make.  Motivated continuation is not a habit of thought anyone can afford, egoist or altruist.  The cost of comfort is too high.  It's important to acquire that habit of gritting your teeth and choosing - just as important as looking for escape routes afterward.