Unspeakable Morality

by Eliezer Yudkowsky6 min read4th Aug 2009119 comments


Ethics & Morality
Personal Blog

It is a general and primary principle of rationality, that we should not believe that which there is insufficient reason to believe; likewise, a principle of social morality that we should not enforce upon our fellows a law which there is insufficient justification to enforce.

Nonetheless, I've always felt a bit nervous about demanding that people be able to explain things in words, because, while I happen to be pretty good at that, most people aren't.

"I remember this paper I wrote on existentialism. My teacher gave it back with an F. She’d underlined true and truth wherever it appeared in the essay, probably about twenty times, with a question mark beside each. She wanted to know what I meant by truth." —Danielle Egan (journalist)

This experience permanently traumatized Ms. Egan, by the way.  Because years later, at a WTA conference, one of the speakers said that something was true, and Ms. Egan said "What do you mean, 'true'?", and the speaker gave some incorrect answer or other; and afterward I quickly walked over to Ms. Egan and explained the correspondence theory of truth:  "The sentence 'snow is white' is true if and only if snow is white"; if you're using a bucket of pebbles to count sheep then an empty bucket is true if and only if the pastures are empty.  I don't know if this cured her; I suspect that it didn't.  But up until that point, at any rate, it seems Ms. Egan had been so traumatized by this childhood experience that she believed there was no such thing as truth - that because her teacher had demanded a definition in words, and she hadn't been able to give a good definition in words, that no good definition existed.

Of which I usually say:  "There was a time when no one could define gravity in exquisitely rigorous detail, but if you walked off a cliff, you would fall."

On the other hand - it is a general and primary principle of rationality that when you have no justification, it is very important that there be some way of saying "Oops", losing hope, and just giving up already.  (I really should post, at some point, on how the ability to just give up already is one of the primary distinguishing abilities of a rationalist.)  So, really, if you find yourself totally unable to justify something in words, one possibility is that there is no justification.  To ignore this and just casually stroll along, would not be a good thing.

And with moral questions, this problem is doubled and squared.  For any given person, the meaning of "right" is a huge complicated function, not explicitly believed so much as implicitly embodied.  And if we keep asking "Why?", at some point we end up replying "Because that is just what the term 'right', means; there is no pure essence of rightness that you can abstract away from the specific content of your values."

But if you were allowed to answer this in response to any demand for justification, and have the other bow and walk away - well, you would no longer be computing what we know as morality, where 'right' does mean some things and not others.

Not to mention that in questions of public policy, it ought to require some overlap in values to make a law.  I do think that human values often overlap enough that different people can legitimately use the same word 'right' to refer to that-which-they-compute.  But if someone wants a legal ban on pepperoni pizza because it's inherently wrong, then I may feel impelled to ask, "Why do you think this is part of the overlap in our values?"

Demands for moral justification have their Charybdis and their Scylla:

The traditionally given Charybdis is letting someone say that interracial marriage should be legally banned because it "feels icky" to them.  We could call this "the unwisdom of repugnance" - if you can just say "That feels repugnant" and win a case for public intervention, then you lose all the cases of what we now regard as tremendous moral progress, which made someone feel vaguely icky at the time; women's suffrage, divorces, atheists not being burned at the stake.  Moral progress - which I currently see as an iterative process of learning new facts, processing new arguments, and becoming more the sort of person you wished you were - demands that people go on thinking about morality, for which purpose it is very useful to have people go on arguing about morality.  If saying the word "intuition" is a moral trump card, then people, who, by their natures, are lazy, will just say "intuition!" all the time, believing that no one is allowed to question that or argue with it; and that will be the end of their moral thinking.

And the Scylla, I think, was excellently presented by Silas Barta when... actually this whole comment is just worth quoting directly:

Let's say we're in an alternate world with strong, codified rules about social status and authority, but weak, vague, unspoken norms against harm that nevertheless keep harm at a low level.

Then let's say you present the people of this world with this "dilemma" to make Greene's point:

Say your country is at war with another country that is particularly aggressive and willing to totally demolish your social order and enslave your countrymen. In planning how to best fight off this threat, your President is under a lot of stress. To help him relieve his stress, he orders a citizen, Bob, to be brought before him and tortured and murdered, while the President laughs his head off at the violence.

He feels much more relieved and so is able to craft and motivate a war plan that leads to the unconditional surrender of the enemy. The President promises that this was just a one-time thing he had to do to handle the tremendous pressure he was under to win the war and protect his people. Bob's family, in turn, says that they are honored by the sacrifice Bob has made for his country. Everyone agrees that the President is the legitimate ruler of the country and the Constitution and tradition give him authority to do what he did to Bob.

Was it okay for the President to torture and kill Bob for his personal enjoyment?

Then, because of the deficiency in the vocabulary of "harms", you would get responses like:

"Look, I can't explain why, but obviously, it's wrong to torture and kill someone for enjoyment. No disrespect to the President, of course."

"What? I don't get it. Why would the President order a citizen killed? There would be outrage. He'd feel so much guilt that it wouldn't even relieve the stress you claim it does."

"Yeah, I agree the President has authority to do that, but God, it just burns me up to think about someone getting tortured like that for someone else's enjoyment, even if it is our great President."

Would you draw the same conclusion Greene does about these responses?

Unfortunately, it does happen to be a fact that most people are not good at explaining themselves in words, unless they've already heard the explanation from someone else.  Even if you challenge a professional philosopher who holds a position, to justify it, and they can't... well, frankly, you can't conclude much even from that, in terms of inferring that no good explanation exists.  Philosophers, I've observed, are not much good at this sort of job either.  It's Bayesian evidence, by the law of conservation of evidence; if a good explanation would be a sign that justification exists, then the absence of such explanation must be evidence that justification does not exist.  It's just not very strong evidence, because we don't strongly anticipate that even professional philosophers will be able to put a justification into words, correctly and convincingly, when justification does in fact exist.

Even conditioning on the proposition that there is overlap in what you and others mean by 'right' - the huge function that is what-we-try-to-do - and that the judgment in question is stable when taken to the limits of knowledge, thought, and reflective coherence - well, it's still not sure that you'd be able to put it into words.  You might be able to.  But you might not.

And we also have to allow a certain probability of convincing-sounding complicated verbal justification, in cases where no justification exists.  But then if you use that as an excuse to flush all disliked arguments down the toilet, you shall be left rotting forever in a pit of convenient skepticism, saying, "All that intellekshual stuff could be wrong, after all."

So here are my proposed rules of conduct for arguing morality in words:

  • "Intuition" is not a trump card.  If you had to spell out what your intuition was, and where it came from (evolution? culture?), and whether it has consequences beyond itself, it's possible that we would find it unconvincing in the stark light of reflection; that we would wish to intuit some other intuition than this.  We can't hold up the intuition for reflective judgment unless we know what it is.  So spelling it out, is important; and if you can win arguments by saying "Intuition!" then no one will bother to spell things out any more.  Please try to say what sort of intuition it is.
  • "I can't put it into words" is believable to some extent, but constitutes weak evidence against the existence of valid justification.  If this is a popular debate and no one on your side, politician or philosopher or interested scientist or eloquent blogger, is able to give a convincing justification in words, then that is stronger evidence that no good justification exists.  The longer the failure continues, the stronger the evidence.
  • Still, at the end of the day, we don't really expect people to be very good at verbalizing moral intuitions, especially since most of them have incoherent explicit metaethics.  So if you can give a justification for your political policy that stutters off into incoherence only at the point of explaining why pain is a bad thing - if you can give reasonable arguments for everything else up until that point - that's probably about as much as we can demand of anyone short of a full-fledged master reductionist.
  • But we also expect that people may pass judgments that they would revoke in the light of better information or new arguments; and, especially before passing to that limit, it may be that sociopaths do not overlap with the values shared by most in a society.  So if A says that event B is inherently wrong and awful, and C disagrees on the grounds that it just doesn't seem all that awful to them, then the burden of argument needs to lie on A before any social, legal, public action is brought into play.  We should bear in mind that people of the past would have a lot of icky feelings about things that we, today, think are not only permitted but virtuous or even mandatory - the challenging of these icky feelings for good and sufficient public justification, was a key element of their relinquishment, which we regard as moral progress.
Ethics & Morality2
Personal Blog