Is Morality Preference?


21


Eliezer_Yudkowsky

Followup toMoral Complexities

In the dialogue "The Bedrock of Fairness", I intended Yancy to represent morality-as-raw-fact, Zaire to represent morality-as-raw-whim, and Xannon to be a particular kind of attempt at compromising between them.  Neither Xannon, Yancy, or Zaire represent my own views—rather they are, in their disagreement, showing the problem that I am trying to solve.  It is futile to present answers to which questions are lacking.

But characters have independent life in the minds of all readers; when I create a dialogue, I don't view my authorial intent as primary.  Any good interpretation can be discussed.  I meant Zaire to be asking for half the pie out of pure selfishness; many readers interpreted this as a genuine need... which is as interesting a discussion to have as any, though it's a different discussion.

With this in mind, I turn to Subhan and Obert, who shall try to answer yesterday's questions on behalf of their respective viewpoints.

Subhan makes the opening statement:

Subhan:  "I defend this proposition: that there is no reason to talk about a 'morality' distinct from what people want."

Obert:  "I challenge.  Suppose someone comes to me and says, 'I want a slice of that pie you're holding.'  It seems to me that they have just made a very different statement from 'It is right that I should get a slice of that pie'.  I have no reason at all to doubt the former statement—to suppose that they are lying to me about their desires.  But when it comes to the latter proposition, I have reason indeed to be skeptical.  Do you say that these two statements mean the same thing?"

Subhan:  "I suggest that when the pie-requester says to you, 'It is right for me to get some pie', this asserts that you want the pie-requester to get a slice."

Obert:  "Why should I need to be told what I want?"

Subhan:  "You take a needlessly restrictive view of wanting, Obert; I am not setting out to reduce humans to creatures of animal instinct.  Your wants include those desires you label 'moral values', such as wanting the hungry to be fed—"

Obert:  "And you see no distinction between my desire to feed the hungry, and my desire to eat all the delicious pie myself?"

Subhan:  "No!  They are both desires—backed by different emotions, perhaps, but both desires.  To continue, the pie-requester hopes that you have a desire to feed the hungry, and so says, 'It is right that I should get a slice of this pie', to remind you of your own desire.  We do not automatically know all the consequences of our own wants; we are not logically omniscient."

Obert:  "This seems psychologically unrealistic—I don't think that's what goes through the mind of the person who says, 'I have a right to some pie'.  In this latter case, if I deny them pie, they will feel indignant.  If they are only trying to remind me of my own desires, why should they feel indignant?"

Subhan:  "Because they didn't get any pie, so they're frustrated."

Obert:  "Unrealistic!  Indignation at moral transgressions has a psychological dimension that goes beyond struggling with a struck door."

Subhan:  "Then consider the evolutionary psychology.  The pie-requester's emotion of indignation would evolve as a display, first to remind you of the potential consequences of offending fellow tribe-members, and second, to remind any observing tribe-members of goals they may have to feed the hungry.  By refusing to share, you would offend against a social norm—which is to say, a widely shared want."

Obert:  "So you take refuge in social wants as the essence of morality?  But people seem to see a difference between desire and morality, even in the quiet of their own minds.  They say things like:  'I want X, but the right thing to do is Y... what shall I do?'"

Subhan:  "So they experience a conflict between their want to eat pie, and their want to feed the hungry—which they know is also a want of society.  It's not predetermined that the prosocial impulse will be victorious, but they are both impulses."

Obert:  "And when, during WWII, a German hides Jews in their basement—against the wants of surrounding society—how then?"

Subhan:  "People do not always define their in-group by looking at their next-door neighbors; they may conceive of their group as 'good Christians' or 'humanitarians'."

Obert:  "I should sooner say that people choose their in-groups by looking for others who share their beliefs about morality—not that they construct their morality from their in-group."

Subhan:  "Oh, really?  I should not be surprised if that were experimentally testable—if so, how much do you want to bet?"

Obert:  "That the Germans who hid Jews in their basements, chose who to call their people by looking at their beliefs about morality?  Sure.  I'd bet on that."

Subhan:  "But in any case, even if a German resister has a desire to preserve life which is so strong as to go against their own perceived 'society', it is still their desire."

Obert:  "Yet they would attribute to that desire, the same distinction they make between 'right' and 'want'—even when going against society.  They might think to themselves, 'How dearly I wish I could stay out of this, and keep my family safe.  But it is my duty to hide these Jews from the Nazis, and I must fulfill that duty.'  There is an interesting moral question, as to whether it reveals greater heroism, to fulfill a duty eagerly, or to fulfill your duties when you are not eager.  For myself I should just total up the lives saved, and call that their score.  But I digress...  The distinction between 'right' and 'want' is not explained by your distinction of socially shared and individual wants.  The distinction between desire and duty seems to me a basic thing, which someone could experience floating alone in a spacesuit a thousand light-years from company."

Subhan:  "Even if I were to grant this psychological distinction, perhaps that is simply a matter of emotional flavoring. Why should I not describe perceived duties as a differently flavored want?"

Obert:  "Duties, and should-ness, seem to have a dimension that goes beyond our whims.  If we want different pizza toppings today, we can order a different pizza without guilt; but we cannot choose to make murder a good thing."

Subhan:  "Schopenhauer:  'A man can do as he wills, but not will as he wills.'  You cannot decide to make salad taste better to you than cheeseburgers, and you cannot decide not to dislike murder.  Furthermore, people do change, albeit rarely, those wants that you name 'values'; indeed they are easier to change than our food tastes."

Obert:  "Ah!  That is something I meant to ask you about.  People sometimes change their morals; I would call this updating their beliefs about morality, but you would call it changing their wants.  Why would anyone want to change their wants?"

Subhan:  "Perhaps they simply find that their wants have changed; brains do change over time.  Perhaps they have formed a verbal belief about what they want, which they have discovered to be mistaken. Perhaps society has changed, or their perception of society has changed.  But really, in most cases you don't have to go that far, to explain apparent changes of morality."

Obert:  "Oh?"

Subhan:  "Let's say that someone begins by thinking that Communism is a good social system, has some arguments, and ends by believing that Communism is a bad social system.  This does not mean that their ends have changed—they may simply have gotten a good look at the history of Russia, and decided that Communism is a poor means to the end of raising standards of living.  I challenge you to find me a case of changing morality in which people change their terminal values, and not just their beliefs about which acts have which consequences."

Obert:  "Someone begins by believing that God ordains against premarital sex; they find out there is no God; subsequently they approve of premarital sex.  This, let us specify, is not because of fear of Hell; but because previously they believed that God had the power to ordain, or knowledge to tell them, what is right; in ceasing to believe in God, they updated their belief about what is right."

Subhan:  "I am not responsible for straightening others' confusions; this one is merely in a general state of disarray around the 'God' concept."

Obert:  "All right; suppose I get into a moral argument with a man from a society that practices female circumcision.  I do not think our argument is about the consequences to the woman; the argument is about the morality of these consequences."

Subhan:  "Perhaps the one falsely believes that women have no feelings—"

Obert:  "Unrealistic, unrealistic!  It is far more likely that the one hasn't really considered whether the woman has feelings, because he doesn't see any obligation to care.  The happiness of women is not a terminal value to him.  Thousands of years ago, most societies devalued consequences to women.  They also had false beliefs about women, true—and false beliefs about men as well, for that matter—but nothing like the Victorian era's complex rationalizations for how paternalistic rules really benefited women. The Old Testament doesn't explain why it levies the death penalty for a woman wearing men's clothing.  It certainly doesn't explain how this rule really benefits women after all.  It's not the sort of argument it would have occurred to the authors to rationalize!  They didn't care about the consequences to women."

Subhan:  "So they wanted different things than you; what of it?"

Obert:  "See, now that is exactly why I cannot accept your viewpoint.  Somehow, societies went from Old Testament attitudes, to democracies with female suffrage.  And this transition—however it occurred—was caused by people saying, 'What this society does to women is a great wrong!', not, 'I would personally prefer to treat women better.'  That's not just a change in semantics—it's the difference between being obligated to stand and deliver a justification, versus being able to just say, 'Well, I prefer differently, end of discussion.'  And who says that humankind has finished with its moral progress?  You're yanking the ladder out from underneath a very important climb."

Subhan:  "Let us suppose that the change of human societies over the last ten thousand years, has been accompanied by a change in terminal values—"

Obert:  "You call this a supposition?  Modern political debates turn around vastly different valuations of consequences than in ancient Greece!"

Subhan:  "I am not so sure; human cognitive psychology has not had time to change evolutionarily over that period.  Modern democracies tend to appeal to our empathy for those suffering; that empathy existed in ancient Greece as well, but it was invoked less often.  In each single moment of argument, I doubt you would find modern politicians appealing to emotions that didn't exist in ancient Greece."

Obert:  "I'm not saying that emotions have changed; I'm saying that beliefs about morality have changed.  Empathy merely provides emotional depth to an argument that can be made on a purely logical level:  'If it's wrong to enslave you, if it's wrong to enslave your family and your friends, then how can it be right to enslave people who happen to be a different color?  What difference does the color make?'  If morality is just preference, then there's a very simple answer:  'There is no right or wrong, I just like my own family better.'  You see the problem here?"

Subhan:  "Logical fallacy:  Appeal to consequences."

Obert:  "I'm not appealing to consequences.  I'm showing that when I reason about 'right' or 'wrong', I am reasoning about something that does not behave like 'want' and 'don't want'."

Subhan:  "Oh?  But I think that in reality, your rejection of morality-as-preference has a great deal to do with your fear of where the truth leads."

Obert:  "Logical fallacy:  Ad hominem."

Subhan:  "Fair enough.  Where were we?"

Obert:  "If morality is preference, why would you want to change your wants to be more inclusive?  Why would you want to change your wants at all?"

Subhan:  "The answer to your first question probably has to do with a fairness instinct, I would suppose—a notion that the tribe should have the same rules for everyone."

Obert:  "I don't think that's an instinct.  I think that's a triumph of three thousand years of moral philosophy."

Subhan:  "That could be tested."

Obert:  "And my second question?"

Subhan:  "Even if terminal values change, it doesn't mean that terminal values are stored on a great stone tablet outside humanity.  Indeed, it would seem to argue against it!  It just means that some of the events that go on in our brains, can change what we want."

Obert:  "That's your concept of moral progress?  That's your view of the last three thousand years?  That's why we have free speech, democracy, mass street protests against wars, nonlethal weapons, no more slavery—"

Subhan:  "If you wander on a random path, and you compare all past states to your present state, you will see continuous 'advancement' toward your present condition—"

Obert:  "Wander on a random path?"

Subhan:  "I'm just pointing out that saying, 'Look how much better things are now', when your criterion for 'better' is comparing past moral values to yours, does not establish any directional trend in human progress."

Obert:  "Your strange beliefs about the nature of morality have destroyed your soul.  I don't even believe in souls, and I'm saying that."

Subhan:  "Look, depending on which arguments do, in fact, move us, you might be able to regard the process of changing terminal values as a directional progress.  You might be able to show that the change had a consistent trend as we thought of more and more arguments.  But that doesn't show that morality is something outside us.  We could even—though this is psychologically unrealistic—choose to regard you as computing a converging approximation to your 'ideal wants', so that you would have meta-values that defined both your present value and the rules for updating them.  But these would be your meta-values and your ideals and your computation, just as much as pepperoni is your own taste in pizza toppings.  You may not know your real favorite ever pizza topping, until you've tasted many possible flavors."

Obert:  "Leaving out what it is that you just compared to pizza toppings, I begin to be suspicious of the all-embracingness of your viewpoint.  No matter what my mind does, you can simply call it a still-more-modified 'want'.  I think that you are the one suffering from meta-level confusion, not I.  Appealing to right is not the same as appealing to desire.  Just because the appeal is judged inside my brain, doesn't mean that the appeal is not to something more than my desires.  Why can't my brain compute duties as well as desires?"

Subhan:  "What is the difference between duty and desire?"

Obert:  "A duty is something you must do whether you want to or not."

Subhan:  "Now you're just being incoherent.  Your brain computes something it wants to do whether it wants to or not?"

Obert:  "No, you are the one whose theory makes this incoherent.  Which is why your theory ultimately fails to add up to morality."

Subhan:  "I say again that you underestimate the power of mere wanting.  And more:  You accuse me of incoherence?  You say that I suffer from meta-level confusion?"

Obert:  "Er... yes?"

To be continued...

 

Part of The Metaethics Sequence

Next post: "Is Morality Given?"

Previous post: "Moral Complexities"