I think what confuses people is that he
1) claims that morality isn't arbitrary and we can make definitive statements about it
2) Also claims no universally compelling arguments.
The confusion is resolved by realizing that he defines the words "moral" and "good" as roughly equivalent to human CEV.
So according to Eliezer, it's not that Humans think love, pleasure, and equality is Good and paperclippers think paperclips are Good. It's that love, pleasure, and equality are part of the definition of good, while paperclips are just part of the definition of paperclippy. The Paperclipper doesn't think paperclips are good...it simply doesn't care about good, instead pursuing paperclippy.
Thus, moral relativism can be decried while "no universally compelling arguments" can be defended. Under this semantic structure, Paperclipper will just say "okay, sure...killing is immoral, but I don't really care as long as it's paperclippy."
Thus, arguments about morality among humans are analogous to Pebblesorter arguments about which piles are correct. In both cases, there is a correct answer.
It's an entirely semantic confusion.
I suggest that ethicists aught to hav...
I think what confuses people is that he 1) claims that morality isn't arbitrary and we can make definitive statements about it 2) Also claims no universally compelling arguments.
How does this differ from gustatory preferences?
1a) My preference for vanilla over chocolate ice cream is not arbitrary -- I really do have that preference, and I can't will myself to have a different one, and there are specific physical causes for my preference being what it is. To call the preference 'arbitrary' is like calling gravitation or pencils 'arbitrary', and carries no sting.
1b) My preference is physically instantiated, and we can make definitive statements about it, as about any other natural phenomenon.
2) There is no argument that could force any and all possible minds to like vanilla ice cream.
I raise the analogy because it seems an obvious one to me, so I don't see where the confusion is. Eliezer views ethics the same way just about everyone intuitively view aesthetics -- as a body of facts that can be empirically studied and are not purely a matter of personal opinion or ad-hoc stipulation -- facts, though, that make ineliminable reference to the neurally encoded preferences of specific o...
I remain confused by Eliezer's metaethics sequence.
Both there and in By Which It May Be Judged, I see Eliezer successfully arguing that (something like) moral realism is possible in a reductionist universe (I agree), but he also seems to want to say that in fact (something like) moral realism actually obtains, and I don't understand what the argument for that is. In particular, one way (the way?) his metaethics might spit up something that looks a lot like moral realism is if there is strong convergence of values upon (human-ish?) agents receiving better information, time enough to work out contradictions in their values, etc. But the "strong convergence of values" thesis hasn't really been argued, so I remain unclear as to why Eliezer finds it plausible.
Basically, I read the metaethics sequence as asserting both things but arguing only for the first.
But I'm not sure about this. Perhaps because I was already familiar with the professional metaethics vocabulary when I read the sequence, I found Eliezer's vocabulary for talking about positions in metaethics confusing.
I meant to explore these issues in a vocabulary I find more clear, in my own metaethics sequence, but I still haven't got around to it. :(
Yeah, I meant to remain ambiguous about how wide Eliezer means to cast the net around agents. Maybe it's psychologically normal humans, maybe it's wider or narrower than that.
I suppose 'The psychological unity of humankind' is sort of an argument that value convergence is likely at least among humans, though it's more like a hand-wave. In response, I'd hand-wave toward Sobel (1999); Prinz (2007); Doring & Steinhoff (2009); Doring & Andersen (2009); Robinson (2009); Sotala (2010); Plunkett (2010); Plakias (2011); Egan (2012), all of which argue for pessimism about value convergence. Smith (1994) is the only philosophical work I know of that argues for optimism about value convergence, but there are probably others I just don't know about.
rigid designators
aside from a lot of arguing about definitions over whether Eliezer counts as a relativist.
I think these are in fact the whole story. Eliezer says loudly that he is a moral realist and not any sort of relativist, but his views amount to saying "Define good and bad and so forth in terms of what human beings, in fact, value; then, as a matter of objective fact, death and misery are bad and happiness and fun are good", which to many people sounds exactly like moral relativism plus terminological games; confusion ensues.
The reason Eliezer's views are commonly mistaken for relativism in the manner you describe is because most people do not have a good grasp on the difference between sense and reference(a difference that, to be fair, doesn't seem to be well explained anywhere). To elaborate:
"Define good and bad and so forth in terms of what human beings, in fact, value" sounds like saying that goodness depends on human values. This is the definition you get if you say "let 'good' mean 'human values'". But the actual idea is meant to be more analogous to this: assuming for the sake of argument that humans value cake, define "good" to mean cake. Obviously, under that definition, "cake is always good regardless of what humans value" is true. In that case "good" is a rigid designator for cake.
The difference is that "good" and "human values" are not synonymous. But they refer to the same thing, when you fully dereference them, namely {happiness, fun and so forth}. This is the difference between sense and reference, and it's why it is necessary to understand rigid designators.
There was one aspect of that which made intuitive sense to me, but which now that I think about it may not have been adequately explained, ever. Eliezer's position seems to be that from some universal reference frame human beings would be viewed as moral relativists. However it is a serious mistake to think that such universal frames exist! So we shouldn't even try to think from a universal frame. From within the confines of a single, specific reference frame, the experience of morality is that of a realist.
EDIT: Put differently, I think Eliezer might agree that there is a metaphorical stone tablet with the rules of morality spelled out - it's encoded in the information patterns of the 3 lbs of grey matter inside your skull. Maybe Eliezer would say that he is a "subjective realist" or something like that. This is strictly different from moral relativism, where choice of morality is more or less arbitrary. As a subjective realist your morality is different than your pebblesorter friend, but it's not arbitrary. You have only limited control over the morality that evolution and culture gifted you.
I think my confusion is less about understanding the view (assuming the Richard's rigid designator interpretation is accurate) and more everyone's insistence on calling it a moral realist view. It feels like everyone is playing word games to avoid being moral subjectivists. I don't know if it was all the arguing with theists or being annoyed with moral relativist social-justice types but somewhere along the way much of the Less Wrong crowd developed strong negative associations with the words used to describe varieties of moral anti-realism.
As far as I can tell most everyone here has the same descriptive picture of what is going on with ethics. There is this animal on planet Earth that has semi-ordered preferences about how the world should be and how things similar to that animal should act. Those of this species which speak the language called "English" write inscriptions like "morality" and "right and wrong" to describe these preferences. These preferences are the result of evolved instincts and cultural norms. Many members of this species have very similar preferences.
This seems like a straightforward description of ethical subjectivism -- the p...
It didn't seem terribly compelling to me, but whether that was a failure of understanding or not I can't really say.
For my own part, I'm perfectly content to say that we care about what we (currently) care about because we care about it, so all of this "moral miracle" stuff about how what we (currently) care about really is special seems unnecessary. I can sort of understand why it's valuable rhetorically when engaging with people who really want some kind of real true specialness in their values, but I mostly think such people should get over it.
I also tend to doubt that "what humanity currently cares about" is as internally consistent and coherently extrapolatable as much of the moral philosophy in the Sequences and elsewhere on this site would seem to imply.
I found it much clearer when I realised he was basically talking about rigid designation. It didn't help when EY started talking about rigid designation and using the terminology incorrectly.
Reference class: I studied academic philosophy.
It didn't help when EY started talking about rigid designation and using the terminology incorrectly.
I didn't notice that, can you elaborate?
Personally, I remain confused about his claim that morality is objective in some sense in The Bedrock of Morality: Arbitrary?, no matter how many times i reread it.
aside from a lot of arguing about definitions over whether Eliezer counts as a relativist
I think the whole point was to taboo "realist" and "relativist." So if people come out of the sequence arguing about those definitions, they don't seem to have gotten anything out of the sequence. So, yes, aside from everything, there's no other problem. But that doesn't help you narrow down the problem. I suspect this is either strong agreement or strong disagreement with gjm, but I don't know which.
Why didn't people (apparently?) understand the metaethics sequence?
Perhaps back up a little. Does the metaethics sequence make sense? As I remember it, a fair bit of it was a long, rambling and esoteric bunch of special pleading - frequently working from premises that I didn't share.
It's been a while since I read (part of) the metaethics sequence. With that said:
I have a pretty strong aversion to the word "right" used in discourse. The word is used to mean a few different things, and people often fail to define their use of it sufficiently for me to understand what they're talking about. I don't remember being able to tell whether Eliezer was attempting to make a genuine argument for moral-realism; when he introduced the seemingly sensical term h-right (recognizing that things humans often feel are "right" are simp...
Let's get some data (vote accordingly):
Did you understand the metaethics sequence, when you read it?
[pollid:572]
If you decide to write that post, it would be great if you started by describing the potential impact of metaethics on FAI design, to make sure that we're answering questions that need answering and aren't just confusions about words. If anyone wants to take a stab here in the comments, I'd be very interested.
Should we expect metaethics to affect normative ethics? Should people who care about behaving morally, therefore care about metaethics at all?
Put another way — Assume that there is a true, cognitivist, non-nihilist, metaethical theory M. (That is, M asserts that there exists at least one true moral judgment.) Do we expect that people who know or believe M will act more morally, or even have more accurate normative-ethical beliefs, than people who do not?
It's conceivable for metaethics to not affect normative ethics — by analogy to the metaphysics of mathem...
Reading the comments on the metaethics sequence, though, hasn't enlightened me about what exactly people had a problem with, aside from a lot of arguing about definitions over whether Eliezer counts as a relativist.
Since you (apparently) understand him., Chris, maybe you could settle the matter.
I didn't understand it the first time, probably because I hadn't yet fully absorbed A Human's Guide to Words, which it mimics heavily.
I recall not being able to identify with the premises... some of them were really quite significant.
I now recall, it was with "The Moral Void, in which apparently I had different answers than expected.
"Would you kill babies if it was inherently the right thing to do?"
The post did discuss morality on/off switches later in the context of religion, as an argument against (wishing for / wanting to find) universally compelling arguments.
The post doesn't work for me because it seems there is an argument against the value of universally compelling arguments with the implicit assumption that since universally compelling argument don't exist, any universally compelling argument would be false.
I happen to (mostly) agree that there aren't universally compelling arguments, but I still wish there were. The metaethics sequence failed to talk me out of valuing this.
Also, there were some particular examples that didn't work for me, since I didn't have a spontaneous 'ugh' field around some of the things that were supposed to be bad.
I see Jack expressed this concept here:
And it definitely is true that much of our moral language function like rigid designators, which hides the causal history of our moral beliefs. This explains why people don't feel like morality changes under counterfactuals-- i.e. if you imagine a world in which you have a preference for innocent children being murdered you don't believe that murdering children is therefore moral in that world. I outlined this in more detail here. I didn't use the term 'rigid designator' in that post, but the point is that what we think is moral is invariant in counterfacturals.
For whatever reason, I feel like my morality changes under counterfactuals.
I happen to (mostly) agree that there aren't universally compelling arguments, but I still wish there were. The metaethics sequence failed to talk me out of valuing this.
But you realize that Eliezer is arguing that there aren't universally compelling arguments in any domain, including mathematics or science? So if that doesn't threaten the objectivity of mathematics or science, why should that threaten the objectivity of morality?
For whatever reason, I feel like my morality changes under counterfactuals.
Can you elaborate?
There seems to be a widespread impression that the metaethics sequence was not very successful as an explanation of Eliezer Yudkowsky's views. It even says so on the wiki. And frankly, I'm puzzled by this... hence the "apparently" in this post's title. When I read the metaethics sequence, it seemed to make perfect sense to me. I can think of a couple things that may have made me different from the average OB/LW reader in this regard: