It almost annoys me, but I feel compelled to vote this up. (I know groundbreaking philosophy is not yet your intended purpose but) I didn't learn anything, I remain worried that the sequence is going to get way too ambitious, and I remain confused about where it's ultimately headed. But the presentation is so good -- clear language, straightforward application of LW wisdom, excellent use of hyperlinks, high skimmability, linked references, flattery of my peer group -- that I feel I have to support the algorithm that generated it.
Most of your comment looks as though it could apply just as well to the most upvoted post on LW ever (edit: second-most-upvoted), and that's good enough for me. :)
There are indeed many LW regulars, and especially SI folk, who won't learn anything from several posts in this series. On the other hand, I think that these points haven't been made clear (about morality) anywhere else. I hope that when people (including LWers) start talking about morality with the usual conceptual-analysis assumptions, you can just link them here and dissolve the problem.
Also, it sounds like you agree with everything in this fairly long post. If so, yours is faint criticism indeed. :)
Looking back at your posts in this sequence so far, it seems like it's taken you four posts to say "Philosophers are confused about meta-ethics, often because they spend a lot of time disputing defintions." I guess they've been well-sourced, which is worth something. But it seems like we're still waiting on substantial new insights about metaethics, sadly.
I admit it's not very fun for LW regulars, but a few relatively short and simple posts is probably the bare minimum you can get away with while still potentially appealing to bright philosopher or academic types, who will be way more hesitant than your typical contrarian to dismiss an entire field of philosophy as not even wrong. I think Luke's doing a decent job of making his posts just barely accessible/interesting to a very wide audience.
it seems like it's taken you four posts to say "Philosophers are confused about meta-ethics, often because they spend a lot of time disputing defintions."
No, he said quite a lot more. E.g. why philosophers do that, why it is a bad thing, and what to do about it if we don't want to fall into the same trap. This is all neccessary ground work for his final argument.
If the state of metaethics were such that most people would already agree on these fundamentals then you would have a point, but lukeprog's premise is that it's not.
An interesting phenomenon I've noticed recently is that sometimes words do have short exact definitions that exactly coincide with common usage and intuition. For example, after Gettier scenarios ruined the definition of knowledge as "Justified true belief", philosophers found a new definition:
"A belief in X is knowledge if one would always have that belief whenever X, and never have it whenever not-X".
(where "always" and "never" are defined to be some appropriate significance level)
Now it seems to me that this definition completely nails it. There's not one scenario I can find where this definition doesn't return the correct answer. (EDIT: Wrong! See great-grandchild by Tyrrell McAllister) I now feel very silly for saying things like "'Knowledge' is a fuzzy concept, hard to carve out of thingspace, there's is always going to be some scenario that breaks your definition." It turns out that it had a nice definition all along.
It seems like there is a reason why words tend to have short definitions: the brain can only run short algorithms to determine whether an instance falls into the category or not. All you've got to do to write the definition is to find this algorithm.
Yep. Another case in point of the danger of replying, "Tell me how you define X, and I'll tell you the answer" is Parfit in Reason and Persons concluding that whether or not an atom-by-atom duplicate constructed from you is "you" depends on how you define "you". Actually it turns out that there is a definite answer and the answer is knowably yes, because everything Parfit reasoned about "indexical identity" is sheer physical nonsense in a world built on configurations and amplitudes instead of Newtonian billiard balls.
PS: Very Tarskian and Bayesian of them, but are you sure they didn't say, "A belief in X is knowledge if one would never have it whenever not-X"?
I'm thinking of Robert Nozick's definition. He states his definition thus:
- P is true
- S believes that P
- If it were the case that (not-P), S would not believe that P
- If it were the case that P, S would believe that P
There is a reason why the Gettier rabbit-hole is so dangerous. You can always cook up an improbable counterexample to any definition.
For example, here is a counterexample to Nozick's definition as you present it. Suppose that I have irrationally decided to believe everything written in a certain book B and to believe nothing not written in B. Unfortunately for me, the book's author, a Mr. X, is a congenital liar. He invented almost every claim in the book out of whole cloth, with no regard for the truth of the matter. There was only one exception. There is one matter on which Mr. X is constitutionally compelled to write and to write truthfully: the color of his mother's socks on the day of his birth. At one point in B, Mr. X writes that his mother was wearing blue socks when she gave birth to him. This claim was scrupulously researched and is true. However, there is nothing in the text of B to indicate that Mr. X treated this claim any differently from all ...
Wham! Okay, I'm reverted to my old position. "Knowledge" is a fuzzy word.
ETA: Or at least a position of uncertainty. I need to research how counterfactuals work.
Well, of course Bayesianism is your friend here. Probability theory elegantly supersedes the qualitative concepts of "knowledge", "belief" and "justification" and, together with an understanding of heuristics and biases, nicely dissolves Gettier problems, so that we can safely call "knowledge" any assignment of high probability to a proposition that turns out to be true.
For example, take the original Gettier scenario. Since Jones has 10 coins in his pocket, P(man with 10 coins gets job) is bounded from below by P(Jones gets job). Hence any information that raises P(Jones gets job) necessarily raises P(man with 10 coins gets job) to something even higher, regardless of whether (Jones gets job) turns out to be true.
The psychological difficulty here is the counterintuitiveness of the rule P(A or B) >= P(A), and is in a sense "dual" to the conjunction fallacy. Just as one has to remember to subtract probability as burdensome details are introduced, one also has to remember to add probability as the reference class is broadened. When Smith learns the information suggesting Jones is the favored candidate, it may not feel like he is l...
Hey Luke,
Thanks again for your work. You are by far the greatest online teacher I've ever come across (though I've never seen you teach face-to-face). you are concise, clear, direct, empathetic, extremely thorough, tactful and accessible. I am in awe of your abilities. You take the fruit that is at the top of the tree and gently place it into my straining arms! Sorry for the exuberant worship but I really want to express my gratitude for your efforts. They definitely aren't wasted on me.
Some thoughts on this and related LW discussions. They come a bit late - apols to you and commentators if they've already been addressed or made in the commentary:
1) Definitions (this is a biggie).
There is a fair bit of confusion on LW, it seems to me, about just what definitions are and what their relevance is to philosophical and other discussion. Here's my understanding - please say if you think I've gone wrong.
If in the course of philosophical discussion, I explicitly define a familiar term, my aim in doing so is to remove the term from debate - I fix...
Analysis [had] one of two reputations. On the one hand, there was sterile cataloging of pointless folk wisdom - such as articles analyzing the concept VEHICLE, wondering whether something could be a vehicle without wheels. This seemed like trivial lexicography.
This work is useful. Understanding how people conceptualize and categorize is the starting point for epistemology. If Wittgenstein hadn't asked what qualified as a game, we might still be trying to define everything in terms of necessary and sufficient conditions.
(I'll reiterate some standard points, maybe someone will find them useful.)
The explicit connection you make between figuring out what is right and fixing people's arguments for them is a step in the right direction. Acting in this way is basically the reason it's useful to examine the physical reasons behind your own decisions or beliefs, even though such reasons don't have any normative power (that your brain tends to act a certain way is not a very good argument for acting that way). Understanding these reasons can point you to a step where the reasoning...
Upvoted for lucidity, but Empathetic Metaethics sounds more like the whole rest of LessWrong than metaethics specifically.
If there are supposed to be any additional connotations to Empathetic Metaethics it would make me very wary. I am wary of the connotation that I need someone to help me decide whether my feelings align with the Truth. I always assumed this site is called LessWrong because it generally tries to avoid driving readers to any particular conclusion, but simply away from misguided ones, so they can make their own decisions unencumbered by bi...
A lot of the issue with things like conceptual analysis, I think, is that people do them badly, and then others have to step in and waste even more words to correct them. If the worst three quarters of philosophers suddenly stopped philosophizing, the field would probably progress faster.
As one example, consider some commonly used definitions for 'morally good':
- that which produces the most pleasure for the most people
- that which is in accord with the divine will
- ...
Those aren't definitions of 'morally good'. They are theories of the morally good. I seriously doubt that there are any real philosophers that are confused about the distinction.
Eliezer advises against reading mainstream philosophy because he thinks it will "teach very bad habits of thought that will lead people to be unable to do real work
Alternative hypothesis: it will teach good habits of thought that will allow people to recognise bad amateur philosophy.
Most of the content in the sequences isn't new as such, but it did draw from many different sources, most of which were largely confined to academia. In synthesis, the product is pretty original. To the best of my knowledge, the LessWrong perspective/community has antecedents but not an obvious historical counterpart.
In that light, I'd expect the catalyzing agent for such a perspective to be the least effective such agent that could successfully accomplish the task. (Or: to be randomly selected from the space of all possible effective agents, which is quite similar in practice.) We are the tool-users not because hominids are optimized for tool use, but because we were the first ones to do so with enough skill to experience a takeoff of civilization. So it's pretty reasonable to expect the sequences to be a little wibbly.
To continue your religious metaphor, Paul wrote in atrocious Greek, had confusingly strong opinions about manbeds, and made it in to scripture because he was instrumental in building the early church communities. Augustine persuasively developed a coherent metaphysic for the religion that reconciled it with the mainstream Neoplatonism of the day, helping to cl...
The dichotomy between the Austere and the Empathic meta ethicist may well be false. I'd like to see more support for it, and specifically for the implicit claim that a question cannot be coherent unless we fully understand all its terms. Answering that claim may involve asking whether we can refer to something with a term when we do not fully understand what we are referring to (although the answer to that is surely "yes!").
I think that some basic conceptual analysis can be important for clarifying discussion given that many of these words are used and will continue to be used. For example, it is useful to know that "justified true belief" is a useful first approximation of what is meant by knowledge, but that the situation is actually slightly more complicated by that.
On the other hand, I don't expect that this will work for all concepts. Some concepts are extremely slippery and will lack enough of a shared meaning that we can provide a single definition. In these cases, we can simply point out the key features that these cases tend to have in common.
Some thoughts on this and related LW discussions. They come a bit late - apols to you and commentators if they've already been addressed or made in the commentary:
1) Definitions (this is a biggie).
There is a fair bit of confusion on LW, it seems to me, about just what definitions are and what their relevance is to philosophical and other discussion. Here's my understanding - please say if you think I've gone wrong.
If in the course of philosophical debate, I explicitly define a familiar term, my aim in doing so is to remove the term from debate - I fix the...
What happen to philosophers like Hume who tried to avoid "mere disputes of words?" Seriously, as much as many 20th century philosophers liked Hume, especially the first book of the Treatise (e.g., the positivists), why didn't they pick up on that?
(I seem to remember some flippant remark making fun of philosophers for these disputes in the Treatise but google finds me nothing)
I would like to see some enlargement on the concept of definition. It is usually treated as a simple concept: A means B or C or D; which one depending on Z. But when we try to pin down C for instance, we find that it has a lot of baggage - emotional, framing, stylistic etc. So does B and D. And in no case is the baggage of any of them the same as the baggage of A. None of - defining terms or tabooing words or coining new words - really works all that well in the real world, although they of course help. Do you see a way around this fuzziness?
Another 'morally good' definition for your list is 'that which will not make the doer feel guilty or shameful in future'. It is no better than the others but quite different.
I don't think you're arguing against conceptual analysis, instead you want to treat a particular conceptual analysis (reductive physicalism) as gospel. What is the claim that there are two definitions of sound that we can confuse, the acoustic vibrations in the air and the auditory experience in a brain, if it's not a reductive conceptual analysis of the concept of sound?
The definition of "right action" is the kind of action you should do.
You don't need to know what "should" means, you just need to do what you should do and not do what you shouldn't do.
One should be able to cash out arguments about the "definition" of "right" as arguments about the actual nature of shouldness.
I was under the impression that the example of an unobserved tree falling in the woods is taken as a naturalized version of Schrodinger's Cat experiment. So the question of whether it makes a sound is not necessarily about the definition of a sound.
Part of the sequence: No-Nonsense Metaethics. Also see: A Human's Guide to Words.
If a tree falls in the forest, and no one hears it, does it make a sound?
Albert: "Of course it does. What kind of silly question is that? Every time I've listened to a tree fall, it made a sound, so I'll guess that other trees falling also make sounds. I don't believe the world changes around when I'm not looking."
Barry: "Wait a minute. If no one hears it, how can it be a sound?"
Albert and Barry are not arguing about facts, but about definitions:
...the first person is speaking as if 'sound' means acoustic vibrations in the air; the second person is speaking as if 'sound' means an auditory experience in a brain. If you ask "Are there acoustic vibrations?" or "Are there auditory experiences?", the answer is at once obvious. And so the argument is really about the definition of the word 'sound'.
Of course, Albert and Barry could argue back and forth about which definition best fits their intuitions about the meaning of the word. Albert could offer this argument in favor of using his definition of sound:
My computer's microphone can record a sound without anyone being around to hear it, store it as a file, and it's called a 'sound file'. And what's stored in the file is the pattern of vibrations in air, not the pattern of neural firings in anyone's brain. 'Sound' means a pattern of vibrations.
Barry might retort:
Imagine some aliens on a distant planet. They haven't evolved any organ that translates vibrations into neural signals, but they still hear sounds inside their own head (as an evolutionary biproduct of some other evolved cognitive mechanism). If these creatures seem metaphysically possible to you, then this shows that our concept of 'sound' is not dependent on patterns of vibrations.
If their debate seems silly to you, I have sad news. A large chunk of moral philosophy looks like this. What Albert and Barry are doing is what philosophers call conceptual analysis.1
I won't argue that everything that has ever been called 'conceptual analysis' is misguided.2 Instead, I'll give examples of common kinds of conceptual analysis that corrupt discussions of morality and other subjects.
The following paragraph explains succinctly what is wrong with much conceptual analysis:
Analysis [had] one of two reputations. On the one hand, there was sterile cataloging of pointless folk wisdom - such as articles analyzing the concept VEHICLE, wondering whether something could be a vehicle without wheels. This seemed like trivial lexicography. On the other hand, there was metaphysically loaded analysis, in which ontological conclusions were established by holding fixed pieces of folk wisdom - such as attempts to refute general relativity by holding fixed allegedly conceptual truths, such as the idea that motion is intrinsic to moving things, or that there is an objective present.3
Consider even the 'naturalistic' kind of conceptual analysis practiced by Timothy Schroeder in Three Faces of Desire. In private correspondance, I tried to clarify Schroeder's project:
As I see it, [your book] seeks the cleanest reduction of the folk psychological term 'desire' to a natural kind, ala the reduction of the folk chemical term 'water' to H2O. To do this, you employ a naturalism-flavored method of conceptual analysis according to which the best theory of desire is one that is logically consistent, fits the empirical facts, and captures how we use the term and our intuitions about its meaning.
Schroeder confirmed this, and it's not hard to see the motivation for his project. We have this concept 'desire', and we might like to know: "Is there anything in the world similar to what we mean by 'desire'?" Science can answer the "is there anything" part, and intuition (supposedly) can answer the "what we mean by" part.
The trouble is that philosophers often take this "what we mean by" question so seriously that thousands of pages of debate concern which definition to use rather than which facts are true and what to anticipate.
In one chapter, Schroeder offers 8 objections4 to a popular conceptual analysis of 'desire' called the 'action-based theory of desire'. Seven of these objections concern our intuitions about the meaning of the word 'desire', including one which asks us to imagine the existence of alien life forms that have desires about the weather but have no dispositions to act to affect the weather. If our intuitions tell us that such creatures are metaphysically possible, goes the argument, then our concept of 'desire' need not be linked to dispositions to act.
Contrast this with a conversation you might have with someone from the Singularity Institute. Within 20 seconds of arguing about the definition of 'desire', someone will say, "Screw it. Taboo 'desire' so we can argue about facts and anticipations, not definitions."5
Arguing about definitions is not always misguided. Words can be wrong:
When the philosophers of Plato's Academy claimed that the best definition of a human was a "featherless biped", Diogenes the Cynic is said to have exhibited a plucked chicken and declared "Here is Plato's Man." The Platonists promptly changed their definition to "a featherless biped with broad nails."
Likewise, if I give a lecture on correlations between income and subjective well-being and I conclude by saying, "And that, ladies and gentlemen, is my theory of the atom," then you have some reason to object. Nobody else uses the term 'atom' to mean anything remotely like what I've just discussed. If I ever do that, I hope you will argue that my definition of 'morality' is 'wrong' (or unhelpful, or confusing, or something).
Some unfortunate words are used in a wide variety of vague and ambiguous ways.6 Moral terms are among these. As one example, consider some commonly used definitions for 'morally good':
Often, people can't tell you what they mean by moral terms when you question them. There is little hope of taking a survey to decide what moral terms 'typically mean' or 'really mean'. The problem may be worse for moral terms than for (say) art terms. Moral terms have more powerful connotations than art terms, and are thus a greater attractor for sneaking in connotations. Moral terms are used to persuade. "It's just wrong!" the moralist cries, "I don't care what definition you're using right now. It's just wrong: don't do it."
Moral discourse is rife with motivated cognition. This is part of why, I suspect, people resist dissolving moral debates even while they have no trouble dissolving the 'tree falling in a forest' debate.
So much moral philosophy is consumed by debates over definitions that I will skip to an example from someone you might hope would know better: reductionist Frank Jackson7:
...if Tom tells us that what he means by a right action is one in accord with God's will, rightness according to Tom is being in accord with God's will. If Jack tells us that what he means by a right action is maximizing expected value as measured in hedons, then, for Jack, rightness is maximizing expected value...
But if we wish to address the concerns of our fellows when we discuss the matter - and if we don't, we will not have much of an audience - we had better mean what they mean. We had better, that is, identify our subject via the folk theory of rightness, wrongness, goodness, badness, and so on. We need to identify rightness as the property that satisfies, or near enough satisfies, the folk theory of rightness - and likewise for the other moral properties. It is, thus, folk theory that will be our guide in identifying rightness, goodness, and so on.8
The meanings of moral terms, says Jackson, are given by their place in a network of platitudes ('clauses') from folk moral discourse:
The input clauses of folk morality tell us what kinds of situations described in descriptive, non-moral terms warrant what kinds of description in ethical terms: if an act is an intentional killing, then normally it is wrong; pain is bad; 'I cut, you choose' is a fair procedure; and so on.
The internal role clauses of folk morality articulate the interconnections between matters described in ethical, normative language: courageous people are more likely to do what is right than cowardly people; the best option is the right option; rights impose duties of respect; and so on.
The output clauses of folk morality take us from ethical judgements to facts about motivation and thus behaviour: the judgement that an act is right is normally accompanied by at least some desire to perform the act in question; the realization that an act would be dishonest typically dissuades an agent from performing it; properties that make something good are the properties we typically have some kind of pro-attitude towards, and so on.
Moral functionalism, then, is the view that the meanings of the moral terms are given by their place in this network of input, output, and internal clauses that makes up folk morality.9
And thus, Jackson tosses his lot into the definitions debate. Jackson supposes that we can pick out which platitudes of moral discourse matter, and how much they matter, for determining the meaning of moral terms - despite the fact that individual humans, and especially groups of humans, are themselves confused about the meanings of moral terms, and which platitudes of moral discourse should 'matter' in fixing their meaning.
This is a debate about definitions that will never end.
In the next post, we'll dissolve standard moral debates the same way Albert and Barry should have dissolved their debate about sound.
But that is only the first step. It is important to not stop after sweeping away the confusions of mainstream moral philosophy to arrive at mere correct answers. We must stare directly into the heart of the problem and do the impossible.
Consider Alex, who wants to do the 'right' thing. But she doesn't know what 'right' means. Her question is: "How do I do what is right if I don't know exactly what 'right' means?"
The Austere Metaethicist might cross his arms and say:
Tell me what you mean by 'right', and I will tell you what is the right thing to do. If by 'right' you mean X, then Y is the right thing to do. If by 'right' you mean P, then Z is the right thing to do. But if you can't tell me what you mean by 'right', then you have failed to ask a coherent question, and no one can answer an incoherent question.
The Empathic Metaethicist takes up a greater burden. The Empathic Metaethicist says to Alex:
You may not know what you mean by 'right.' You haven't asked a coherent question. But let's not stop there. Here, let me come alongside you and help decode the cognitive algorithms that generated your question in the first place, and then we'll be able to answer your question. Then not only can we tell you what the right thing to do is, but also we can help bring your emotions into alignment with that truth... as you go on to (say) help save the world rather than being filled with pointless existential angst about the universe being made of math.
Austere metaethics is easy. Empathic metaethics is hard. But empathic metaethics is what needs to be done to answer Alex's question, and it's what needs to be done to build a Friendly AI. We'll get there in the next few posts.
Next post: Pluralistic Moral Reductionism
Previous post: What is Metaethics?
1 Eliezer advises against reading mainstream philosophy because he thinks it will "teach very bad habits of thought that will lead people to be unable to do real work." Conceptual analysis is, I think, exactly that: a very bad habit of thought that renders many people unable to do real work. Also: My thanks to Eliezer for his helpful comments on an early draft of this post.
2 For example: Jackson (1998), p. 28, has a different view of conceptual analysis: "conceptual analysis is the very business of addressing when and whether a story told in one vocabulary is made true by one told in some allegedly more fundamental vocabulary." For an overview of Jackson's kind of conceptual analysis, see here. Also, Alonzo Fyfe reminded me that those who interpret the law must do a kind of conceptual analysis. If a law has been passed declaring that vehicles are not allowed on playgrounds, a judge must figure out whether 'vehicle' includes or excludes rollerskates. More recent papers on conceptual analysis are available at Philpapers. Finally, read Chalmers on verbal disputes.
3 Braddon-Mitchell (2008). A famous example of the first kind lies at the heart of 20th century epistemology: the definition of 'knowledge.' Knowledge had long been defined as 'justified true belief', but then Gettier (1963) presented some hypothetical examples of justified true belief that many of us would intuitively not label as 'knowledge.' Philosophers launched a cottage industry around new definitions of 'knowledge' and new counterexamples to those definitions. Brian Weatherson called this the "analysis of knowledge merry-go-round." Tyrrell McAllister called it the 'Gettier rabbit-hole.'
4 Schroeder (2004), pp. 15-27. Schroeder lists them as 7 objections, but I count his 'trying without desiring' and 'intending without desiring' objections separately.
5 Tabooing one's words is similar to what Chalmers (2009) calls the 'method of elimination'. In an earlier post, Yudkowsky used what Chalmers (2009) calls the 'subscript gambit', except Yudkowsky used underscores instead of subscripts.
6 See also Gallie (1956).
7 Eliezer said that the closest thing to his metaethics from mainstream philosophy is Jackson's 'moral functionalism', but of course moral functionalism is not quite right.
8 Jackson (1998), p. 118.
9 Jackson (1998), pp. 130-131.
Braddon-Mitchell (2008). Naturalistic analysis and the a priori. In Braddon-Mitchell & Nola (eds.), Conceptual Analysis and Philosophical Naturalism (pp. 23-43). MIT Press.
Chalmers (2009). Verbal disputes. Unpublished.
Gallie (1956). Essentially contested concepts. Proceedings of the Aristotelean Society, 56: 167-198.
Gettier (1963). Is justified true belief knowledge? Analysis, 23: 121-123.
Jackson (1998). From Metaphysics to Ethics: A Defense of Conceptual Analysis. Oxford University Press.
Schroeder (2004). Three Faces of Desire. Oxford University Press.