Some thoughts on this and related LW discussions. They come a bit late - apols to you and commentators if they've already been addressed or made in the commentary:
1) Definitions (this is a biggie).
There is a fair bit of confusion on LW, it seems to me, about just what definitions are and what their relevance is to philosophical and other discussion. Here's my understanding - please say if you think I've gone wrong.
If in the course of philosophical discussion, I explicitly define a familiar term, my aim in doing so is to remove the term from debate - I fix the value of a variable to restrict the problem. It'd be good to find a real example here, but I'm not convinced defining terms happens very often in philosophical or other debate. By way of a contrived example, one might want to consider, in evaluating some theory, the moral implications of actions made under duress (a gun held to the head) but not physically initiated by an external agent (a jostle to the arm). One might say, "Define 'coerced action' to mean any action not physically initiated but made under duress" (or more precise words to the effect). This done, it wouldn't make sense simply to object that my conclusion regarding coerced actions doesn't apply to someone physically pushed from behind - I have stuipulated for the sake of argument I'm not talking about such cases. (in this post, you distinguish stipulation and definition - do you have in mind a distinction I'm glossing over?)
Contrast this to the usual case for conceptual analyses, where it's assumed there's a shared concept ('good', 'right', 'possible', 'knows', etc), and what is produced is meant to be a set of necessary and sufficient conditions meant to capture the concept. Such an analysis is not a definition. Regarding such analyses, typically one can point to a particular thing and say, eg, "Our shared concept includes this specimen, it lacks a necessary condition, therefore your analysis is mistaken" - or, maybe "Intuitively, this specimen falls under our concept, it lacks...". Such a response works only if there is broad agreement that the specimen falls under the concept. Usually this works out to be the case.
I haven't read the Jackson book, so please do correct me if you think I've misunderstood, but I take it something like this is his point in the paragraphs you quote. Tom and Jack can define 'right action' to mean whatever they want it to. In so doing, however, we cease to have any reason to think they mean by the term what we intuitively do. Rather, Jackson is observing, what Tom and Jack should be doing is saying that rightness is that thing (whatever exactly it is) which our folk concepts roughly converge on, and taking up the task of refining our understanding from there - no defining involved.
You say,
... Jackson supposes that we can pick out which platitudes of moral discourse matter, and how much they matter, for determining the meaning of moral terms
Well, not quite. The point I take it is rather that there simply are 'folk' platitudes which pick-out the meanings of moral terms - this is the starting point. 'Killing people for fun is wrong', 'Helping elderly ladies across the street is right' etc, etc. These are the data (moral intuitions, as usually understood). If this isn't the case, there isn't even a subject to discuss. Either way, it has nothing to do with definitions.
Confusion about definitions is evident in the quote from the post you link to. To re-quote:
...the first person is speaking as if 'sound' means acoustic vibrations in the air; the second person is speaking as if 'sound' means an auditory experience in a brain. If you ask "Are there acoustic vibrations?" or "Are there auditory experiences?", the answer is at once obvious. And so the argument is really about the definition of the word 'sound'.
Possibly the problem is that 'sound' has two meanings, and the disputants each are failing to see that the other means something different. Definitions are not relevant here, meanings are. (Gratuitous digression: what is "an auditory experience in a brain"? If this means something entirely characterizable in terms of neural events, end of story, then plausibly one of the disputants would say this does not capture what he means by 'sound' - what he means is subjective and ineffable, something neural events aren't. He might go on to wonder whether that subjective, ineffable thing, given that it is apparently created by the supposedly mind-independent event of the falling of a tree, has any existence apart from his self (not to be confused with his brain!). I'm not defending this view, just saying that what's offered is not a response but rather a simple begging of the question against it. End of digression.)
2) In your opening section you produce an example meant to show conceptual analysis is silly. Looks to me more like a silly attempt at an example of conceptual analysis. If you really want to make your case, why not take a real example of a philosophical argument -preferably one widely held in high regard at least by philosophers? There's lots of 'em around.
3) In your section The trouble with conceptual analysis, you finally explain,
The trouble is that philosophers often take this "what we mean by" question so seriously that thousands of pages of debate concern which definition to use... .
As explained above, philosophical discussion is not about "which definition to use" -it's about (roughly, and among other things) clarifying our concepts. The task is difficult but worthwhile because the concepts in question are important but subtle.
Within 20 seconds of arguing about the definition of 'desire', someone will say, "Screw it. Taboo 'desire' so we can argue about facts and anticipations, not definitions."
If you don't have the patience to do philosophy, or you don't think it's of any value, by all means do something else -argue about facts and anticipations, whatever precisely that may involve. Just don't think that in doing this latter thing you'll address the question philosophy is interested in, or that you've said anything at all so far to show philosophy isn't worth doing. In this connection, one of the real benefits of doing philosophy is that it encourages precision and attention to detail in thinking. You say Eliezer Yudkowsky "...advises against reading mainstream philosophy because he thinks it will 'teach very bad habits of thought that will lead people to be unable to do real work.'" The original quote continues, "...assume naturalism! Move on! NEXT!" Unfortunately Eliezer has a bad habit of making unclear and undefended or question-begging assertions, and this is one of them. What are the bad habits, and how does philosophy encourage them? And what precisely is meant by 'naturalism'? To make the latter assertion and simultaneously to eschew the responsibility of articulating what this commits you to is to presume you can both have your cake and eat it too. This may work in blog posts -it wouldn't pass in serious discussion.
(Unlike some on this blog, I have not slavishly pored through Eliezer's every post. If there is somewhere a serious discussion of the meaning of 'naturalism' which shows how the usual problems with normative concepts like 'rational' can successfully be navigated, I will withdraw this remark).
Upvoted for thoughtfulness and thoroughness.
(in this post, you distinguish stipulation and definition - do you have in mind a distinction I'm glossing over?)
I'm using 'definition' in the common sense: "the formal statement of the meaning or significance of a word, phrase, etc." A stipulative definition is a kind of definition "in which a new or currently-existing term is given a specific meaning for the purposes of argument or discussion in a given context."
A conceptual analysis of a term using necessary and sufficient conditions is...
Part of the sequence: No-Nonsense Metaethics. Also see: A Human's Guide to Words.
If a tree falls in the forest, and no one hears it, does it make a sound?
Barry: "Wait a minute. If no one hears it, how can it be a sound?"
Albert and Barry are not arguing about facts, but about definitions:
Of course, Albert and Barry could argue back and forth about which definition best fits their intuitions about the meaning of the word. Albert could offer this argument in favor of using his definition of sound:
Barry might retort:
If their debate seems silly to you, I have sad news. A large chunk of moral philosophy looks like this. What Albert and Barry are doing is what philosophers call conceptual analysis.1
The trouble with conceptual analysis
I won't argue that everything that has ever been called 'conceptual analysis' is misguided.2 Instead, I'll give examples of common kinds of conceptual analysis that corrupt discussions of morality and other subjects.
The following paragraph explains succinctly what is wrong with much conceptual analysis:
Consider even the 'naturalistic' kind of conceptual analysis practiced by Timothy Schroeder in Three Faces of Desire. In private correspondance, I tried to clarify Schroeder's project:
Schroeder confirmed this, and it's not hard to see the motivation for his project. We have this concept 'desire', and we might like to know: "Is there anything in the world similar to what we mean by 'desire'?" Science can answer the "is there anything" part, and intuition (supposedly) can answer the "what we mean by" part.
The trouble is that philosophers often take this "what we mean by" question so seriously that thousands of pages of debate concern which definition to use rather than which facts are true and what to anticipate.
In one chapter, Schroeder offers 8 objections4 to a popular conceptual analysis of 'desire' called the 'action-based theory of desire'. Seven of these objections concern our intuitions about the meaning of the word 'desire', including one which asks us to imagine the existence of alien life forms that have desires about the weather but have no dispositions to act to affect the weather. If our intuitions tell us that such creatures are metaphysically possible, goes the argument, then our concept of 'desire' need not be linked to dispositions to act.
Contrast this with a conversation you might have with someone from the Singularity Institute. Within 20 seconds of arguing about the definition of 'desire', someone will say, "Screw it. Taboo 'desire' so we can argue about facts and anticipations, not definitions."5
Disputing definitions
Arguing about definitions is not always misguided. Words can be wrong:
Likewise, if I give a lecture on correlations between income and subjective well-being and I conclude by saying, "And that, ladies and gentlemen, is my theory of the atom," then you have some reason to object. Nobody else uses the term 'atom' to mean anything remotely like what I've just discussed. If I ever do that, I hope you will argue that my definition of 'morality' is 'wrong' (or unhelpful, or confusing, or something).
Some unfortunate words are used in a wide variety of vague and ambiguous ways.6 Moral terms are among these. As one example, consider some commonly used definitions for 'morally good':
Often, people can't tell you what they mean by moral terms when you question them. There is little hope of taking a survey to decide what moral terms 'typically mean' or 'really mean'. The problem may be worse for moral terms than for (say) art terms. Moral terms have more powerful connotations than art terms, and are thus a greater attractor for sneaking in connotations. Moral terms are used to persuade. "It's just wrong!" the moralist cries, "I don't care what definition you're using right now. It's just wrong: don't do it."
Moral discourse is rife with motivated cognition. This is part of why, I suspect, people resist dissolving moral debates even while they have no trouble dissolving the 'tree falling in a forest' debate.
Disputing the definitions of moral terms
So much moral philosophy is consumed by debates over definitions that I will skip to an example from someone you might hope would know better: reductionist Frank Jackson7:
But if we wish to address the concerns of our fellows when we discuss the matter - and if we don't, we will not have much of an audience - we had better mean what they mean. We had better, that is, identify our subject via the folk theory of rightness, wrongness, goodness, badness, and so on. We need to identify rightness as the property that satisfies, or near enough satisfies, the folk theory of rightness - and likewise for the other moral properties. It is, thus, folk theory that will be our guide in identifying rightness, goodness, and so on.8
The meanings of moral terms, says Jackson, are given by their place in a network of platitudes ('clauses') from folk moral discourse:
And thus, Jackson tosses his lot into the definitions debate. Jackson supposes that we can pick out which platitudes of moral discourse matter, and how much they matter, for determining the meaning of moral terms - despite the fact that individual humans, and especially groups of humans, are themselves confused about the meanings of moral terms, and which platitudes of moral discourse should 'matter' in fixing their meaning.
This is a debate about definitions that will never end.
Austere Metaethics vs. Empathic Metaethics
In the next post, we'll dissolve standard moral debates the same way Albert and Barry should have dissolved their debate about sound.
But that is only the first step. It is important to not stop after sweeping away the confusions of mainstream moral philosophy to arrive at mere correct answers. We must stare directly into the heart of the problem and do the impossible.
Consider Alex, who wants to do the 'right' thing. But she doesn't know what 'right' means. Her question is: "How do I do what is right if I don't know exactly what 'right' means?"
The Austere Metaethicist might cross his arms and say:
The Empathic Metaethicist takes up a greater burden. The Empathic Metaethicist says to Alex:
Austere metaethics is easy. Empathic metaethics is hard. But empathic metaethics is what needs to be done to answer Alex's question, and it's what needs to be done to build a Friendly AI. We'll get there in the next few posts.
Next post: Pluralistic Moral Reductionism
Previous post: What is Metaethics?
Notes
1 Eliezer advises against reading mainstream philosophy because he thinks it will "teach very bad habits of thought that will lead people to be unable to do real work." Conceptual analysis is, I think, exactly that: a very bad habit of thought that renders many people unable to do real work. Also: My thanks to Eliezer for his helpful comments on an early draft of this post.
2 For example: Jackson (1998), p. 28, has a different view of conceptual analysis: "conceptual analysis is the very business of addressing when and whether a story told in one vocabulary is made true by one told in some allegedly more fundamental vocabulary." For an overview of Jackson's kind of conceptual analysis, see here. Also, Alonzo Fyfe reminded me that those who interpret the law must do a kind of conceptual analysis. If a law has been passed declaring that vehicles are not allowed on playgrounds, a judge must figure out whether 'vehicle' includes or excludes rollerskates. More recent papers on conceptual analysis are available at Philpapers. Finally, read Chalmers on verbal disputes.
3 Braddon-Mitchell (2008). A famous example of the first kind lies at the heart of 20th century epistemology: the definition of 'knowledge.' Knowledge had long been defined as 'justified true belief', but then Gettier (1963) presented some hypothetical examples of justified true belief that many of us would intuitively not label as 'knowledge.' Philosophers launched a cottage industry around new definitions of 'knowledge' and new counterexamples to those definitions. Brian Weatherson called this the "analysis of knowledge merry-go-round." Tyrrell McAllister called it the 'Gettier rabbit-hole.'
4 Schroeder (2004), pp. 15-27. Schroeder lists them as 7 objections, but I count his 'trying without desiring' and 'intending without desiring' objections separately.
5 Tabooing one's words is similar to what Chalmers (2009) calls the 'method of elimination'. In an earlier post, Yudkowsky used what Chalmers (2009) calls the 'subscript gambit', except Yudkowsky used underscores instead of subscripts.
6 See also Gallie (1956).
7 Eliezer said that the closest thing to his metaethics from mainstream philosophy is Jackson's 'moral functionalism', but of course moral functionalism is not quite right.
8 Jackson (1998), p. 118.
9 Jackson (1998), pp. 130-131.
References
Braddon-Mitchell (2008). Naturalistic analysis and the a priori. In Braddon-Mitchell & Nola (eds.), Conceptual Analysis and Philosophical Naturalism (pp. 23-43). MIT Press.
Chalmers (2009). Verbal disputes. Unpublished.
Gallie (1956). Essentially contested concepts. Proceedings of the Aristotelean Society, 56: 167-198.
Gettier (1963). Is justified true belief knowledge? Analysis, 23: 121-123.
Jackson (1998). From Metaphysics to Ethics: A Defense of Conceptual Analysis. Oxford University Press.
Schroeder (2004). Three Faces of Desire. Oxford University Press.