Can We Naturalize Moral Epistemology?
8Satya Benson
4TAG
4tylermjohn
1RogerDearnaley
10Satya Benson
3RogerDearnaley
9Satya Benson
1RogerDearnaley
11Said Achmiz
2RogerDearnaley
3Satya Benson
4RogerDearnaley
3Said Achmiz
2RogerDearnaley
5Davidmanheim
10Kaarel
3Davidmanheim
5Charlie Steiner
4tylermjohn
2WhatsTrueKittycat
2Joseph Miller
-4RogerDearnaley
New Comment
What's the state of existing empirical evidence on whether Moral Reasoning is Real?
My own observations tell me that it is not. Certainly, some people engage in moral reasoning and are satisfied with their results to varying degrees, but it appears to me that this is a small proportion of humans.
My preliminary investigation into the research confirms my existing belief that most moral reasoning is post-hoc, and that while human values can change it is almost never due to reasoned arguments and instead a social and emotional process. When moral reasoning seems to work, endorsement is often shallow and attitudes can revert within days.
I am frequently reminded that I underestimate the degree to which my own view on this is not universally held, however.
My own observations tell me that it is not. Certainly, some people engage in moral reasoning and are satisfied with their results to varying degrees, but it appears to me that this is a small proportion of humans
Something can be real and scarce.
I wanted to answer as OP, but I don't have a particularly informed view on this. And my view is biased by having spent a lot of time around moral philosophers who are hugely, hugely disproportionately likely to fall into the "moral reasoning is real" camp — and by having evolved from conservative Evangelical ethics to classical utilitarianism!
One fairly robust observation in psychology is that people do feel pressure to engage in moral consistency reasoning. One example of this comes from the literature on the Meat Eating Paradox. Let me reproduce an overview from pp. 573-574 here:
In a series of five studies, Brock Bastian and colleagues have demonstrated a link between seeing animals as food, on one hand, and seeing animals as having diminished mental lives and moral value, on the other hand. We will here describe three.
In a first study, participants were asked to rate the degree to which each of a diverse group of thirty-two animals possessed ten mental capacities, and then were asked how likely they would be to eat the animal and how wrong they believe eating that animal is. Perceived edibility was negatively associated with mind possession (r = –.42, p < .001), which was in turn associated with how the perceived wrongness of eating the animal (r = .80, p < .001).
In a second study, participants were asked to eat dried beef or dried nuts and then judge a cow’s cognitive abilities and desert of moral treatment on two seven-point scales. Participants in the beef condition (M = 5.57) viewed the cow as significantly less deserving of moral concern than those in the control condition (M = 6.08).
In a third study, participants were informed about Papua New Guinea’s tree kangaroo and informed variably that tree kangaroos have a steady population, that they are killed by storms, that they are killed for food, or that they are foraged for food. Bastian and colleagues found that categorizing tree kangaroos as food and no other features of these cases led participants to attribute less capacity for suffering and less moral concern.
Additionally, a sequence of five studies from Jonas Kunst and Sigrid Hohle demonstrates that processing meat, beheading a whole roasted pig, watching a meat advertisement without a live animal versus one with a live animal, describing meat production as “harvesting” versus “killing” or “slaughtering,” and describing meat as “beef/pork” rather than “cow/pig” all decreased empathy for the animal in question and, in several cases, significantly increased willingness to eat meat rather than an alternative vegetarian dish.
Psychologists involved in these and several other studies believe that these phenomena occur because people recognize an incongruity between eating animals and seeing them as beings with mental life and moral status, so they are motivated to resolve this cognitive dissonance by lowering their estimation of animal sentience and moral status.
I think studies like these at least imply that people are driven to resolve local tensions in their moral intuitions. (E.g. tensions between views about moral status, consciousness, right action, and belief about one's own virtues.) How often would resolving these tensions lead to radical departures in moral views? I'm not sure! It seems to mostly depend on how much psychological pressure this creates and how interconnected people's web of moral intuitions are. In people with highly interconnected moral webs and a lot of psychological pressure towards consistency, you'd expect much bigger changes.
1Evolutionary ethics is a real, scientific field of biology — it makes actual, falsifiable predictions. However, that's not what most people think of when they say "moral reasoning".
Evolutionary ethics aims to help people understand why we value the things we do. It doesn't have the ability to say anything about what we ought to value.
Evolutionary ethics provides a solution to the "ought-from-is" problem — in a cold uncaring universe governed by physical laws, where does the preference ordering/utility function of human values come from? That is a question about humans, and evolutionary ethics is the name of the scientific field that studies and answers it.
In order to decide "what we ought to value", you need to create a preference ordering on moral systems, to show that one is better than another. You can't use a moral system to do that — any moral system (that isn't actually internally inconsistent) automatically prefers itself to all other moral systems, so using a moral system to select a moral system is just a circular argument — the same logic applies to any moral system you plug in to such an argument.
So to discuss "what we ought to value" you need to judge moral systems and their consequences using something that is both vaguer and more practical than a moral system. Such as psychology, or sociology, or political expedience, or some combination of these. All of which occur in the context of human nature and human moral intuitions and instincts — which is exactly what evolutionary ethics studies and provides a theoretical framework to explain.
Thanks for explaining.
So to discuss "what we ought to value" you need to judge moral systems and their consequences using something that is both vaguer and more practical than a moral system. Such as psychology, or sociology, or political expedience, or some combination of these.
I think this is tempting but ultimately misguided, because the choice of a 'more practical and vague' system by which to judge moral systems is just a second order moral system in itself which happens to be practical and vague. This is metanormative regress.
The only coherent solution to the "ought-from-is" problem I've come across is normative eliminativism - 'ought' statements are either false or a special type of descriptive statement.
I encourage you to look into evolutionary ethics (and evolutionary psychology in general): I think it provides both a single, well-defined (though vague) ethical foundation and an answer to the "ought-from-is" problem. It's a branch of science, rather than philosophy, so we are able to do better than just agreeing to disagree.
I’ve looked into these things, and as far as I can tell, all such fields or theories either do not attempt to solve the is-ought problem (as e.g. evo psych does not), or attempt to do so but (absolutely unsurprising) completely fail.
What am I missing? What’s the answer?
Humans are living, evolved agents. They thus each individually have a set of goals they attempt to optimize: a preference ordering on possible outcomes. Evolution predicts that, inside the distribution the creature evolved in, this preference ordering will be nearly as well aligned to the creature's evolutionary fitness as is computationally feasible for the creature.
This is the first step in ought-from-is: it gives us a preference ordering, which if approximately coherent (i.e. not significantly Dutch-bookable — something evolution seems likely to encourage) implies an approximate utility function — a separate one for each human (or other animal). As in "this is what I want (for good evolutionary reasons)". So, using agent fundamentals terminology, the answer to the ought-from-is question "where does the preference ordering on states of the world come from?" is "every evolved intelligent animal is going to have a set of evolved and learned behaviors that can be thought of as encoding a preference ordering (albeit one that may not be completely coherent, to the extent that it only approximately fulfills the criteria for the coherence theorems). " [It even gives us a scale on the utility function, something a preference ordering doesn't give us, in terms of the approximat effect on the evolutionary fitness of the organism: which ought correlate fairly well with the effort the organism is willing to put in to optimizing the outcome. This solves things like the utility monster problem.]
So far, that's just Darwinism, or arguably the subfield Evolutionary Psychology, since it's about the evolution of behavior. And so far the preference ordering "ought" is "what I want" rather than an ethical system, so arguably doesn't yet deserve the term "ought" — I want to have a billion dollars, but saying that I thus "ought" to have a billion dollars is a bit of a stretch linguistically. Arguably so far we've only solved "want-from-is".
Evolutionary Ethics goes on to explain why humans, as an intelligent social animal, are evolved to have a set of moral instincts that lets them form a set of conventions for compromises between the preference orderings of all the individual members of a tribe or other society of humans, in order to reduce intra-group conflicts by forming a "social compact" (to modify Hobbes' terminology slightly). For example, the human sense of fairness encourages sharing of food from successful hunting or gathering expeditions, our habit of forming friendships produces alliances, and so forth. The results of this are not exactly a single coherent preference ordering on all outcomes for the society in question , let alone a utility function, more a set of heuristics on how the preference orderings of individual tribal members should be reconciled ('should' is here being used in the sense that, if you don't do this and other members of the society find out, there are likely to be consequences). In general, members of the society are free to optimize whatever their own individual preferences are, unless this significantly decreases the well-being (evolutionary fitness) of other members of the society. My business is mine, until it intrudes on someone else: but then we need to compromise.
So now we have a single socially agreed "ought" per society — albeit one fuzzier and with rather more internal structure than people generally encode into utility functions: it's a preference ordering produced by a process whose inputs are many preference orderings, (and might thus be less coherent). This moral system will be shaped both by humans' evolved moral instincts (which are mostly shared across members of our species, albeit less so by socipaths), as is predicted by evolutionary ethics, and also by sociological, historical and political processes.
So, in philosophical terminology:
- moral realism: no (However, human evolved moral instincts do tend to provide some simple consistent moral patterns across human societies, as long as you qualify all your moral statements with the rider "For humans, …". So one could argue for a sort of 'semi-realism' for some simple moral statements, like "incest is bad" — that has a pretty clear evolutionary basis, and is pretty close to socially universal.)
- moral relativism: yes — per society, and for some basic patterns/elements for the entire human species, but with no guarantees that these would apply to a very different intelligent social species (though there might well be commonalities for good evolutionary reasons — anything with sexual reproduction and deleterious recessives is likely to evolve an incest taboo.).
Given Said Achmiz's comment already has 11 upvotes and 2 agreement points, should I write a post explaining all this? I had thought it all rather obvious to anyone who looks into evolutionary ethics and thinks a bit about what this means for moral philosophy (as quite a number of moral philosophers have done), but perhaps not.
This comment does really help me understand what you're saying better. If you write a post expanding it, I would encourage you to address the following related points:
- Can you have some members of a society who don't share some of the consistent moral patterns which evolved, or do you claim that every member reliably holds these morals?
- Can someone decide what they ought to value using this system? How?
- Is it wrong if someone simply doesn't care about what society values? Why?
- How can we tell that your story tells us what we ought to value rather than simply explaining why we value the things we do?
- Do you make a clear distinction between normative ethics and descriptive ethics? What is it?
Thanks, I'll keep that in mind when deciding what to cover in the post when I write it.
Briefly for now, just to continue the discussion a bit:
Can you have some members of a society who don't share some of the consistent moral patterns which evolved, or do you claim that every member reliably holds these morals?
The former (sociopaths, for example, are genetically predisposed to be less moral, and it has often been suggested this behavior is an adapted form of social opportunism, in game theory terms a different strategy, perhaps one with a stable equilibrium frequency, rather than being simply a genetic disease) — though they may get punished or shunned as a result, if their morality is different in a way that other members of the society disapprove of.
Can someone decide what they ought to value using this system? How?
How a person wants to make decisions is up to them. Most people make these decisions in a way that is influenced by their own moral instincts, social pressures, their circumstances and upbringing, their personality, expedience, and so forth. Generally, acting contrary to your instincts and impulses is challenging to do and stressful — it's probably easier to go against them only when there's a clear rational need. For example, if you're rationally aware that they are maladaptive or antisocial in modern society.
Is it wrong if someone simply doesn't care about what society values? Why?
In the context of their society of humans, yes, it is considered wrong (in almost all societies). Note that this is a morally relative statement, not a morally realist one. However, simply not caring at all is pretty atypical behavior under human moral intuitions, and is generally also pretty maladaptive (unless, say, you have absolute power). So from an evolutionary ethics point of view, it seems likely to be maladaptive behavior that will often get you imprisoned, exiled or killed. So as relative statements go, this is a pretty strong one.
How can we tell that your story tells us what we ought to value rather than simply explaining why we value the things we do?
The point of evolutionary ethics is that there is no meaningful, uniquely defined, separate sense of "ought" much stronger than "according to most common moral systems for this particular social species, or most similar species". So the best you can do is explain why we, or most societies of a certain type, or most societies of a certain species, believe that that's something you "ought" to do. This approach isn't a form of moral realism.
Do you make a clear distinction between normative ethics and descriptive ethics? What is it?
Normative ethics describes my opinion about what I think people should do. Descriptive ethics describes what many people think people should do. In a society that has a social compact, the latter carries a lot more weight. However, I'm perfectly happy to discuss ethical system design: if we altered the ethics of our (or some other) society in a certain way, then the effects on the society would be this or that, which would or wouldn't tend to increase or decrease things like human flourishing (which is itself explained by evolutionary psychology). That sounds a lot like normative ethics, but there's a key difference: the discussion is based on a (hopefully mutually agreed) assessment of the relative merits of the predicted consequences, not "because I said so" or "because I heard God say so".
Given Said Achmiz’s comment already has 11 upvotes and 2 agreement points, should I write a post explaining all this? I had thought it all rather obvious to anyone who looks into evolutionary ethics and thinks a bit about what this means for moral philosophy (as quite a number of moral philosophers have done), but perhaps not.
I’m afraid that what you’ve written here seems… confused, and riddled with gaps in reasoning, unjustified leaps, etc. I do encourage you to expand this into a post, though. In that case I will hold off on writing any detailed critical reply, since the full post will be a better place for it.
Another question to ask, even assuming faultless convergence, related to uniqueness, is whether the process of updates has a endpoint at all.
That is, I could imagine that there exists series of arguments that would convince someone who believes X to believe Y, and a set that would convince someone who believes Y to believe X. If both of these sets of arguments are persuasive even after someone has changed their mind before, we have a cycle which is compatible with faultless convergence, but has no endpoint.
I'd rather your "that is" were a "for example". This is because:
- It's also possible for the process of updates to not be getting arbitrarily close to any endpoint (with a notion of closeness that is imo appropriate for this context), without there being any sentence on which one keeps changing one's mind. If we're thinking of one's "ethical state of mind" as being given by the probabilities one assigns to some given countable collection of sentences, then here I'm saying that it can be reasonable to use a notion of convergence which is stronger than pointwise convergence. For math, if one just runs a naive proof search and assigns truth value 1 to proven sentences and 0 to disproven sentences, one could try to say this sequence of truth value assignments is converging to the assignment that gives 1 to all provable sentences and 0 to all disprovable sentences (and whatever the initialization assigns to all independent sentences, let's say), but I think that in our context of imagining some long reflection getting close to something in finite time, it's more reasonable to say that one isn't converging to anything in this example — it seems pretty intuitive to say that after any finite number of steps, one hasn't really made much progress toward this kinda-endpoint (after all, one will have proved only finitely many things, and one still has infinitely many more things left to prove). Bringing this a tad closer to ethical reality: we could perhaps imagine someone repeatedly realizing that projects they hadn't really considered before are worth working on, infinitely many times, with what they are up to thus changing [by a lot] [infinitely many times]. To spell out the connection of this to the math example a bit more: the common point is that novelty can appear in the sentences/things considered, so one can have novelty even if novelty doesn't keep showing up in how one relates to any given sentence/thing. I say more about these themes here.
I feel sad that your hypotheses are almost entirely empirical, but seem like they include just enough metaethically-laden ideas that you have to go back to describing what you think people with different commitments might accept or reject.
My checklist:
Moral reasoning is real (or at least, the observables you gesture towards could indeed be observed, setting aside the interpretation of what humans are doing)
Faultless convergence is maybe possible (I'm not totally sure what observables you're imagining - is an "argument" allowed to be a system that interacts with its audience? If it's a book, do all people have to read the same sequence of words, or can the book be a choose your own adventure that tells differently-inclined readers to turn to different pages? Do arguments have to be short, or can they take years to finish, interspersed with real-life experiences?), but also I disagree with the connotation that this is good, that convergence via argument is the gold standard, that the connection between being changed by arguments and sharing values is solid rather than fluid.
No Uniqueness
No Semi-uniqueness
Therefore Unificiation is N/A
I feel sad about this too. But this is common in impure scientific disciplines, e.g. medical studies often refer to value-laden concepts like proper functioning. The ideal would be to gradually naturalize all of this so we can talk to each other about observables without making any assumptions about interpretation of open-textured terminology. What I want to show here is primarily an existence proof that we can fully naturalize this discussion, but I haven't yet managed to do this.
I think this is a very good question about arguments. And I do think we will have to make value judgments about what kinds of moral deliberation processes we think are "good" otherwise we are merely making predictions about behaviour rather than proposing an approach to alignment. An end result I would like would be one where the moral realist and the antirealist can neutrally discuss empirical hypotheses about what kinds of arguments would lead to what kind of updating, and discuss this separately from the question of which kinds of updating we like. This would allow for a more nuanced conversation, where instead of saying "I'm a realist therefore keep the future open" or "I'm an antirealist therefore lock it down" we can say "Let's set aside capital letters and talk about what really motivated people in moral cognition. I think, empirically, this is how people reason morally and what people care about; personally, I want to make X intervention in the way people reason morally and would invite you to agree with me."
This is very well put, and I think it drives at the heart of the matter very cleanly. It also jives with my own (limited) observations and half-formed ideas about how AI alignment also in some ways demands progress in ethical philosophy towards a genuinely universal and more empirical system of ethics.
Also, have you read C.S. Lewis' Abolition of Man, by chance? I am put strongly in mind of what he called the "Tao", a systematic (and universal) moral law of sorts, with some very interesting desiderata, such as being potentially tractable to empirical (or at least intersubjective) investigation, and having a (to my mind) fairly logical idea of how moral development could take place through such a system. It appears to me to be a decent outline of how your naturalized moral epistemology could be cashed out (though not necessarily the only way).
This is the best proposal I've read for making progress on the question of "What should we point the ASI at?"
Are humans such that they can be brought through a series of arguments and endorse all of the premises in the arguments and end up with radically different moral views
I don't feel quite satisfied with this operationalization. It could Goodharted (especially by an ASI) by finding adversarial inputs to humans.
I also think someone could decide that their meta-preference is to endorse their unconsidered preferences and that the act of thinking about moral arguments is actually against their morality. This may sound absurd but probably accurately describes many people's responses to drowning child thought experiments. Most people don't act as if they have utility functions and I don't see why people should be required to adopt one. To the extent that it matters a lot what people think about their preferences upon reflection, they will probably notice the inconsistencies in their preference once they start Dutch-booking themselves by manifesting inconsistent preferences with the help of ASI.
kinds of questions we know how to solve, and could solve with the help of AI
I predict that once people can talk to ASIs, many will quickly see the bizareness of the realist view and a lot more people will start saying "I don't know what you're talking about, can you please make this clearer?".
So many disagreements… Moral philosophy is not functionally useful: it just enumerates all the ways one could agree to disagree. It doesn't actually answer any questions, it just lists and categorizes the possible answers and arguments, including things like disagreeing whether a moral question even has an answer (that's Realism vs Antirealism for you).
Are there any more practical alternatives that might interest a rationalist? Yes, there are. I wrote an entire sequence about them. Rather than my trying to sketch them here, go read that, comment on and discuss it.
Curated and popular this week