Followup toIs Morality Preference?

In the dialogue "Is Morality Preference?", Obert argues for the existence of moral progress by pointing to free speech, democracy, mass street protests against wars, the end of slavery... and we could also cite female suffrage, or the fact that burning a cat alive was once a popular entertainment... and many other things that our ancestors believed were right, but which we have come to see as wrong, or vice versa.

But Subhan points out that if your only measure of progress is to take a difference against your current state, then you can follow a random walk, and still see the appearance of inevitable progress.

One way of refuting the simplest version of this argument, would be to say that we don't automatically think ourselves the very apex of possible morality; that we can imagine our descendants being more moral than us.

But can you concretely imagine a being morally wiser than yourself—one who knows that some particular thing is wrong, when you believe it to be right?

Certainly:  I am not sure of the moral status of chimpanzees, and hence I find it easy to imagine that a future civilization will label them definitely people, and castigate us for failing to cryopreserve the chimpanzees who died in human custody.

Yet this still doesn't prove the existence of moral progress.  Maybe I am simply mistaken about the nature of changes in morality that have previously occurred—like looking at a time chart of "differences between past and present", noting that the difference has been steadily decreasing, and saying, without being able to visualize it, "Extrapolating this chart into the future, we find that the future will be even less different from the present than the present."

So let me throw the question open to my readers:  Whither moral progress?

You might say, perhaps, "Over time, people have become more willing to help one another—that is the very substance and definition of moral progress."

But as John McCarthy put it:

"If everyone were to live for others all the time, life would be like a procession of ants following each other around in a circle."

Once you make "People helping each other more" the definition of moral progress, then people helping each other all the time, is by definition the apex of moral progress.

At the very least we have Moore's Open Question:  It is not clear that helping others all the time is automatically moral progress, whether or not you argue that it is; and so we apparently have some notion of what constitutes "moral progress" that goes beyond the direct identification with "helping others more often".

Or if you identify moral progress with "Democracy!", then at some point there was a first democratic civilization—at some point, people went from having no notion of democracy as a good thing, to inventing the idea of democracy as a good thing.  If increasing democracy is the very substance of moral progress, then how did this moral progress come about to exist in the world?  How did people invent, without knowing it, this very substance of moral progress?

It's easy to come up with concrete examples of moral progress.  Just point to a moral disagreement between past and present civilizations; or point to a disagreement between yourself and present civilization, and claim that future civilizations might agree with you.

It's harder to answer Subhan's challenge—to show directionality, rather than a random walk, on the meta-level.  And explain how this directionality is implemented, on the meta-level: how people go from not having a moral ideal, to having it.

(I have my own ideas about this, as some of you know.  And I'll thank you not to link to them in the comments, or quote them and attribute them to me, until at least 24 hours have passed from this post.)


Part of The Metaethics Sequence

Next post: "The Gift We Give To Tomorrow"

Previous post: "Probability is Subjectively Objective"

New Comment
101 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

I don't think anyone can really argue that a large-scale decrease in global violence and violent death is a sign of moral progress. So I must point to this Steven Pinker conference where he lays out some statistics showing the gradual decline of violence and violent death throughout our history:

This has actually been trenchantly criticized on statistical grounds. The basic idea is that if the Cuban Missile Crisis(or numerous other similar events) had ended badly the conclusion would have been reversed. And according to people who were there, such as President John F Kennedy, it very well could have ended badly.
I haven't looked at the original criticism, but the "basic idea" as you describe it seems to introduce a source of bias: we have more visibility of luckily avoided ways in which things could have gone badly for recent events than for older ones, so if you try to take those into account then you will skew the change over time in the direction opposite to the one Pinker claims. (If you also look for unluckily avoided ways in which things could have gone well then maybe the bias goes away.)

You can only answer the question if you have some sort of answer to the question, "What is moral?" If democracy is moral, then the first democrats got their walking randomly when they accidentally stepped on the "Golden path" of moral progress. Luckily they were able to recognize it as such.

Also, assuming that society would settle on Chimp rights correctly (however correctness is determined) either as human or not, is assuming your conclusion (or building an experiment that might not bear fruit until your unfrozen).


Readers Note:

Since there is a ‘directionality’ to physics (ie. the universe moves from a simpler to a more complex state), and there is also an analogue to a ‘directionality’ in logic/mathematics (ie more complex ideas are built from simpler ideas), isn’t it apriori highly plausible that there’s also an analogue to a ‘directionality’ in the realm of values (ie. moral progress)?

Let me remind all readers that years ago I speculated on multiple transhumanist lists there may be three different ways to define causality. I don’t see a difference between ‘caus... (read more)

" the future will be even less different from the present than the present."

instead of

" the future will be even less different from the present than the present from the past."


Why is there a direction to the shifting moral zeitgeist?

E.g. see the work of Robert Wright: How cooperation (eventually) trumps conflict

Technology is the single most important thing for morality. As technology allows better resources, communication, documentation, safer paths for society emerge as in the difference between bonobos and chimps, where resources makes the species less aggressive. Also when we become economically dependent on each other due to specialization and can be held responsible for our actions due to documentation, the threshold for cheating increases. Also we seem to want to generalize as many principles we dare to, if we are healthy, feel safe and have plenty of resou... (read more)

Technology is power. It could also enable a boot stomping human face forever.

I'm by no means sure that the idea of moral progress can be salvaged. But it might be interesting to try and make a case that we have fewer circular preferences now than we used to.

Wiseman, if everyone were blissed-out by direct stimulation of their pleasure center all the time, would that by definition be moral progress?

Marshall, how is your "usefulness" not isomorphic to the word "good"? Useful for what?

Lowly Undergrad, early societies didn't have this idea of reducing violent death to zero - through what mechanism did they acquire this belief, given that they didn't start out with the idea that it was "moral progress"?

Robin Brandt, is whatever increasing technology does to a society, moral progress b... (read more)

I realize it's been years, but - didn't early societies want to encourage peace (in general, since the Hated Enemy obviously needs to be fought) and reduce violent crime? My model of early societies does, in fact, have something roughly isomorphic to "reduce violent death", even if they don't explicitly extrapolate this all the way to "someday, violent death should be nonexistent" - and, let's face it, most modern societies don't really do this either, it's just too far away. Do you have a source for asserting otherwise? Or, if you've changed your mind, do you remember why you claimed this?

If you take the list of things that were moral yesterday and the list that are moral today, and look for pairs between the lists that are kind of the same idea, but just in different quantity (e.g. like and love) then you could step back and see if there is an overall direction.

The key idea is to recognize when two things with different names are really different amounts of some higher more abstract idea.

Eliezer: Wiseman, if everyone were blissed-out by direct stimulation of their pleasure center all the time, would that by definition be moral progress?

Compared to todays state of affairs in the world? Yes, I think that would be enormous moral progress compared to right now (so long as the bliss was not short term and would not burn out eventually and leave everyone dead. So long as the bliss was of an individual's choice. So long as it really was everyone in bliss, and others didn't have to suffer for it. Etc. etc.)

The best discussion of moral progress I've seen yet is in Heinlein's Starship Troopers, where morality progresses by becoming more inclusive. Once, it was family, everyone else was fair game; then, tribe, race, religion, nation, now we recognize (at least officially, there are still many on lower "rungs") the human species as being deserving of our consideration. In Starship Troopers, Heinlein had one of his teachers in "History and Moral Philosophy" say that they were developing morality for dealing with intelligent aliens.

For the co... (read more)

Paul, do you think that your own morality is optimum or can you conceive of someone more moral than yourself - not just a being who better adheres to your current ideals, but a being with better ideals than you?

Yes I can.

If you take the view that ethics and aesthetics are one and the same, then in general it's hard to imagine how any ideals other than your own could be better than your own for the obvious reason that I can only measure them against my own.

What interests me about the rule I propose (circular preferences are bad!) is that it is exclusively a... (read more)

One possibility: we can see a connection between morality and certain empirical facts -- for example, if we believe that more moral societies will be more stable, we might think that we can see moral progress in the form of changes that are brought about by previous morally related instability. That's not very clear -- but a much clearer and more sophisticated variant on that idea can perhaps be seen in an old paper by Joshua Cohen, "The Arc of the Moral Universe" (google scholar will get it, and definitely read it, because a) it's brilliant, an... (read more)


A few processes to explain moral progress (but probably not all of it): a) Acquiring new knowledge (e.g. the knowledge that chimps and humans are, on an evolutionary scale, close relatives), which leads us to throw away moral judgements that make assumptions which are inconsistent with such knowledge. b) Morality is only one of the many ends that we pursue, and as an end it becomes easier to pursue once you are amply fed, watered and clothed. In other words, improvements in material conditions enable improvements in morality. c) Conquest of one culture by ... (read more)

"the knowledge that chimps and humans are, on an evolutionary scale, close relatives"

So what? The differences are so profound that humans should be considered a different class, maybe even a new phylum. The basic one is possession of language and culture. "Animal rights" is a stupid idea. I am against mistreatment of animals, but recognize that it is more an aesthetic than ethical position.

Eliezer:Robin Brandt, is whatever increasing technology does to a society, moral progress by definition, or does increasing technology only tend to cause moral progress?

I see, I answered quite a different question there, I had a funny feeling of that while writing that comment.

Increasing technology tends to cause moral progress yes, by making moral choices economically and experientially(as in our experience of things) more strategic/optimal. It all boils down into satisfying our adapted pattern-recognizers that gives us pleasure or a feeling of righteousn... (read more)

1) Supposing that moral progress is possible, why would I want to make such progress?

2) Psychological experiments such as the Stanford prison experiment suggest to me that people do not act morally when empowered not to do so. So if I were moral I would prefer to remain powerless, but I do not want to be powerless, therefore I perform my moral acts unwillingly.

3) Suppose that agents of type X act more morally than agents of type Y. Also suppose that the moral acts impact on fitness such that type Y agents out-reproduce type X agents. If the product of popu... (read more)

My view is similar to Robin Brandt's, but I would say that technological progress has caused the appearance of moral progress, because we responded to past technological progress by changing our moral perceptions in roughly the same direction. But different kinds of future technological progress may cause further changes in orthogonal or even opposite directions. It's easy to imagine for example that slavery may make a comeback if a perfect mind control technology was invented.


@billswift: I do not want to divert the thread onto the topic of animal rights. It was only an example in any case. See Paul Gowder's comment previous to mine for a more detailed (and different) example of how empirical knowledge can affect our moral judgements.

Marshall, how is your "usefulness" not isomorphic to the word "good"? Useful for what?

I suppose I just want to avoid the preachiness of the word good. It is unfortunately coherent to die for goodness. It is not very useful to die for usefullness.

Useful for what? This doesn't seem like a useful question. Usefulness is obvious and thus no need to ask.

I do not wish to lose my way or be carried away by the bigness of the nominalisation "morality". Occam's Razor should also be applied here - in a pleasant and gentle way.

If one defines morality in a utilitarian way, in which a moral person is one who tries for the greatest possible utility of everyone in the world, that sidesteps McCarthy's complaint. In that case, the apex of moral progress is also, by definition, the world in which people are happiest on average.

It's easy to view moral progress up to this point as progress towards that ideal. Ending slavery increases ex-slaves' utility, hopefully less than it hurts ex-slaveowners. Ending cat-burning increases cats' utility, hopefully less than it hurts that of cat-burnin... (read more)

Re: if we all cooperated with each other all the time, would that by definition be moral progress?

If we all cooperated with each other all the time, that would be moral progress.

Moral progress simply means a systematic improvement of morals over time - so widespread cooperation would indeed represent an improvement over today's fighting and deceit.

It's harder to answer Subhan's challenge - to show directionality, rather than a random walk, on the meta-level.
Even if one is ignorant of what humans mean when they talk about morality, or what aspects of the environment influence it, it should be possible to determine whether morality-development over time follows a random walk empirically: a random walk would, on average, cause more repeated reversals of a given value judgement than a directional process.
For performing this test, one would take a number of moral judgements that have changed in the pas... (read more)

Gee, this seems awfully similar to Timeless Physics, doesn't it?

A possibility that I have mentioned here before has to do with positive feedback loops in an isolated society between economic growth and luxury spending on moral coherence. On this account, people always had qualms about slavery but considered it to impractical to seriously consider abandoning it. When feeling rich they abandoned it anyway, either as conspicuous consumption or as luxury spending on simplicity. Having done so, it turned out, made them richer, affirming this sort of apparent luxury spending or conspicuous consumption as actually being mo... (read more)

Or as Saul Alinsky put it “[C]oncern with ethics increases with the number of means available and vice versa.” It is easy to be ethical when you have little at stake.

Re: direct stimulation of their pleasure center

Morality is normally concerned with conduct, not feelings.


"If everyone were to live for others all the time, life would be like a procession of ants following each other around in a circle."

Someone actually gets it right. Greed is moral. Greed is good.

Imagine a country that abolishes capital punishment, then, a few years later, brings it back. Have they made moral progress? Have they regressed? More importantly, who's to say?

Imagine also an alien who arrives on Earth, hears of what we've done with laws and societies and says 'what the hell? They've been morally regressing all this time?!'

Looking forward to the next post. The moral valuation of sentient/conscious matter over 'dumb' matter is something I have trouble wrapping my head around.

This has been mentioned many times, by Peter Singer, for instance, but one way towards moral progress is by expanding the domain over which we feel morally obligated. While we may have evolved to feel morally responsible in our dealings with close relatives and tribesmen, it is harder to hold ourselves to the same standards when dealing with whoever we consider to be not part of this group. Maybe we can attribute some of moral progress to a widening of who we consider to be a part of our tribe, which would be driven by technology forcing us to live and i... (read more)

Yes, this is what I was going to say. As time goes on, we seem to add more and more people and groups of people to the catagory we treat morally. Most major changes in morality over time you could describe this way; the elimination of slavery, women's suffrage, laws of war, better treatment for mental illness, even the idea that it's bad to torture cats for your own amusement could all be called "expanding the group of beings who we feel we have to treat ethically".
Gwern has a rather interesting refutation of this idea.
It's interesting, but its claim may be flawed. The fact is, to the absolute best of our ability to judge, neither gods nor dead people actually exist. So it is not the case that there exist entities that have been pushed out of the circle. On the other hand women and minorities and cats do exist. And they have to various extents been brought into the moral circle. To the extent that the categories of existent entities and morally relevant entities have increased their overlap, that's progress. Or it's at least movement in a consistent direction.

Still waiting for someone to take the necessary first step towards a rational understanding of the issue.

Any time now, folks.

Maybe I had better not join the discussion, I just want to say that nearly everyone you will ever meet, they get something and they try to hold onto it for as long as possible, and all their actions are defined by this.

Also everyone will argue what they are hardwired for: sex and eating, till they turn blue.

If we all cooperated with each other all the time, that would be moral progress. -- Tim Tyler

I agree with Tim. Morality is all about cooperation.

If everyone were to live for others all the time, life would be like a procession of ants following each other around in a circle. -- John McCarthy, via Eliezer Yudkowsky

This is a reductio ad absurdum argument against the idea that morality is an end. I agree with what it implies: Morality is a means, not an end. Cooperation is a means we each use to achieve our personal goals.


As I said previously, I think "moral progress" is the heroic story we tell of social change, and I find it unlikely that these changes are really caused by moral deliberation. I'm not a cultural relativist but I think we need to be more attuned to the fact that people inside a culture are less harmed by its practices than outsiders feel they would be in that culture. You can't simply imagine how you would feel as, say, a woman in Islam. Baselines change, expectations change, and we need to keep track of these things.

As for democracy, I think ther... (read more)

I think a lot of people are confusing a) improved ability to act morally, and b) improved moral wisdom.

Remember, things like "having fewer deaths, conflicts" does not mean moral progress. It's only moral progress if people in general change their evaluation of the merit of e.g. fewer deaths, conflicts.

So it really is a difficult question Eliezer is asking: can you imagine how you would have/achieve greater moral wisdom in the future, as evaluated with your present mental faculties?

My best answer is yes, in that I can imagine being better able to... (read more)


Yvain: I think you're equivocating between two definitions of utility, "happiness" and "the quantity that's maximized". This dual meaning is really unfortunate.

Sebastian: moral progress might be random except that people (very plausibly) try not to return to a rejected past state. This would be directionless (or move in an arbitrary direction) but produce very few reversals.

poke: pursuing knowledge could be painful and depressing but still intuitively moral.

I see a bit of what looks like terminal/instrumental confusion in this thread. I... (read more)

A particularly interesting question is, what would people of e.g. Roman empire or mediaeval France think about today's society. We can compare the morality of the past with contemporary standards, but we can't see the future. I wonder whether mediaeval people would find our morality less despicable than we find theirs. If such comparison was possible, one could define some sort of objective (or subjectively objective?) criterion - simply put together two societies with different moral codes and watch how many will convert from first to the second and vice ... (read more)

Some changes in morality come about because people notice that their previous ideas contained incorrect probability assessments. These changes can be considered moral progress.

Example: people find a logical inconsistency in their moral thinking, and correct for it.

Example: people notice that they have been assuming it necessary to be Homo sapiens or to be of a specific gender or color in order to have conscious experience, and that they don't actually have any basis for such an assumption.

As long as our knowledge about the universe (including our own thoug... (read more)


I don't think discovering better instrumental values toward the same terminal values you always had counts as moral progress, at least if those terminal values are consciously, explicitly held.

Why on earth not? Aristotle thought some people were naturally suited for slavery. We now know that's not true. Why isn't that moral progress?

(Similarly, general improvements in reasoning, to the extent they allow us to reject bad moral arguments as well as more testable kinds of bad arguments, could count as moral progress.)

A moral state X represents progress from moral state Y if people in both moral state X and moral state Y agree that X is better after being presented with the arguments. That is, X represents progress from Y if all it takes is the right way of thinking about it to convince someone from Y to move to X.

Paul, I think values and beliefs have both changed in that case - we (I hope I'm right to generalize!) don't judge that any facts about a person could make it right to enslave them. Most of us have scrapped the whole teleological framework Aristotle used to say that.

I probably should have said "...counts as the sort of moral progress Eliezer is talking about", the reason being that updating beliefs/instrumental values isn't a matter of metaethics, and is unproblematically directional.

Nominull, in your first sentence, does "people" mean ... (read more)


Fair enough, but consider the counterfactual case: suppose we believed that there were some fact about a person that would permit enslaving that person, but learned that the set of people to whom those facts applied was the null set. It seems like that would still represent moral progress in some sense.

Perhaps not the sort that Eliezer is talking about, though. But I'm not sure that the two can be cleanly separated. Consider slavery again, or the equality of humanity in general. Much of the moral movement there can be seen as changing interpretati... (read more)

"[W]e (I hope I'm right to generalize!) don't judge that any facts about a person could make it right to enslave them."

I'm not so sure, Nick. Taboo person, and consider our treatment of certain nonhuman animals.

People will always consider their own beliefs moral and those of their predecessors who disagreed to be less so. People who believe in "moral progress" are adherents of a religion, whether they recognize it or not.

"Lowly Undergrad, early societies didn't have this idea of reducing violent death to zero - through what mechanism did they acquire this belief, given that they didn't start out with the idea that it was "moral progress"?"

While it is certainly difficult to imagine the mindset of people who existed ten of thousands of years before us, I think since they were still human beings, we can assume they were somewhat similar to you and I. From this basic assumption think we can look toward Peter Singer's philosophy of the moral circle. The s... (read more)

Why on earth not? Aristotle thought some people were naturally suited for slavery. We now know that's not true.
No, we don't. We know no such thing.

Morally, we do know such a thing.
This sounds like an is-ought confusion. "Some people would be happier as slaves." is an is-statement -- it's either right or wrong (true or false) as a matter of fact, regardless of morality. "Slavery oughtn't exist" is a moral statement -- it only has a truth value according to a particular ethical/moral set. I don't know whether "naturally suited for slavery" is supposed to be a "is" or an "ought" statement (descriptive or prescriptive). If it's an is-statement then our moral sense is irrelevant to whether the statement is true or false as a matter of fact.
I agree generally with your point, but this sentence assumes "happier" is an objective quality - which may not be true. if we were to taboo "happier" in that sentence, the new phrasing might include a moral claim. Consider: "Everyone is happier if jocks can haze nerds without complaint" --> "Jocks by show virtue by hazing nerds, and nerds show virtue by accepting hazing without complaint." The second sentence contains a number of explicit and implicit moral claims. Those moral claims are also present in the first sentence, just concealed by the applause light word "happy."
That; something we don't know. Moral statements might be uniformly false (error theory), neither true nor false (expressivism), have singe non-relative truth values (moral realism) etc.
Just to mention my own view on the subject in a single line (though I ought really illustrate it with examples): I'm guessing that the moral algorithm in our brains is executing an unconscious estimation of our preferences while attempting a depersonalization of context. As such a moral statement is on the one hand dependent on individual preferences (moral values) but on the other hand it can also be true/false on an objective level, as some parts of the above algorithm are objective and some are subjective.
But you might believe it's moral to increase net happiness, or moral to enforce people's rights, one of which is a right to personal autonomy. So it's truth value is still only determined "according to a particular ethical/moral set."
No., because your beliefs might be wrong. Aris was asserting relativism against error theory, non-congitivism and realism. Relativism is the claim that some set of statements have truth values, and have truth values that are relative to something. Relativism is not proven simply by producing evidence of conflicting beliefs.
A small note: I don't consider myself a moral relativist, though I understand why my statement was misunderstood as such, as I was vastly simplifying my position. My actual position on morality probably needs a full discussion post to be fully explained. I do think that for any X, X can only be called "morally wrong" according to some moral/ethical system, but I also think that there may exist objective criteria which might invalidate some moral/ethical systems altogether, or even rank (at least somewhat) the validity of the various moral/ethical systems.
I'm well aware they could be wrong - in fact that's my whole point. The answer depends on which moral theory is true - even though only one theory is actually true. It's a counterfactual. Since Aris says they're not a moral reletavist, I suspect this is at least similar to what they intended. If it's not, I'd still endorse it.
why would an "ethciical/moral set" be what makes a moral claim (realistically) true? Realists tend to think claims are rendered true by some sort of fact.
The position of realism + belief in such a fact is one such "ethical/moral set", as I meant it. I think that may have come across as referring to different terminal values or something?
Realists think claims are made true by facts, not beliefs in facts.
Right, but it depends which facts are true. The answer is contingent on these disputed facts. ETA: edited for wording.
Sure. Which theory will be believed to be true depends on which facts are believed to be true, and which theory is actually true will depend on which facts are actually true. But beleiving one thing to be true becuae you believe another to be tirue is non argument for relativism, although careless wording can make it seem that way.
Sorry, I'm having trouble parsing that :( Possibly because I'm misreading the typos?
I believe we've established that shminux's position allows for some models to be made more accurate than others by events, not beliefs in events. I think these two positions are analogous.
Different conversation ;) We're talking about moral realism here, sort of.
Whoops! You're entirely right; I should have read the parent-tree before responding, but I was confident that I knew what the conversation was. Retracted.
Do we, though? I think most people would agree that it's acceptable to enslave sentient creatures that are actually happy under such a system - albeit reluctantly, due to the signalling implications - and this seems consistent with the fact that historically, societies kept slaves believed this (if not alieved it.)

There is a tendency for older generation to feel nostalgic for the time of their youth and for the younger generation to strive for changing the status quo. So I wonder whether the modern perception of moral progress (as opposed to perennial complaints of moral degradation popular among our ancestors) comes from the youth being more economically and politically empowered than ever before, which allows it to dominate public discourse.

i also consider morality to be about cooperation... in this sense moral progress predates humanity specifically i consider the evolution of multicellular life to be an example of moral progress

@Paul Gowder

What Caledonian said.

Not true? Please, please post about that. Not about moral progress etc, but how you have come to hold that any moral belief can be an objectively true belief. This is surely what 'we now know that's not true' implies.

Aristotle would probably ask you for evidence that he is flatly wrong. He might also ask you why your judgment is true, and his is not.

While I might not agree that we should enslave anyone, I'd certainly have the courtesy to admit to Aristotle that a moral is only as true as a society and an era holds it to be.... (read more)

* I wouldn't want "enslave him" to become a universal law (Kant I) * Enslaving people treats them as means, not ends (Kant II) * I wouldn't want to become a random member of society that permits slavery (Rawls) etc

Not everyone has the same intuition about the wrongness of slavery, though, and "they're not us and they're more use to us this way" is justification enough for some. People have divergent intuitions about empirical and logical propositions, too, but in those cases there's an obvious (if not always practical) way to settle things: go and look, or find a (dis)proof. You can trivially demonstrate that 1+1≠3, but it's hard to see how you could reject with nearly as much rigor even something as ridiculous as "it's good to enslave people born o... (read more)


Thanks for the clarification.

I submit that's enough to constitute all the knowledge we need to say that kind of behavior is immoral.

So we're saying what we think is moral based on our knowledge. I'd say that's pretty watertight. We know what we feel is right, but the more we can tie it to objective facts about the world, the stronger our position. However, I'd still argue that we can never move beyond merely believing in our morals, by definition. (Yes, I said it!) The moment we state that we know that our morals are true for all time and space, we're setting ourselves up for a fall that we can't recover from.

Sorry for repost, but note also that my earlier comment made no reference to slavery, and I of course agree that slavery isn't right. My beef was with the assertion of a true moral.

Whether you agree with it or not, Obama's "moral progress" means a change in US law to comport more closely to (his present view of) morality, not a change in the moral views of Americans. It is quite possible to view oneself as the apex of possible morality and still believe in the possibility of moral progress on other people's part.

I disagree with Obama because I disagree with some of the goals of his morality, but I don't see that as any reason to attack his semantics.


I see moral progress as 1) increased empathy, defined as increasingly satisfying, increasingly accurate mental models of sentient beings, including oneself, and 2) increased ability to predict the future, to map out the potential chains of causality for one's actions.

Inspired by this article I think one way of doing it might be to show directionality in terms of evolutionary novelty. That is, look at what parts of our evolutionary psychology we have rationally worked against as a culture, and why we came to those more intellectual conclusions. That way, the measure of our progress could be in how we learn to fix the mistakes of the stupid natural selection.

However, that sounds a lot to me like r... (read more)

Part of the answer could lie into "what would someone teleported to another culture think ?" I don't think it totally solves the question, but it's a hint, or a part of the answer.

If you take someone from now, and he's teleported to dark ages, with absolute monarchy, serfdom, capital punishment with the most horrible ways of killing, torture, ... he will be horrified.

If you take someone from the dark ages and teleport him now, he'll probably be very lost at first, but I don't think he would be horrified by the fact we manage to take more-or-less reasonable decisions using democracy (at least as reasonable at what the kings used to do), that the society doesn't collapse into crime and chaos when we suppress death penalty, serfdom, torture, ...

Many people who, in the past, advocated the use of what we now consider barbaric (torture, death penalty, dictatorship, ...) did it saying "there is no alternative", "if we don't maintain order, it'll be chaos and everyone will murder each other", "if you don't have a king, no decision will be taken", ...

The same applies to points which are debated right now in western societies, like "painless"... (read more)

...the fact we manage to take more-or-less reasonable decisions using democracy (at least as reasonable at what the kings used to do), that the society doesn't collapse into crime and chaos when we suppress death penalty, serfdom, torture, ...

Recently, it has been quite fashionable on LW to profoundly disagree with all of those points. At the very least, someone's going to say that, when an attempt to suppress slavery was made, the US society did for a while collapse into chaos unheard of before or since.

Speaking quite frankly (and in purple prose), though, there are few other things in the realm of the mind I'd desire right now than to be able to trust securely in all those points, and rest well, knowing that the job of SIAI and partly LW is simply to fight our way upwards before the sky comes crashing down - not also to run as fast as possible from the eldritch monster born of our own shadow!

Temporary chaos frequently happen when changes are made - but that's not what I was referring to. The issue of "will chaos occur when moving from slavery to no slavery" is a different issue than "would a society without slavery be more chaotic". That can justify inertia (keeping things as they are), but is not in itself an argument for or against slavery (or anything else). And that fact that despite that inertia we still see things like torture or slavery mostly disappearing is a good indicator or moral progress.

Eh, I'm just not the go-to guy here. You should try talking to people like:

  • sam0345 (low-level combat tutorial)

  • TGGP (online co-op mode)

  • Aurini (MEDIUM) - and he might end up just opening the gate and letting you pass if you look like enough of a bro - has recently been witnessed in a brawl against a pick-up raid. Pick-up, get it? Get it? Eh heh!

  • Konkivistador (HARD)

  • steven0461 (BONUS CONTENT; need the Meta^2-Contrarian Edition DLC to unlock - BUY NOW for only LW$ 5499)

  • Vladimir_M (VERY HARD)

  • ??? (IMPOSSIBLE)



Edit: Lyrics need to be included obviously:

Test your mind, Test your mind,
Test your mind, Test your mind. 
Konkvistador, TGGP, Roko, Will_Newsome,
steven, cousin_it, Vladimir.
Konkvistador, TGGP, Roko, Will_Newsome,
steven, cousin_it, Vladimir.
(Modus ponens!)
(Ceteris paribus)
(Aumann's agreement)
Test your mind, Test your mind.
Konkvistador, TGGP, Roko, Will_Newsome,
steven, cousin_it, Vladimir.

Since I'm apparently a stepping stone on the path to the Final Boss of the contrarian Internet, I wonder what my fatality is.

So, we have an agreement that outright flattering each other in the future shall be reprociated with positive karma loops, as long as it's done in a sufficiently nerdy manner? C'mon, bro, just say yeah!


Past behaviour is an excellent predictor of future behaviour. Nerdy flattery and humour seem to be consistently rewarded on LessWrong.

:reads the edit: Now you're just adding insult to injury, except that "injury" is "awesomeness" and "insult" is "nostalgia".

We are glad to announce an upcoming full-fledged expansion pack: 'The Twisting Way'

Engage the enigmatic genius Will_Newsome and rescue Lady AspiringKnitter from his unspeakable experiments; survive the shamanistic Rites of Hanson (not for the sake of survival!); endure stigma and uproar as you optimize your threads for the gaze of the feared Outsiders; boldly embark upon the Doomed Quest for Mencius' Magnificient Monocle, and more!

I like the criteria above. If people on one side are arguing that x is "necessary" and people on the other are arguing that x is "horrible", then it should be clear that x is horrible and something should be done about it. (make x less horrible, find an alternative to x, remove whatever makes x necessary) Applies well to things like medical testing on animals, prisons, and death.
This is interesting. I was going to offer what I thought were counterexamples, such as abortion, masturbation, and drug use, but now I'm not so sure. For one thing, the "something" that should be done might simply be to prevent anyone from feeling horror (such as for masturbation) and for another it seems that there should be ways to mitigate the negative consequences of 'horrible' things (such as for abortion, by transferring an unwanted fetus to an artificial womb for adoption, or deliberately mitigating the unwanted side effects of drugs).
Seems like there are more such people than we'd expect. (Are you in Europe too?)

The discussion in the comments has been interesting, but I believe I have a simple answer to Eliezer's question (please tell me if I am mistaken). Consider a society that has a moral idea say, like valuing bodily autonomy, but they don't give woman that right. They often kill women for the organs to give to men and children, due to an old tribal culture mainly forgotten. Unfortunately, certain rituals and dogma still continue on. One day, a leading public intellectual points this out on tv, and they change their actions to fit in with their true moral beliefs, and stop acting on non-moral ones. Wouldn't this be an example of moral progress?

Consider a different society that has a moral idea like valuing the bodily autonomy of non-women, but for various historical reasons this has historically been expressed as "valuing bodily autonomy" without specifying gender. Their behavior has been identical to the example you give, until one day someone points this out, and they start expressing it as "valuing bodily autonomy for non-women" instead, while continuing to do everything else the way they used to. Is this also an example of moral progress? If not, why not?
3Ben Pace
I see. I've said that if people become more aligned with their meta-morals in practice, then it is progress... And you've offered that their meta-morals might seem or be bad anyway, so it wouldn't seem to be progress to us. I suppose, to be able to show my progress to be directional and not arbitrary, I'd have to present a perfect, objective basis for morality. I won't be doing that in this post (sorry) so my point is redundant. Thanks for clearing that up with me.

I was going to say something about moral progress being changes in society that result in global increase in happiness, but I ran into some problems pretty fast following that thought. Hell, if we could poll every single living being from 11th century and 21st century and ask them to rate their happiness from 1-10 why do I have a feeling we'd end up with same average in both cases?

If you gave me exensional definition of moral progress by listing free speech, end of slavery and democracy, and then ask me for intensional definition, I'd say moral progress is global and local increase in co-operation between humans. That does not necessarily mean increase in global happiness.