Question for Obert: Suppose there are intelligent aliens in a galaxy far far away... There is a pretty good chance they will discover math. They might use different symbols and they might represent their data differently but they will discover math because the universe pretty much runs on math. To them 2 + 3 will equal 5. Would they discover morality? Would their 'morality' be the same thing as our 'morality' here? Does morality converge into one thing like math does, no matter where you start from?
There are some aliens here on earth already. Chimpanzees. They're pretty alien. Do they have math?
If superintelligent aliens found us, would they think we had math?
The universe runs on math? Really? I'd say the universe runs on the universe. We model how the universe runs using math. The map is not the territory.
By "aliens", I think PK2 meant "intelligent aliens able to build a technological civilization". And for that, you do need "maths".
As for the universe "running on maths", he added "pretty much", which is not very precise, but can mean something like "it really seems like the universe is running on maths", which is true. Our map of the universe relies on maths and is very accurate (making very accurate prediction on the universe itself), so any other accurate map of the universe has to include something which will look very much like maths. Two very accurate maps of the same territory will share many similar characteristics.
Admittedly OT meta-comment: I think Robin Hanson's an interesting thinker, and he has the discipline to self edit that this blog medium seems to require. Eliezer, on the other hand, seems both sophomorically self-impressed and logorrheic. Maybe there are interesting ideas lurking in these posts, but life is entirely too short to read them. Any chance of you guys getting, y'know, separate blogs, since the target audiences appear to be pretty different?
Anchovies have to die to make anchovy pizza, so depending on who you talk to it might still be immoral to eat anchovy pizza even if you want to.
There is not always a clear cut case that is best for every one, and part of morality is weighing the wants and needs of one being verses another in such cases.
morality is another one of those things that is true in the sense of consensus, which is a different meaning of truth from mathematical truth, or physical truth.
In the prisoners dilemma it is always advantageous to speak when the other player's action is fixed. If you get to be both players and decide what both players do then it is adventurous to make both players not speak. Morality amounts to getting to be both players.
In some other cases there may be multiple reasonable options to choose when you control all players involved, possibly depending on how you weigh the wants and needs of the players, and as such multiple reasonable conflicting moralities.
Anchovy pizza is a good example. You have to weigh a delicious meal for a human against killing fish.
Obert: "In some ways the human species itself strikes me as being a sort of toddler in the 'No!' stage." I am interested to know why.
I think the problem I have with the math example, and it may be that this is extensible to morality, is this:
If I have a certain quantity of apples, or sheep, or whatever, my mind has a tool (a number) ready to identify some characteristic about that quantity (how many it is). But that's all that number is: a tool. A reference.
Eliezer is right in saying that the teacher's teaching "2+3=5" doesn't make it true any more than the teacher's teaching "2+3=6" makes it true. But that's not because two plus three "actually" equals five. It's because we, as learning animals, have learned definitions of these concepts, and we conceive of them as being fundamental. We think of math as a fundamental part of reality, when it is in fact a low-level, extremely useful, but all-in-the-mind tool used to manipulate our understanding of reality. We're confusing the map with the territory.
Taking this over to morality:
"Killing is wrong" isn't true because someone told us it's true, any more than "Killing is right" would be true if someone were to tell us that. But that's not because killing another human being "actually" is wrong. It's because we, as learning animals, have learned (or evolved the low-level emotions that serve as a foundation for this rule) definitions of right and wrong, and we conceive of them as being fundamental. We think of morality as a fundamental part of reality, when it is in fact, an all-in-the mind tool. Should we throw it out because it's merely evolved? No. It's useful (at least for the species). But we shouldn't confuse the map with the territory.
This is still pretty fuzzy in my mind; please criticize, especially if I've made some fundamental error.
Keith Adams, if you're too busy to read the post and understand it, don't clog the comments.
Obert wins this one on points, but I'm still not convinced. Consider:
1) if humanity were extinguished in a flash, 2 + 3 = 5 would still hold. 2) if humanity were extinguished in a flash, murder would still be wrong.
While 1 is problematic in its own way, 2 just seems incoherent to me. Morality is generated by the actions of self-aware beings, and as such I can only ever think of it as a fluid abstract. Morality is as people do, nothing more. PK's aliens can conceivably hold theft to be perfectly moral.
One says 'murder is wrong'. I say 'show me the evidence'.
Another thing way to look at this idea of math being a tool that exists only in the mind has occurred to me:
Does addition happen outside the mind? What is something "plus" something else? If we've got a quantity of two sheep, and a quantity of three sheep, and they're standing next to each other, then we can consider the two quantities together, and count five sheep. But let's say a quantity of two sheep wander through a meadow until they come across a quantity of three sheep, and then stop. Where did the actual addition happen? Outside the mind, there are only quantities.
Physics is made of math. Physics describes the physical universe.
The physical universe is not made of math.
If you figure out a way to describe a physical event by taking some measured numbers and cutting them into an infinite number of pieces and putting them together in a different way, does that mean that the physical event happened by dividing something into an infinite number of pieces and rearranging them? Possibly. More likely the event just happened. The math comes from our need to start with the things we can measure and compute from that things that we want to know.
Would an alien society build the same math we do? Probably, if they happen to measure the same things and want to compute the same things. If they can easier measure something different or they care about something different then maybe not.
The idea that we shouldn't take things from other people without their permission is a human one. No predator has that idea about their prey. The concept of human ownership is complex and changeable too. We come up with lots of exceptions -- imminent domain, the abolition of slavery, etc. It's a matter of continuing negotiation. I'm not supposed to burn my own charcoal in my own backyard -- something about pollution. Is it really my own charcoal or my own land or my own air if I can't do as I want with them? If one of my neighbors tells on me the fire department will confiscate my grill. My five-year-old doesn't understand the complexities of the concept, but she knows how to argue about it. "We said we were going to share this toy and now Eris won't share!" "I want it now, she can have it later!" Sometimes my solution is to have a "toy timeout" and take it away from both of them for awhile.
How can anybody argue this stuff is simple and easy and obvious when anybody can see how much we argue about it?
GBM, I tend to agree with where you're going (or at least where you're starting). But saying that we shouldn't throw morality out because it's "useful" seems a little question-begging. It may indeed be useful for some purpose, X, but if someone doesn't care about X, this doesn't really give you much traction.
There's a big difference between saying "morality is the product of human minds" and saying "morality is purely arbitrary". Similarly, there's a big difference between saying "there are objective reasons why we make the moral judgments we do" and "all moral questions have objective answers which in no way depend on human minds".
Life is not a zero sum game. I think nearly everyone would agree that it would be advantageous to nearly everyone if one could somehow guarantee that neither one's self nor one's loved ones would be killed at the cost of forgoing the ability to kill one's enemies. I think this fact, not repeated arbitrary assertion, is the basis for the nearly universal belief that "murder is wrong". I think the fact that, in many societies, refraining from killing those outside one's own tribe does nothing to prevent those outside the tribe from killing one's self or one's loved ones, and not arbitrary bigotry, is the reason that in those societies killing those outside one's tribe does not count as murder.
When I first started reading the post, I had Keith's reaction, 'Get down to the point!', but I'm now very interested to see where Eliezer is going with this...
Obert: "I rather expect so. I don't think we're all entirely past our childhoods. In some ways the human species itself strikes me as being a sort of toddler in the 'No!' stage."
This in a way explains some of my own questions about my behavior... The first and only time I tried cocaine, I was shocked by just how much I loved it (I had thought it would be like smoking a joint and drinking three cups of coffee, fuck was I wrong)... And I thought to myself, "This is way too much fun, I don't care if you didn't crash, DON'T do it again." I think I realize that reactions that beyond my control, really are beyond my control, and thus should not be tampered with in my 'sophomoric' state.
I have to agree with PK and Ben; there's a heck of a lot more pressure for minds to converge on 2+3=5 than on any ethical statement. A mind that believes 2+3=6 will make wrong predictions about reality; a mind that has 'wrong' 'beliefs' about murder won't. (George has a point about game theory, but that's different from regarding someone else's death as terminally undesirable.) "The Platonic computation I implement judges murder as undesirable regardless of what anybody thinks" isn't the same as "murder is wrong regardless of what anybody thinks". I could define 'wrong' according to the output of my computation, but such an agent-relative definition would be silly.
...unless most humans converge to the same terminal values, in which case we could sensibly define "wrong" as the output of the computation implemented by humanity. There, it adds up to normality.
...well, kind of. That definition won't do by itself for moral arguments - it'd be like the calculator that computes "what does this calculator compute as the result of 2 + 3?" - any answer is correct. Some actual content is still needed.
Physics is made of math. Physics describes the physical universe. The physical universe is not made of math.
Two out of three isn't bad. The trick, of course, is to identify which two.
The universe itself is a giant RPOP. It started from a low-complexity state (low entropy density at the big bang) and moves to higher-complexity states (higher entropy density aka the 2nd law of thermodynamics) as time advances.
Whilst ‘morality’ in the sense of ‘computing possible courses of action’ is in the mind, ‘morality’ in the sense of ‘terminal values’ is not. It is only using the latter definition of ‘morality’ (ie. as denoting ‘moral archetypes’ or ‘terminal values’) that I’m claiming that morality is objective.
The objective terminal values are built into the optimization pressure for the RPOP of the universe manifesting as the 2nd law of thermodynamics.
In short; different regions of the universe could not interact with each in a consistent way, unless there was an implicit notion of ‘beauty’ (least action, symmetry, simplicity) built-in. For example, a QM field force interaction between two particles can be re-interpreted as a ‘communication’ between the two particles, which implies one particle generates an internal ‘representation’ of the other particle. These internal representations are governed by the ‘least action’ principles, in which the objective terminal values are implicit.
What if a neutrino storm made you believe "If a neutrino storm makes me believe 'stealing is wrong', then stealing is wrong'"?
Children in some cultures (including present day cultures) are taught by their parents and others that stealing from outsiders is right, and they believe it.
It's possible that after growing up they begin to realize that stealing is wrong but it is too late; they keep doing it anyway.
Just a typo alert:
"But if my taste in pizza changes, that changes the consequneces of eating, which changes the moral justification, and so the moral judgment changes as well."
consequneces-->consequences
"•The teacher, by saying that we're allowed to stand on our desks, can make it right to do so.
After reading the above
•The teacher, by saying that I'm allowed to take something from another child's backpack, cannot make it right to do so."
This^ was a relief.
If I want to bleep bleep bleep a consenting adult, it doesn't matter whether society approves.
Unless bleep bleep bleep has externalities.
In the case of 2 + 3 = 5, it's at least reasonable to wonder if math existed before humans. Physics itself seems to be made of math, and if we don't tell a story where physics was around before humans could observe it, it's hard to give a coherent account of how we got here.
This is wrong. Depending on how you define "math" and "physics", of course, but Eliezer should have been more careful with his word choice if he wanted to be unambiguous.
Wikipedia says, "Mathematics is the abstract study of topics such as quantity (numbers), structure, space, and change." "Abstract study" was not being conducted on Earth before humans existed; a fortiori, "math" did not yet exist. Next, physics. Once again, Wikipedia says, "Physics is the natural science that involves the study of matter and its motion through space and time, along with related concepts such as energy and force." Science is a human endeavor, so, once again, a fortiori, physics did not exist before humans did.
Physics is not "made of math". We describe physical laws using certain mathematical expressions, but to say that the laws themselves are those expressions is to confuse the map and the territory. As for giving a coherent account of how we got here, I think evolutionary biology does a fine job. In particular, it doesn't seem to conflict with our known physical laws, given our discovery of DNA, etc. Our own discoveries in physics tell us that we should expect that atoms existed before we did! And we arose from the interactions of those atoms. No incoherence there.
Followup to: Math is Subjunctively Objective, The Moral Void, Is Morality Given?
Thus I recall the study, though I cannot recall the citation:
Children, at some relatively young age, were found to distinguish between:
- The teacher, by saying that we're allowed to stand on our desks, can make it right to do so.
- The teacher, by saying that I'm allowed to take something from another child's backpack, cannot make it right to do so.
Obert: "Well, I don't know the citation, but it sounds like a fascinating study. So even children, then, realize that moral facts are givens, beyond the ability of teachers or parents to alter."
Subhan: "You say that like it's a good thing. Children may also think that people in Australia have to wear heavy boots from falling off the other side of the Earth."
Obert: "Call me Peter Pan, then, because I never grew up on this one. Of course it doesn't matter what the teacher says. It doesn't matter what I say. It doesn't even matter what I think. Stealing is wrong. Do you disagree?"
Subhan: "You don't see me picking your pockets, do you? Isn't it enough that I choose not to steal from you—do I have to pretend it's the law of the universe?"
Obert: "Yes, or I can't trust your commitment."
Subhan: "A... revealing remark. But really, I don't think that this experimental result seems at all confusing, in light of the recent discussion of subjunctive objectivity—a discussion in which Eliezer strongly supported my position, by the way."
Obert: "Really? I thought Eliezer was finally coming out in favor of my position."
Subhan: "Huh? How do you get that?"
Obert: "The whole subtext of 'Math is Subjunctively Objective' is that morality is just like math! Sure, we compute morality inside our own brains—where else would we compute it? But just because we compute a quantity inside our own brains, doesn't mean that what is computed has a dependency on our own state of mind."
Subhan: "I think we must have been reading different Overcoming Bias posts! The whole subtext of 'Math is Subjunctively Objective' is to explain away why morality seems objective—to show that the feeling of a fixed given can arise without any external referent. When you imagine yourself thinking that killing is right, your brain-that-imagines hasn't yet been altered, so you carry out that moral imagination with your current brain, and conclude: 'Even if I thought killing were right, killing would still be wrong.' But this doesn't show that killing-is-wrong is a fixed fact from outside you."
Obert: "Like, say, 2 + 3 = 5 is a fixed fact. Eliezer wrote: 'If something appears to be the same regardless of what anyone thinks, then maybe that's because it actually is the same regardless of what anyone thinks.' I'd say that subtext is pretty clear!"
Subhan: "On the contrary. Naively, you might imagine your future self thinking differently of a thing, and visualize that the thing wouldn't thereby change, and conclude that the thing existed outside you. Eliezer shows how this is not necessarily the case. So you shouldn't trust your intuition that the thing is objective—it might be that the thing exists outside you, or it might not. It has to be argued separately from the feeling of subjunctive objectivity. In the case of 2 + 3 = 5, it's at least reasonable to wonder if math existed before humans. Physics itself seems to be made of math, and if we don't tell a story where physics was around before humans could observe it, it's hard to give a coherent account of how we got here. But there's not the slightest evidence that morality was at work in the universe before humans got here. We created it."
Obert: "I know some very wise children who would disagree with you."
Subhan: "Then they're wrong! If children learned in school that it was okay to steal, they would grow up believing it was okay to steal."
Obert: "Not if they saw that stealing hurt the other person, and felt empathy for their pain. Empathy is a human universal."
Subhan: "So we take a step back and say that evolution created the emotions that gave rise to morality, it doesn't put morality anywhere outside us. But what you say might not even be true—if theft weren't considered a crime, the other child might not feel so hurt by it. And regardless, it is rare to find any child capable of fully reconsidering the moral teachings of its society."
Obert: "I hear that, in a remarkable similarity to Eliezer, your parents were Orthodox Jewish and you broke with religion as a very young child."
Subhan: "I doubt that I was internally generating de novo moral philosophy. I was probably just wielding, against Judaism, the morality of the science fiction that actually socialized me."
Obert: "Perhaps you underestimate yourself. How much science fiction had you read at the age of five, when you realized it was dumb to recite Hebrew prayers you couldn't understand? Children may see errors that adults are too adept at fooling themselves to realize."
Subhan: "Hah! In all probability, if the teacher had in fact said that it was okay to take things from other children's backpacks, the children would in fact have thought it was right to steal."
Obert: "Even if true, that doesn't prove anything. It is quite coherent to simultaneously hold that:"
Subhan: "Fine, it's coherent, but that doesn't mean it's true. The morality that the child has in fact learned from the teacher—or their parents, or the other children, or the television, or their parents' science fiction collection—doesn't say, 'Don't steal because the teacher says so.' The learned morality just says, 'Don't steal.' The cognitive procedure by which the children were taught to judge, does not have an internal dependency on what the children believe the teacher believes. That's why, in their moral imagination, it feels objective. But where did they acquire that morality in the first place? From the teacher!"
Obert: "So? I don't understand—you're saying that because they learned about morality from the teacher, they should think that morality has to be about the teacher? That they should think the teacher has the power to make it right to steal? How does that follow? It is quite coherent to simultaneously hold that—"
Subhan: "I'm saying that they got the morality from the teacher! Not from some mysterious light in the sky!"
Obert: "Look, I too read science fiction and fantasy as a child, and I think I may have been to some degree socialized by it—"
Subhan: "What a remarkable coincidence."
Obert: "The stories taught me that it was right to care about people who were different from me—aliens with strange shapes, aliens made of something other than carbon atoms, AIs who had been created rather than evolved, even things that didn't think like a human. But none of the stories ever said, 'You should care about people of different shapes and substrates because science fiction told you to do it, and what science fiction says, goes.' I wouldn't have bought that."
Subhan: "Are you sure you wouldn't have? That's how religion works."
Obert: "Didn't work on you. Anyway, the novels said to care about the aliens because they had inner lives and joys—or because I wouldn't want aliens to mistreat humans—or because shape and substrate never had anything to do with what makes a person a person. And you know, that still seems to me like a good justification."
Subhan: "Of course; you were told it was a good justification—maybe not directly, but the author showed other characters responding to the argument."
Obert: "It's not like the science fiction writers were making up their morality from scratch. They were working at the end of a chain of moral arguments and debates that stretches back to the Greeks, probably to before writing, maybe to before the dawn of modern humanity. You can learn morality, not just get pressed into it like a Jello mold. If you learn 2 + 3 = 5 from a teacher, it doesn't mean the teacher has the power to add two sheep to three sheep and get six sheep. If you would have spouted back '2 + 3 = 6' if the teacher said so, that doesn't change the sheep, it just means that you don't really understand the subject. So too with morality."
Subhan: "Okay, let me try a different tack. You, I take it, agree with both of these statements:"
Obert: "Well, there are various caveats I'd attach to both of those. Like, in any circumstance where I really did prefer to kill someone, there'd be a high probability he was about to shoot me, or something. And there's all kinds of ways that eating an anchovy pizza could be wrong, like if I was already overweight. And I don't claim to be certain of anything when it comes to morality. But on the whole, and omitting all objections and knock-on effects, I agree."
Subhan: "It's that second statement I'm really interested in. How does your wanting to eat an anchovy pizza make it right?"
Obert: "Because ceteris paribus, in the course of ordinary life as we know it, and barring unspecified side effects, it is good for sentient beings to get what they want."
Subhan: "And why doesn't that apply to the bit about killing, then?"
Obert: "Because the other person doesn't want to die. Look, the whole reason why it's right in the first place for me to eat pepperoni pizza—the original justification—is that I enjoy doing so. Eating pepperoni pizza makes me happy, which is ceteris paribus a good thing. And eating anchovy pizza—blegh! Ceteris paribus, it's not good for sentient beings to experience disgusting tastes. But if my taste in pizza changes, that changes the consequneces of eating, which changes the moral justification, and so the moral judgment changes as well. But the reasons for not killing are in terms of the other person having an inner life that gets snuffed out—a fact that doesn't change depending on my own state of mind."
Subhan: "Oh? I was guessing that the difference had something to do with the social disapproval that would be leveled at murder, but not at eating anchovy pizza."
Obert: "As usual, your awkward attempts at rationalism have put you out of touch with self-evident moral truths. That's just not how I, or other real people, actually think! If I want to bleep bleep bleep a consenting adult, it doesn't matter whether society approves. Society can go bleep bleep bleep bleep bleep -"
Subhan: "Or so science fiction taught you."
Obert: "Spider Robinson's science fiction, to be precise. 'Whatever turns you on' shall be the whole of the law. So long as the 'you' is plural."
Subhan: "So that's where you got that particular self-evident moral truth. Was it also Spider Robinson who told you that it was self-evident?"
Obert: "No, I thought about that for a while, and then decided myself."
Subhan: "You seem to be paying remarkably close attention to what people want. Yet you insist that what validates this attention, is some external standard that makes the satisfaction of desires, good. Can't you just admit that, by empathy and vicarious experience and evolved fellow-feeling, you want others to get what they want? When does this external standard ever say that it's good for something to happen that someone doesn't want?"
Obert: "Every time you've got to tell your child to lay off the ice cream, he'll grow more fat cells that will make it impossible for him to lose weight as an adult."
Subhan: "And could something good happen that no one wanted?"
Obert: "I rather expect so. I don't think we're all entirely past our childhoods. In some ways the human species itself strikes me as being a sort of toddler in the 'No!' stage."
Subhan: "Look, there's a perfectly normal and non-mysterious chain of causality that describes where morality comes from, and it's not from outside humans. If you'd been told that killing was right, or if you'd evolved to enjoy killing—much more than we already do, I mean—or if you really did have a mini-stroke that damaged your frontal lobe, then you'd be going around saying, 'Killing is right regardless of what anyone thinks of it'. No great light in the sky would correct you. There is nothing else to the story."
Obert: "Really, I think that in this whole debate between us, there is surprisingly litle information to be gained by such observations as 'You only say that because your brain makes you say it.' If a neutrino storm hit me, I might say '2 + 3 = 6', but that wouldn't change arithmetic. It would just make my brain compute something other than arithmetic. And these various misfortunes that you've described, wouldn't change the crime of murder. They would just make my brain compute something other than morality."
Part of The Metaethics Sequence
Next post: "Changing Your Metaethics"
Previous post: "Math is Subjunctively Objective"