"it" isn't a single theory.

The argument that Everettian MW is favoured by Solomonoff induction, is flawed.

If the program running the SWE outputs information about all worlds on a single output tape, they are going to have to be concatenated or interleaved somehow. Which means that to make use of the information, you gave to identify the subset of bits relating to your world. That's extra complexity which isn't accounted for because it's being done by hand, as it were..

Good points. I think the term moral realism is probably used in a variety of ways in the public sphere. I think the relevant sense is “will alignment solve itself because a smart machine will decide to behave in a way we like”. If there’s some vague sense of stuff everyone “should” do, but it doesn’t make them actually do it, then it doesn’t matter for this purpose.

I think “the good is what you should do” is remarkably devoid of useful meaning. People often mean very little by “should”, are unclear both to others and themselves, and use it in different ways in different situations.

For understanding human ethics, the important thing is that it grounds out in punishments and rewards -- the good is what you should do , and if you don't do it, you face punishment. Another thing that means is that a theory of ethics must be sufficient to justify putting people in jail. But a definition is not a theory.

My theory is that “good” is usually defined as an emotion, not another set of words, and that emotion roughly means “I want that person on my team” (when applied to behavior),

If your whole theory of ethics is to rubber stamp emotions or opinions, you end up with a very superficial theory that is open to objections like the Open Question argument. Just because somebody feels it is good to do X does not mean it was necessarily is --it is an open question. If the good is your emotions , then it is a closed question...your emotions are your emotions , likewise your values are your values, and your opinions are your opinions. The openness of the question "you feel that X is good, but is it really?" is a *theoretical" reason for believing that "goodness" works more like "truth" and less like "belief".

(And the OQA is quite likely what this passage by Nostalgebraist hints at:-

*Who shoots down the enemy soldiers while thinking, “if I had been born there, it would have been all-important for their side to win, and so I would have shot at the men on this side. However, I was born in my country, not theirs, and so it is all-important that my country should win, and that theirs should lose.

There is no reason for this. It could have been the other way around, and everything would be left exactly the same, except for the ‘values.’

I cannot argue with the enemy, for there is no argument in my favor. I can only shoot them down.)

because evolution engineered us to find useful teammates, and that feeling is its mechanism for

And having gathered our team to fight the other team, we can ask ourselves whether we might actually be the baddies.

The *practical* objection kicks in when there are conflicts between subjective views.

A theory of ethics needs to justify real world actions -- especially actions that impact other people , especially actions that impact other people negatively.( It's not just about passively understanding the world, about 'what anticipated experiences come about from the belief that something is “good” or “bad”?')Why should someone really go to jail ,if they havent really done anything wrong? Well, if the good is what you should do, jailing people is justifiable , because the kind of ting you shouldn't do is the kind of thing you deserve punishment for.

Of course, the open question argument doesn't take you all the way to full strength moral realism. Less obviously, there are many alternatives to MR. Nihilism is one: you can't argue that emotivism is true because MR is false -- emotivism might be wrong because ethics is nothing. Emotivism might also be wrong because some position weaker than MR is right.

If good means “what you should do” then it’s exactly the big claim Steve is arguing against.

If Steve is saying that the moral facts need to be intrinsically motivating, that is a stronger claim than "the good is what you should do", ie, it is the claim that "the good is what you would do". But, as cubefox points out, being intrinsically motivating isn't part of moral realism as defined in the mainstream. (it is apparently part of moral realism as defined in LW, because of something EY said years ago). Also, since moral realism is metaethical claim, there is no need to specify the good at object level.

I’d be happy to come back later and give my guesses at what people tend to mean by “good”; it’s something like “stuff people do whom I want on my team” or “actions that make me feel positively toward someone”.

Once again, theories aren't definitions.

People don't all have to have the same moral theory. At the same time, there has to be a common semantic basis for disagreement, rather than talking past, to take place. "The good is what you should do" is pretty reasonable as a shared definition, since it is hard to dispute, but also neutral between "the good" being define personally, tribally, or universally.

Faith in maths prodigies can be misplaced. Faith in maths can be misplaced. No one has ever proved that you can solve everything with maths. The people who believe it believe it because a guru figure said so.

Positivism isn't necessarily true, and if it is, it still doesn't get you to 6, because LP recommends you have no metaphysics which would imply no solipsistic metaphysics. (LP might be compatible with the claim that your own sense-data are all you can know , but that isn't quite the same thing).

There's a soft patch around 5 and 6. Why is testability important? It's a charactersitic of science, but science assumes an external world. It's not a characteristic of philosophy -- good explanation is enough in philosophy, and the general posit of some sort of external world does explanatory work. And it's separate from the specific posit that the external world is knowable in some particular way.

That's just ordinary compatibilism -- as I said, "it’s not libertarian free will." All the work is being done by using a definition of free will that doesn't require indeterministic "elbow room", so none of it is being done by all the physics and metaphysics. If it is valid, it would be just as valid under naturalistic monism, supernaturalistic determinism, etc.

And compatibilism isn't universally accepted as the solution to free will because the quale of freedom is libertarian -- one feels that one could have done otherwise. (At least , mine is like that).

An additional non physical layer of consciousness might buy you qualia, but delivers no guarantee that they will be accurate... a quale of libertarian free will is necessarily illusory under determinism.

An additional non physical layer of consciousness might have bought you downwards causation and libertarian free will.

But you are not legitimising it as a subjective impression that correctly represents reality... only as an illusion: you can feel free in a deterministic world, but you can't be free in one.

Under physicalist epiphenomenalism (which is the standard approach to the mind-matter relation), the mind is super-impressed on reality, perfectly synchronized, and parallel to it.

Under dualist epiphenomenalism, that might be true. Physicalism has it either that consciousness is non existent rather than causally idle (eliminitavism), or identical to physical brain states (and therefore sharing their causal powers).

Understanding why some physical systems make an emergent consciousness appear (the so called “hard problem of consciousness”) or finding a procedure that quantify the intensity of consciousness emerging from a physical system (the so called “pretty hard” problem of consciousness) is impossible:

You could have given a reason why.