"propounding neutrality is just as attackable as propounding any particular side."

Indeed. (I hope Robin is reading.)

If we sufficiently value episodes of aesthetic appreciation (in general, not only when done by us), etc., then the "compromise" could be a net positive, even from the perspective of our current values.

(But perhaps the point is that our values are in fact not so agent-neutral.)

Seconding Peter -- the post should say "one boxing", right?

Eliezer - 'I would be willing to get a PhD thesis if it went by the old rules and the old meaning of "Prove you can make an original, significant contribution to human knowledge and that you've mastered an existing field", rather than, "This credential shows you have spent X number of years in a building."'

British and Australasian universities don't require any coursework for their PhDs, just the thesis. If you think your work is good enough, write to Alan Hajek at ANU and see if he'd be willing to give it a look.

Hmm, reminds me of a post I wrote two years earlier.

Incidentally, I think one of Bond's "real life examples" exposes an important ambiguity:

A: "I can even handle misplaced apostrophes every now and then. Not excessive amounts of them, [...]" B: "Perhaps double-check your grammar before you write a grammar rant that refers to 'amounts of apostrophes'." C: " ...the ad hominem nature of [B's reply] takes the sanctimonious angle that any who criticize must be without stain."

Bond writes, "B's reply was not ad hominem. It was not a counter-argument to A, but an attempt to point out what B saw as A's hypocrisy."

But actually it is ad hominem, i.e. directed 'to the person', though not (of course) an ad hominem fallacy in the usual sense. See: Ad hominem tu quoque.

Aristotle, anyone?

"asking how is it that the word 'right' came to refer to rightness is like asking why 'green' means green"

Yeah, that's not exactly what I meant. As I see it there are two stages: there's the question how the symbols 'right' (or 'green') get attached to the concept that they do, and then there's the more interesting question of how this broad sense of the term determines -- in combination with the actual facts -- what the term actually refers to, i.e. what property the concept denotes. So I should have asked how it is that our sense of the concept 'right', as we hold it in our minds, determines what external property is ultimately denoted by the term. (Compare how the concept 'water' ultimately denotes the property of being H2O.) It is this step of Eliezer's account, I think, which looks to some to be suspiciously relativistic, and in conflict with the sense of the term as they understand it. Maybe he's picking out the right property (hard to tell when he's said so little about it, as you say). But the meta-properties, the concept, the procedure by which what we have in mind picks out a particular thing in the word, that just seems all wrong.

Larry, no, the mix-up is yours. I didn't say anything about morality, I was talking about the word 'right', and the meta-semantic question how it is that this word refers to rightness (some particular combination of terminal values) rather than, say, p-rightness.

Some of these (e.g. Roko's) concerns might be clarified in terms of the distinctions between sense, reference, and reference-fixing descriptions. I take it Eliezer wants to use 'right' as a rigid designator to denote some particular set of terminal values, but others have pointed out that this reference fact is fixed by means of a seemingly 'relative' procedure (namely, whatever terminal values he himself happens to hold, on some appropriate [if somewhat mysterious] idealization). There is also some concern that this doesn't match the plain meaning or sense of the term 'right', as everyone else understands it.

"I can only plead that when I look over my flawed mind and see a core of useful reasoning, that I am really right, even though a completely broken mind might mistakenly perceive a core of useful truth."

"humans have received a moral gift, which Pebblesorters lack, in that we started out interested in things like happiness instead of just prime pebble heaps. Now this is not actually a case of someone reaching in from outside with a gift-wrapped box... it is only when you look out from within the perspective of morality, that it seems like a great blessing that there are humans around"

Quick question: do you intend the latter deflationary remarks to apply to your 'epistemic gift' too? That is, would you emphasize that your methods of reasoning are merely considered to be a gift from within your own perspective, and there's not any further sense to the notion of 'good priors' or a 'broken mind' or 'useful reasoning' beyond the brute fact that you happen to use these words to refer to these particular epistemic norms? Or do you think there's an important difference between the kind of (moral vs. epistemic) 'mistakes' made respectively by the Pebblesorters and the anti-Inductors?