"It's a whole lot easier to modify the source code in the interior of your own mind, then to take that modification, and sell it to a friend who happens to be written on different source code"

If I understand the sentence correctly, "then" should be "than".

"to make predictions with probability 1 (which, you must concede, is a plausible reading of "no danger whatsoever")"

I don't know what Eliezer will concede, but in fact it is more than a plausible reading. It's what the phrase means. Actually, "no danger" means "probability 1 of no conversion", and "whatsoever" means "for real, folks, I mean probability 1 exactly". And I don't see anything religious about it. Hyperbolic perhaps, but not religious.

Nice post, Eliezer.

(By the way, B's argument is wrong because A's argument is not really ad hominem... but if it were, he would be right when he says it should be rejected. That's what we should do with fallacious arguments in general: reject them)

I liked this made-up example:

A: "All rodents are mammals, but a weasel isn't a rodent, so it can't be a mammal." B: "Well, you're a rodent and a weasel, so there goes your argument."

B's reply is ad hominem, but it's also funny.

I didn't like this one:

A: "Listen up, asshole. All rodents are mammals, but a weasel isn't a rodent, so it can't be a mammal." B: "Yet another ad hominem argument. Ignore this one, folks." A is abusive, and his argument is fallacious, but it's not ad hominem. B's reply, ironically, is ad hominem...

Well, it's not that I dind't like the example itself. It's just that, given the lack of context, I don't see how B's reply is ad hominem. B's reply does not say "Yet another ad hominem argument by that notorious ad hominem arguer A", which would be ad hominem. It says "Yet another ad hominem argument", period. If this exchange took place in a discussion in which B was trying to portray A as a fallacious arguer, it would be ad hominem, but we can't reach this conclusion with the information we are given.

The author misuses the term ad hominem, so just reject all his arguments!

I get what you're saying, Thomas, you're just answering another question. If some living thing were powerful enough to use a whole planet the size of Jupiter to its advantage, I agree with you that that would be the living thing that remains, rather than others. And then, if some living thing were even more powerful and could turn all the stars in the Universe into something useful, then that living thing would be the winner. But notice the "ifs". There's no if in "But Robin and I both suspect, I think, that the fate of the universe, and all those distant stars that outweigh us, will end up shaped by Life".

I still don't get how Eliezer and (according to Eliezer) Robin think that life (or Life, whatever, a capital letter won't do the trick) is going to shape the stars and (if I understand correctly) the whole Universe. For all we know, despite all the replication and optimization, living things don't seem to have done much of that. Living things have shaped (to some extent) the surface of one planet. Big deal. If we changed the orbit of that same planet, I would think it impressive, compared to what we've done in the past. But still, big deal. It would only be one planet. Shape the Universe? How? And, if you don't know how, why do you think life will do it?

(Let me guess, intelligence can do anything. All right. Intelligence is God. But, why?)

"But Robin and I both suspect, I think, that the fate of the universe, and all those distant stars that outweigh us, will end up shaped by Life"

And how do you suspect that is going to happen?

IgnoranceNeverPays: "This is a common thing among people who know enough that they think they know something but don't actually know enough to really know something."

Can you say that really really fast?