The problem here is that the second option you offer does nothing to explain what a compelling argument is; it just passes the recursive buck onto the word "committed". I know you said you recognize that, but unless we can show that this line of reasoning is coherent (let alone leads to a relevant conclusion, let alone correct) then there's no reason to assume that Eliezer's point isn't trivial in the end. Philosophers have believed a lot of silly things, after all. The only sensible resolution I can come up with is where you take "committed to x" to mean "would, on reflection and given sufficient (accurate) information and a great deal more intelligence, believe x". The problem is that this is still trivially false in the entirety of mindspace. You might, although I doubt it, be able to establish a statement of that form over all humans (I think Eliezer disagrees with me on the likelihood here). You could certainly not establish one given a mindspace that includes both humans and paper clip maximizers.
I know you said you recognize that, but unless we can show that this line of reasoning is coherent (let alone leads to a relevant conclusion, let alone correct) then there's no reason to assume that Eliezer's point isn't trivial in the end.
If what you're saying is this, then we agree: EY doesn't here present an argument that UCMAs are likely to be false, but he does successfully argue that a certain class of generalizations over mind-space are likely to be false (such as generalizations about what minds will find persuasive) along with the assumption th... (read more)
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.