Oh my, the confusion.
First off, the quoted argument was, as far as I can tell, entirely meant as an illustrative abstraction. The culprit here is the devious function X().
Suppose I take the set of all possible logically coherent statements that could be made about any given mind. Within this set, 'X' is any given statement about one mind. X(m) represents whether this given statement is True, False or Undefined / Undecidable for this mind 'm'.
For all X1..Xn, for a given mind 'm1', find all the X that are true. Then for all X() for m2, find those that are true. Supposing any given X has 50% probability of being true of any given m, then X1 being true for m1 has probability 0.5, being true for both m1 and m2 has probability 0.25, and so on dividing the odds by 2 for each additional mind for which the conjunctive must hold true.
So for any given X, for m1..m(10^12), X has (1 / 2^12) probability of being true if we assume a priori 50% chance of that statement being true.
Conversely, for any given X, X(m) will be true for at least one m with 2^12 / (2^12 + 1) probability.
The central inference in the argument is that we do not know the structure of 'all possible minds' or of 'all possible arguments', but it is reasonable to believe that the space of all possible minds is sufficiently large and versatile that the subset A() of all possible statements X(), where A() are statements of the form "A(m) is true if mind 'm', when presented with the argument A, will change some specific belief / internal state / thought / behavior to state Y", is not true for all A(m). This latter part of the argument rests mostly on the following reasoning:
If there is any given argument A that will convince all currently known minds such that A(m) is true, and all known minds accept the argument, we can almost certainly construct a mind nearly identical to one of these m, but for which the input A is forbidden, or that will self-destruct immediately upon receiving it, or where the first and only possible result of A(m) is always false, or where arguments of the form which A takes are simply incoherent.
For example, if A is a UCMA of the form where you speak or write certain words and have the listener hear or read them, make the listener unable to understand the language. If the UCMA implies translation to a given language of choice that the listener understand, craft the listener so that the listener does not understand or have any languages. If the UCMA is some form of complex hacking by specifying complicated inputs that abuse internal properties of the mind in question... well, how likely is it that all possible minds share these exact internal properties?
This is a complex subject since "minds" and "arguments" are loaded terms with lots of anthropomorphic stuff hidden behind them, and much of the actual meat of the problem lies in things philosophers still disagree on despite overwhelming probability.
Excellently explained, thank you. The argument you present seems to me to be on the whole reasonable, but it involves two assumptions no UCMA enthusiast I know of would ever accept.
Supposing any given X has 50% probability of being true of any given m...
And
"A(m) is true if mind 'm', when presented with the argument A, will change some specific belief / internal state / thought / behavior to state Y"
These two assumptions aren't argued for, nor are they attributed to any UCMA enthusiast, so I cant see any reason why she should accept them. Do they seem plausible to you? If so, can you give me reasons to accept them?
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.