There's actually no need to settle for finite truncations of a decision agent. The unlosing decision function (on lotteries) can be defined in first-order logic, and your proof that there are finite approximations of a decision function is sufficient to use the compactness theorem to produce a full model.
[This comment is no longer endorsed by its author]Reply
I've just made an enrollment deposit at the University of Illinois at Urbana-Champaign, and I'm wondering if any other rationalists are going, and if so, would they be interested in sharing a dorm?
Perhaps instead of immediately giving up and concluding that it's impossible to reason correctly with MWI, it would be better to take the born rule at face value as a predictor of subjective probability.
The AI is a program. Running on a processor. With an instruction set. Reading the instructions from memory. These instructions are its programming. There is no room for acausal magic here. When the goals get modified, they are done so by a computer, running code.
If p + q = 1, then p(A or B) = 1. The equivalence statement about A and B that we're updating can be stated as (A or B) iff (A and B). Since probability mass is conserved, it has to go somewhere, and everything but A and B have probability 0, it has to go to the only remaining proposition, which is g(p, q), resulting in g(p, q) = 1. Stating this as p+q was an attempt to find something from which to further generalize.
[This comment is no longer endorsed by its author]Reply
My first reaction to the second question is to consider the case in which p + q = 1. Then, the answer is clearly that g(p, q) = p + q. I suspect that this is incomplete, and that further relevant information needs to be specified for the answer to be well-defined.
[This comment is no longer endorsed by its author]Reply
static double averageUtility(double x) {
int sum = 0;
for(int i = 0; i < trials; i++) {
sum += utility(x); //iteratively generates the sum to ... (read more)
[This comment is no longer endorsed by its author]Reply
There's actually no need to settle for finite truncations of a decision agent. The unlosing decision function (on lotteries) can be defined in first-order logic, and your proof that there are finite approximations of a decision function is sufficient to use the compactness theorem to produce a full model.