reavowed

Hi! My name's Iain - I'm a mathematician by education and a software developer by vocation, but I recently quit my long-time job in order to try and think more about problems that actually matter to the future of humanity. My long term goals include curing death and immanentizing the eschaton. I welcome direct messages and offline discussion of posts and comments I make on this site, though I am terrible at replying to things in a timely fashion.

10

I think my core issue with the above is the nature of the specification of the problem of "replacing in ". Allowing passing to an arbitrary equivalent program before replacing exact instances of seems overly permissive, and to allow in exactly the kind of principle-of-explosion issue that logical counterfactuals have. Suppose for instance that and both halt with a defined value, say . As framed above, I can take to be a program that computes (for some ), where is the result of substituting all exact instances of in with any program returning that is not identically . Then is (provably) equivalent to , but if I can ensure that the substituted version of returns any value I like just by varying . This doesn't say anything interesting about the appearance of in - in fact, the replacement of with is exactly to ensure that the main substitution is a no-op.

I feel like a statement that more accurately corresponds to the substitution I'm trying to do above would be something like "There's a program C that can be obtained from B by replacing subprograms that are provably equivalent to A with the constant program returning 1, such that C() = 2". For one thing, this appears to have the useful property that if makes no mention of or anything equivalent to it, then I can't stop such a from being equivalent (in fact, identical) to no matter what the value of is. Does this still have the same problem?

30

I agree that that is a problem that both this approach to counterfactuals and the FDT logical-counterfactual approach share. The particular problem I was hoping this approach avoids was the one of assuming mutually-exclusive logical facts, such that all-but-one of them must necessarily be false, and the implications this has for the agent's consistency and reasoning about its actions. Are you saying that they are the same problem, that the second problem is comparatively not worth solving, or something else?

10

I have indeed read many of those posts already (though I appreciate some reference to them in the original post would have been sensible, I apologise). Chris_Leong's Deconfusing Logical Counterfactuals comes pretty close to this - the counterfactual model I'm interested in corresponds to their notion of "Raw Counterfactual", but AFAICT they're going in a somewhat different direction with the notion of "erasure" (I don't think it should be necessary to forget that you've seen a full box in the transparent variant of Newcomb's problem, if you explicitly consider that you be in Omega's simulation), so they haven't followed the notion through to a full description of what a raw-counterfactual-based decision theory would look like. I can't find any further discussion of the idea, and Chris_Leong now seems to be walking back on it (that's part of the reason I'm asking the question).

50

Ah, got there. From , we get specifically and thus . But we have directly as a theorem (axiom?) about the behaviour of , and we can lift this to , so also and thus .

70

I'm having difficulty following the line of the proof beginning "so, either way, PA is inconsistent". We have and , which together imply that , but I'm not immediately seeing how this leads to ?

50

In fact, all you know is that your credence of event H is somewhere in the interval [0.4, 0.6]

This really isn't how I understand credences to work. Firstly, they don't take ranges, and secondly, they aren't dictated to me by the background information, they're calculated from it. This isn't immediately fatal, because you can say something like:

The coin was flipped one quintillion times, and the proportion of times it came up heads was A, where A lies in the range [0.4, 0.6]

This is something you could actually tell me, and would have the effect that I think is intended. Under this background information X, my credence P(H | X) is just 0.5, but I have that P(H | X, A=a) = a for any a in [0.4, 0.6].

This is more than just a nitpick. We've demoted the range [0.4, 0.6] from being *a priori* privileged as the credence, to just another unknown value in the background information. When you then say "I'm maximising minimum expected utility", the obvious objection is then - why have you chosen to minimise only over A, rather than any of the other unknown values in the background information? In particular, why aren't you minimising over the value C, which represents the side the coin lands on?

But of course, if you minimise over **all** the unknowns, it's a lot less interesting as a decision framework, because as far as I can tell it reduces to "never accept any risk of a loss, no matter how small the risk or the loss".

Again, I agree that the problem of identifying what logical structures (whereever they occur) count as implementing a particular function is a deep and interesting one, and not one that I am claiming to have solved. But again, I do not agree that it is a problem I have introduced? An FDT agent correctly inferring the downstream causal results of setting FDT(P––,G––)=a would, in general, have to identify FDT(P––,G––) being computed inside a Game of Life simulation, if and where such a calculation so occured.

While I am indeed interested in exploring the answers to your questions, I don't see that they represent a specific challenge to the idea that the above notion of counterfactuals might be worth exploring further, in the way that your original claim would.