You're relying on the fact that you have uncertainty about Omega's prediction, which is really an accidental feature of the problem, not shared by other problems in the same vein.
Imagine a variant where both boxes are transparent and you can see what's inside, but the contents of the boxes were still determined by Omega's prediction of your future decision. (I think this formulation is due to Gary Drescher.) I'm a one-boxer in that variant too, how about you? Also see Parfit's Hitchhiker, where the predictor's decision depends on what you would do if you already knew the predictor decided in your favor, and Counterfactual Mugging, where you already know that your decision cannot help the current version of you (but you'd precommit to it nonetheless).
The most general solution to such problems that we currently know is Wei Dai's UDT. Informally it goes something like this: "choose your action so that the fact of your choosing it in your current situation logically implies the highest expected utility (weighted over all apriori possible worlds before you learned your current situation) compared to all other actions you could take in your current situation".
Extremely counterfactual mugging is the simplest such variation IMO. Though it has the same structure as Parfit's Hitchhiker, it's better because issues of trust and keeping promises don't come into it. Here it is:
Omega will either award you $1000 or ask you to pay him $100. He will award you $1000 if he predicts you would pay him if he asked. He will ask you to pay him $100 if he predicts you wouldn't pay him if he asked.
Omega asks you to pay him $100. Do you pay?
Imagine a variant where both boxes are transparent and you can see what's inside
That seems like a weird situation. Given two boxes, both of them with money, I'd take both. Given instead that one box is empty, I'd just take the one with money. So I'd default to doing whatever Omega didn't predict, barring me being told in advance about the situation and precommitting to one-boxing.
Sorry for not making things clear from the start. In Gary's version of the transparent boxes problem, Omega doesn't predict what you will do, it predicts what you would do if both boxes contained money. Your actions in the other case are irrelevant to Omega. Would you like to change your decision now?
So, basically, I know that if I take both boxes, and both boxes have money, I'm either in a simulation or Omega was wrong? In that case, precommiting to one-boxing seems sensible.
Drescher then goes on to consider the case where you know that Omega has a fixed 99% chance of implementing this algorithm, and a 1% chance of instead implementing the opposite of this algorithm, and argues that you should still one-box in that case if you see the million.
"choose your action so that the fact of your choosing it in your current situation logically implies the highest expected utility (weighted over all apriori possible worlds before you learned your current situation) compared to all other actions you could take in your current situation".
That sounds awkward. Would you say it's equivalent to "choose globally winning strategies, not just locally winning actions?"
As far as I know, you understand UDT and can answer that question yourself :-) But to me your formulation sounds a little vague. If a newbie tries to use it to solve Counterfactual Mugging, I think he/she may get confused about the intended meaning of "global".
I still don't know if I understand UDT :D
And yeah, "globally winning" probably should have been replaced with "optimal," since the "local" means something specific about payoff matrices and I don't want to imply the corresponding "global."
Imagine a variant where both boxes are transparent and you can see what's inside, but the contents of the boxes were still determined by Omega's prediction of your future decision. (I think this formulation is due to Gary Drescher.) I'm a one-boxer in that variant too, how about you?
Of course you are, Omega says so! Short of being infinitely confident in Omega's abilities (and my understanding of them), I'd reach for both boxes. Are you predicting I will see the million will dissolve into thin air?
Are you trying to pre-commit in order to encourage potential Omegas to hand over the money? Is there more extensive discussion of this variant?
I'd reach for both boxes. Are you predicting I will see the million will dissolve into thin air?
No. Based on your comment, I'm predicting you won't see the million in the first place.
Yes, there is a more extensive discussion, in Good and Real. Basically, you act for the sake of what would be the case if you had acted that way.
In this case, there is actually a plausible causal reason to one-box: you could be the instance that Omega is simulating in order to make its prediction. But even if not, there are all sorts of cases where we act for the sake of what would be the case if we did. Good and Real discusses this extensively.
Related to: Newcomb's Problem and Regret of Rationality and Newcomb's Problem: A Problem for Casual Decision Theories.
From the list of standard problems in rationality that have been talked to death but still don't have a strong consensus, allow me to re-present Newcomb's Problem:
This problem is famous for not only the fact that the answer is extensively controversial, but also for people who think they should one-box (take only box B) or two-box (take both box A and box B) almost always are very certain of their answer. I'm one of those people -- I'm very certain I should one-box.
Since I missed out on earlier Newcomb's discussion, I'd like to explore my approach to the problem here.
In Newcomb's Problem, there are only four possible outcomes, and they end up in this tree:
Omega predicts you will one-box:
Omega predicts you will two-box:
We can use this knowledge to create an expected value formula for all four options (where P is the chance Omega guessed your choice correctly):
EV(One-box) = P*$1000000 + (1-P)*$0
EV(Two-box) = P*$1000 + (1-P)*$1001000
If we set the two equations equal to each other, we can solve for the value for P that would make both strategies equally viable, expected value wise. That value is 50.05%. So as long as we are confident that Omega has a better than 50.05% chance at predicting our choice, the expected value formula says we can expect to earn more from one-boxing.
But what of the dominant strategy approach? Clearly if Omega predicted one-box, you earn more by two-boxing and if Omega predicted two-box, you also earn more by two-boxing. Does this not count as a viable approach, and a worthy defense of two-boxing?
However, following this logic is specifically betting that it is likely Omega will predict incorrectly ...and if P < 50%, this is a losing bet.