How to formalize predictors

by cousin_it1 min read28th Jun 20184 comments



In Newcomb's Problem, what information does the predictor get if you flip a coin? Here's some options:

1) "Bob has 50:50 odds of one-boxing or two-boxing"

2) "Bob will one-box" (chosen randomly with the same odds as your decision)

3) "Bob will one-box" (guaranteed to be the same as your decision)

I think (1) is a poor formalization, because the game tree becomes unreasonably huge, and some strategies of the predictor (like "fill the box unless the probability of two-boxing is exactly 1") leave no optimal strategy for the player.

And (3) seems like a poor formalization because it makes the predictor work too hard. Now it must predict all possible sources of randomness you might use, not just your internal decision-making.

That leaves us with (2). Basically we allow the predictor to "sample" your decision at any information set in the game. That means we can add some extra nodes to the player's information sets and get rid of the predictor entirely, ending up with a single player game. Newcomb's Problem and Counterfactual Mugging can be easily analyzed in this way, leading to the same answers we had before (one-box, pay up). It also gives a crisp formalization of the transparent boxes Newcomb problem, where we sample the player's decision at the information set of "seeing both boxes filled".

I think this might end up useful for bounded versions of Counterfactual Mugging, which are confusing to everyone right now. But also it feels good to nail down my understanding of the simple version.