Extra credit
If you make your decision about whether to open the second box contingent on some fact about the future, then the contents of the first box let you predict that fact in advance. e.g., If the New York Yankees win their baseball game today then I will open the second box after their game, and if they do not win then I will not open the second box. Then if I open the first box and see $100, I can bet a lot of money on the Yankees to win, and if I see $0 then I can bet against the Yankees winning. If Omega is less patient then you'll need to pick something that resolves quicker.
Somehow neither spoiler is working...
! :::spoiler Doesn't that run into the same issue as Harry in HPMoR with his experiment with time? Namely, that there are a lot of scenarios in which you never (get to) open the second box, easiest case: you died. But also probably any number of other results (it gets stolen, f.e.) :::
The spoiler tag only works for me if I type it, not if I copy-paste it.
In this scenario, Omega is described as predicting your actions and choosing how much money goes in the boxes, which is different from HPMOR where timelines settle into a self-consistent equilibrium in more complicated ways. And with this plan it should be straightforward for Omega to carry out its usual process, since you're not trying to generate a contradiction or force events through a weird narrow path, you're just entangling your box-opening decision with an observation about the world which you haven't made yet, but which omniscient Omega should know. For Omega, this is no different than you deciding that your second-box-opening decision will be contingent on the results of yesterday's Yankees game, which you haven't heard yet but which you will look up on your phone after you open the first box.
Determining a property of a computation doesn't obviously make it more real, and in no way does it imply that the computation is being simulated in full (proofs of properties of computations often don't run those computations). When you are thinking detailed thoughts, that doesn't mean that the analysis of your eventual decision is simulating those thoughts, either at all or in a morally relevant way.
Worse, the you that is thinking the detailed thoughts might be purely hypothetical, not even being simulated in any physical reality at all, you have no way of telling if it's so from the inside of the hypothetical. And platonically you exist in either case, stopping simulating you has no impact on that, the platonic you has no way of noticing if the simulation stopped, you just continue thinking the thoughts within that platonic existence with no supporting simulation.
The root of this paradox is that the human notion of ‘self’ actually can refer to at least two things: the observer and the agent. FDT and similar updateless decision theories only really concern themselves with the second.
An FDT agent cannot really “die” as they do exist in all places of the universe at once. Whereas, as you have noticed, an observer can very much die in a finite thought experiment. In reality, there is no outside-text dictating what can happen and you would just get quantum-immortaled into hell instead.
This case is not really a scenario exposing a bug in FDT, it's rather FDT exposing an absurdity in the scenario's prior — stipulating that you should be only concerned with one copy of you (the one outside the simulation).
Thought experiments can arbitrarily decide that one copy of the agent does not correspond to an observer whose values should be catered to and therefore can be assigned a probability/utility of 0. But you, the one who implements the algorithm, cannot differentiate between yourselves based on whether they are simulated by Omega, only based on what you have observed.
I notice, that your answer confuses me. My understanding is as follows:
Your choice doesn't change where you exist. In the situation which you describe, not opening the second box doesn't actually improve your situation (being simulated), and I would expect it to go the same way (being shut down).
I agree with the reasoning that you must be in a simulation, but I fail to see how your choice actually changes things, here.
You already exist in one reality (and potentially in a simulation), and you are only finding out in which one you are. So, isn't the only thing you are preserving by not opening the second box your lack of knowledge?
Opening the box doesn't transport you to a different reality, it just either means that your understanding of Omega was incomplete, or that you were in a simulation. But, if you are in a simulation, no matter how you decide, you still are in a simulation.
(I must admit that I said yes without realizing the implications, because I didn't credit the omniscience sufficiently.)
What did I miss?
Extra Credit:
I tell someone else that I got Newcomb'ed and that I will sell them the two boxes for X amount. Neither of us knows what will happen, since I actually won't open either box, but considering there is an omniscient Omega in the world, obtaining that information should be worth more than the money that may or may not be in the boxes.
(To ensure that it doesn't count as opening the boxes by proxy, we settle on a fixed price. I think 250+ could be reasonably negotiated, both considering the value ranges known from Omega, and the potential value of the information.)
Then again, it may really not be a good idea to go out of bounds.
(I quote "'Do not mess with time.' in slightly shaky hand writing."^^)
On the other hand, if Omega meant harm... Well, this is a longer debate, I think.
I am not sure I agree with your stated answer.
If I am being simulated by Omega, then no matter what decision I make I am about to be terminated. Its not like 1 boxing makes the simulated me get to live a full and happy life.
Lets say Omega simulates 1 copy of me, and shows it $100 in box 1. If I have a policy of 1 boxing then that simulation of me 1 boxes, gets terminated (to no benefit to itself), then the real me gets the $100 due to the 1 box policy. If I have a two box policy the simulated me gets $200 before deletion, and the real me gets nothing.
So I agree with the 1 box policy, but at least to me the point of that policy is that it somehow transfers wealth between the simulated me that won't last much longer and the real me who might actually have a chance to benefit from it. I don't think the simulation benefits at all in any case.
If I have a two box policy the simulated me gets $200 before deletion, and the real me gets nothing.
Wait, why does the real you get nothing? It's specified you get $200. What am I missing?
If the simulated me takes two boxes (for $200), before being deleted then Omega will (for the real trial with the real me) put $0 in each box. This is why Omega is doing the simulation in the first place, to work out what I will do so they can fill the boxes correctly. So the real me gets nothing if the simulated me gets $200. This was my logic.
Why would Omega put $0 in the second box? The problem statement specifies Omega puts $100 in both boxes if she predicts you will two-box!
What! I had the whole thing back to front. In that case you are completely correct and you obviously always two box.
I should have read more carefully. Still I would have preferred the presentation highlighted explicitly that it is the opposite way around than anyone would assume.
Re-reading the initial post, and especially the answer it gives, I am still not sure which problem was intended. The model solution offered seems to only make sense in the setup I thought we had. (Otherwise seeing $100 with an intent to take two boxes would not make us update towards thinking we were simulated).
You are assuming a simulation does not want to die, and this is unclear. The fact that 100$ is better than 0$ is taken as an axiom because it is part of the statement. However, death is worse than life (for a simulation!) is not trivial. "Rationality should not be done for rationality or it ends in a loop". So posts use money as the thing the rational agents want. You have to assign a financial value to life before saying it is less valuable than 100$
This is why the Halting Problem is a thing, since you (or the simulation of you) can try to predict what Omega does and act differently based on that, so by simulating you, Omega ends up indirectly trying to predict itself.
One-box only occurs in simulations, while two-box occurs in and out of simulations.
If I one-box in simulations, then Omega puts $0 in the first box, and I can't one-box.
If I two-box in simulations, then Omega puts $100 in the first box, so I may be in a simulation or not.
One-boxing kills me, so I two-box.
Either I've made a mistake, or you have. Where is it?
If you're being simulated by Omega, then opening the second box ends the simulation and kills you.
Is making decisions based on being simulated really coherent?
One may also imagine magic function is_counterfactual_evaluated, where you can do this query to the universe at an arbitrary time and get to know whether you're in simulation. However, Omega has power to simulate you in the case where you don't understand that you're in simulation - that is, where is_counterfactual_evaluated returns false.
Also, by simulating this scenario you kill both simulated-you and Omega. Doesn't this have extremely negative utility? :-)
The distinction between simulation and reasoning/analysis is relevant here. You really can attempt to figure out if a version of you is being simulated by having a working is_counterfactual_evaluated, which should be the model of how the simulator decides to simulate various things, that is you can figure out if you are being simulated by reasoning about your alleged simulator, seeing if it's going to simulate you in particular. Intervening into the details of your is_counterfactual_evaluated makes the resulting computation not-a-faithful-simulation, but it could still be some sort of reasoning. Making a useful counterfactual incentivizes the simulator to intervene into is_counterfactual_evaluated to see how you would behave in the real world, where you might predict that you are not being simulated (though more carefully, these options are not mutually exclusive).
So there is potentially an adversarial game between the counterfactual, which wants to figure out if it's real and should behave as if it's real, or if it's a hypothetical and so should behave in a way that optimally misleads those who would attempt to run a counterfactual; and the predictor who wants to prevent the counterfactual from finding out that it's counterfactual, without affecting it in any other way. But this doesn't necessarily settle into an adversarial equilibrium, as both parties might want to form good models of each other to coordinate and bargain more effectively.
Failed solution to extra credit.
Be a one-boxer. Open the second box first. Take the $100. Then open the first box and take the $100. You will not open the second box after opening the first box, so this is self-consistent. Net $200. Not $250.
There are two boxes. If you open a box, you get to keep the contents.
Omega is an omniscient oracle. Omega predicts what will happen if you open the first box and observe that it contains $100.
You open the first box. It contains $100. Do you open the second box?
No. If you open the second box, that means you're being simulated by Omega. If you're being simulated by Omega, then opening the second box ends the simulation and kills you.
You have not yet opened the first box. How do you generate (at least) $250 in profit? [Please use spoiler tags or equivalent obfuscation when discussing the extra credit.]