What is the point of this thought experiment? To demonstrate that thought experiments involving mind reading agents (such as the AI in "Newcomb's Paradox" or Paul Ekman) can create scenarios in which any decision theory will be worse off than another. If we allow these agents to exist then no...
Here is the complete solution to Newcomb's paradox. Setup: Omega performs a brainscan on you at 12pm, and expects you to choose a box at 6pm. Based on the brainscan, Omega makes a prediction and classifies you into two categories, either you will take only 1 box, or you will...
That just means the AI cares about a particular class of decision theories rather than a specific one like Dick Kick'em. I could re-run the same thought experiment but instead Dick Kick'em says:
"I am going to read you mind and if you believe in a decision theory that one-boxes in Newcomb's Paradox I will leave you alone, but if you believe in any other decision theory I will kick you in the dick"
In this variation, Dick Kick'em would be judging the agent based on the exact same criterea that the AI in Newcomb's problem is using. All I have done is remove the game afterwards but that is somewhat irrelevant since the AI doesn't judge you on your actions, just what you would do if you were in a Newcomb-type scenario.
Omega only cares about your beliefs insofar as they affect your actions
So does Dick Kick'em, since he only cares about distinct decision theories that a particular agent believes in, and that in turn decides the agent's actions.
The agent in this scenario doesn't necessarily know if the driver can read faces or not, in the original problem the agent isn't aware of this information. Surely if FDT advises you pay him on arrival in the face reading scenario, you would do the same in the non-face reading scenario since the agent can't tell them apart.
Mindreading agents do happen in real life but they are often wrong and can be fooled. Most decision theories on this website don't entertain either of these possibilities. If we allow "fooling a predictor" as a possible action then the solution to Newcomb's problem is easy: simply fool the predictor and then take both boxes.
Seriously though, Newcomb's setup is not adversarial in the same way, the predictor rewards or punishes you for actions, not beliefs.
This cannot be true becuase it would violate cause and effect. The predictor will decide to reward/punish you with the amount of money put in the boxes. This reward/punishment is done BEFORE any action is taken, and so it is based purely on the beliefs of the agent. If it were based on the actions of the agent, that would mean that the cause of the reward/punishment happens AFTER the decision was made, which violates cause and effect. The cause must come BEFORE.
Yet FDT does appear to be superior to CDT and EDT in all dilemmas where the agent’s beliefs are accurate and the outcome depends only on the agent’s behavior in the dilemma at hand
This is not true in cases even where mind-reading agents do not exist.
Consider the desert dilemma again with Paul Ekman, except he is actually not capable of reading people's mind. Also assume your goal here is to be selfish and gain as much utility for yourself as possible. You offer him $50 in exchange for him taking you out of the desert and to the nearest village, where you will be able to draw out the money and pay... (read more)
I believe that the AI does care about your beliefs, just not specific beliefs. The AI only cares about if your decision theory falls into the class of decision theories that will pick two boxes, and if it does then it punishes you. Sure, unlike Dick Kick'em the AI isn't looking for specific theories just any theory within a specific class, but it is still the same thing. The AI is punishing the agent by putting in less money based soley on your beliefs. In Newcomb's paradox, the AI scans your brain BEFORE you take any action whatsoever, the punishment cannot be based on your actions, the punishment from the AI is based only on your beliefs. This is exactly the same as the Dick Kick'em Paradox; Dick will punish you purely on your beliefs, not any action. The only difference is that in Newcomb's paradox you get to play a little game after the AI has punished you.
What is the point of this thought experiment?
To demonstrate that thought experiments involving mind reading agents (such as the AI in "Newcomb's Paradox" or Paul Ekman) can create scenarios in which any decision theory will be worse off than another. If we allow these agents to exist then no decision theory will be stricly better than anther.
Let's say we have two competing decision theories ADT and BDT, these can be any two decision theories you like, so long as they are different. Here, different is meant that there exists a scenario where ADT leads to a different decision than BDT.
Consider a mind-reading agent like Paul Ekman or a super intelligent AI... (read 160 more words →)
Well the decision theory is only applied after assessing the available options, so it won't tell you to do things you aren't capable of doing.
I suppose the bamboozle here is that it is that it seems like a DT question, but, as you point out, it's actually a question about physics. However, even in this thread, people dismiss any questions about that as being "not important", and instead try to focus on the decision theory, which isn't actually relevant here.
For example:
I generally think that free will is not so relevant in Newcomb's problem. It seems that whether there is some entity somewhere in the world that can predict what I'm doing shouldn't make a difference for whether I have free will or not, at least if this entity isn't revealing its predictions to me before I choose.
Personally I don't believe that the problem is actually hard. None of the individual cases are hard, the decsions are pretty simple once we go into each case. Rather I think this question is more of a philosophical bamboozle that is actually more of a question about human capabilities, and disguises itself as a problem about decision theory.
As I talk about in the post, the answer to the question changes depending on which decisions we afford the agent. Once that has been determined, it is just a matter of using min-max to find the optimal decision process. So people's disagreements aren't actually about decision theory, rather just disagreements about which choices are... (read more)
Here is the complete solution to Newcomb's paradox.
Omega performs a brainscan on you at 12pm, and expects you to choose a box at 6pm. Based on the brainscan, Omega makes a prediction and classifies you into two categories, either you will take only 1 box, or you will take 2 boxes. The AI is very good at classifying brainscans.
Box A is transparent and contains £1k
Box B has £1m if Omega thinks you are a 1-box person, or contains £0 if you are a 2-box person.
Do you choose to take both boxes, or only box B?
Note: I mention the brainscan to make things more concrete, the exact method of how Omega performs the... (read 817 more words →)
True beliefs doesn't mean omniscience. It is possible to have only true beliefs but still not know everything. In this case, the agent might not know if the driver can read minds but still have accurate beliefs otherwise.