That just means the AI cares about a particular class of decision theories rather than a specific one like Dick Kick'em. I could re-run the same thought experiment but instead Dick Kick'em says:
"I am going to read you mind and if you believe in a decision theory that one-boxes in Newcomb's Paradox I will leave you alone, but if you believe in any other decision theory I will kick you in the dick"
In this variation, Dick Kick'em would be judging the agent based on the exact same criterea that the AI in Newcomb's problem is using. All I have done is remove the game afterwards but that is somewhat irrelevant since the AI doesn't judge you on your actions, just what you would do if you were in a Newcomb-type scenario.
Omega only cares about your beliefs insofar as they affect your actions
So does Dick Kick'em, since he only cares about distinct decision theories that a particular agent believes in, and that in turn decides the agent's actions.
The agent in this scenario doesn't necessarily know if the driver can read faces or not, in the original problem the agent isn't aware of this information. Surely if FDT advises you pay him on arrival in the face reading scenario, you would do the same in the non-face reading scenario since the agent can't tell them apart.
Mindreading agents do happen in real life but they are often wrong and can be fooled. Most decision theories on this website don't entertain either of these possibilities. If we allow "fooling a predictor" as a possible action then the solution to Newcomb's problem is easy: simply fool the predictor and then take both boxes.
Seriously though, Newcomb's setup is not adversarial in the same way, the predictor rewards or punishes you for actions, not beliefs.
This cannot be true becuase it would violate cause and effect. The predictor will decide to reward/punish you with the amount of money put in the boxes. This reward/punishment is done BEFORE any action is taken, and so it is based purely on the beliefs of the agent. If it were based on the actions of the agent, that would mean that the cause of the reward/punishment happens AFTER the decision was made, which violates cause and effect. The cause must come BEFORE.
Yet FDT does appear to be superior to CDT and EDT in all dilemmas where the agent’s beliefs are accurate and the outcome depends only on the agent’s behavior in the dilemma at hand
This is not true in cases even where mind-reading agents do not exist.
Consider the desert dilemma again with Paul Ekman, except he is actually not capable of reading people's mind. Also assume your goal here is to be selfish and gain as much utility for yourself as possible. You offer him $50 in exchange for him taking you out of the desert and to the nearest village, where you will be able to draw out the money and pay him. He can't read your mind but judges that the expected value is positive given most people in this scenario would be telling the truth. CDT says that you should simply not pay him when you reach the village, but FDT has you $50 short. In this real world scenario, that doesn't include magical mind-reading agents, CDT is about $50 up from FDT.
The only times FDT wins against CDT is in strange mind-reading thought experiments that won't happen in the real world.
I believe that the AI does care about your beliefs, just not specific beliefs. The AI only cares about if your decision theory falls into the class of decision theories that will pick two boxes, and if it does then it punishes you. Sure, unlike Dick Kick'em the AI isn't looking for specific theories just any theory within a specific class, but it is still the same thing. The AI is punishing the agent by putting in less money based soley on your beliefs. In Newcomb's paradox, the AI scans your brain BEFORE you take any action whatsoever, the punishment cannot be based on your actions, the punishment from the AI is based only on your beliefs. This is exactly the same as the Dick Kick'em Paradox; Dick will punish you purely on your beliefs, not any action. The only difference is that in Newcomb's paradox you get to play a little game after the AI has punished you.
Well the decision theory is only applied after assessing the available options, so it won't tell you to do things you aren't capable of doing.
I suppose the bamboozle here is that it is that it seems like a DT question, but, as you point out, it's actually a question about physics. However, even in this thread, people dismiss any questions about that as being "not important", and instead try to focus on the decision theory, which isn't actually relevant here.
For example:
I generally think that free will is not so relevant in Newcomb's problem. It seems that whether there is some entity somewhere in the world that can predict what I'm doing shouldn't make a difference for whether I have free will or not, at least if this entity isn't revealing its predictions to me before I choose.
Personally I don't believe that the problem is actually hard. None of the individual cases are hard, the decsions are pretty simple once we go into each case. Rather I think this question is more of a philosophical bamboozle that is actually more of a question about human capabilities, and disguises itself as a problem about decision theory.
As I talk about in the post, the answer to the question changes depending on which decisions we afford the agent. Once that has been determined, it is just a matter of using min-max to find the optimal decision process. So people's disagreements aren't actually about decision theory, rather just disagreements about which choices are available to us.
If you allow the agent to decide their brainstate and also act independently from it, then it is easy to see the best solution is (1-box brainstate) -> (Take 2 boxes). People who say "that's not possible because omega is a perfect predictor" are not actually disagreeing about the decision theory, rather it's just disagreeing about if humans are capable of doing that.
True beliefs doesn't mean omniscience. It is possible to have only true beliefs but still not know everything. In this case, the agent might not know if the driver can read minds but still have accurate beliefs otherwise.