aDPOLISH
6
6
aDPOLISH has not written any posts yet.
aDPOLISH has not written any posts yet.
You're right about the payoff matrix, I guess newcomb's problem doesn't have a payoff matrix at all, since there's no payoff defined for the person filling the boxes.
What do you mean by "prove a fact about their decision theory"? Do you mean that you're proving "a rational AI would use decision theory X and therefore use strategy Y", or do you only mean "GIVEN that an AI uses decision theory X, they would use strategy Y"?
There seems to be a belief floating around this site that an AI could end up using any old kind of decision theory, depending on how it was programmed. Do you subscribe to this?
The "horrible strategy", newcomb's... (read more)
I take it your implication is that you could play the game with a superintelligent entity somewhere far in spacetime. If this is your plan, how exactly are you going to get the results back? Not really a test if you don't get results.
No, it's not. You might be able to guess that a superintelligence would like negentropy and be ambivalant toward long walks on the beach, but this kind of "simulating" would never, ever, ever, ever allow you to beat it at paper scissor rock. Predicting which square of a payoff matrix it will pick, when it is to the interest of the AI to pick a different square than you think it will, is a problem of the latter type.
This is a general purpose argument against all reasoning relating to superintelligences, and aids your argument no more than mine.
The only way you can "bargain" with somebody from the past is if you can be PRESENT in the past in a simulation they're running. That's how newcomb's problem works, omega is simulating you at time 0, and via that simulation, you have two-way interaction.
In this scenario, YOU have to be simulating THE AI at time 0, in your human imagination. This is not possible. The fact that the payoff matrix is simple does not make the opponent's reasoning simple, and in order for "bargaining" to happen, their reasoning has to be not only simple, but, crucially, tied to your actions. The situation is, exactly as I described, newcomb's problem with you preparing the boxes and omega picking.
Oh really? You have an omega sitting around you can test game theory problems with? Omniscient super-intelligent being, maybe in your garage or something?
Seriously though, for the decision of the person who picks the box to influence the person who puts in the money, the person who puts in the money has to be able to simulate the thinking of the person who picks the box. That means you have to simulate the thinking of Omega. Given that omega is smart enough to simulate YOUR thinking in perfect detail, this is patently impossible.
The only reason for omega to two-box is if your decision is conditional on his decision, and much as he might wish it was, no amount of super-intelligence or super-rationality on his part is going to give you that magical insight into his mind. He knows whether you put the money in the box, and he knows that which box he picks has no influence on it.
This "horrible strategy" is basically built on a kind of farcical newcomb's problem where both boxes are transparent, you're the one filling them with money, and omega is the one picking. It doesn't work at all.
If people would stop cranking their hypothetical scenarios up to 11 in the interest of "seriousness", maybe they could stop hyperventilating long enough to notice these things...
Actually come to think of it, an even better analogy than a switched up newcomb's problem is a switched up parfit's hitchhiker. The human vs. human version works, not perfectly by any means, but at least to some extent, because humans are imperfect liars. You can't simulate another human's brain in perfect detail, but sometimes you can be a step ahead of them.
If the hitchhiker is omega, you can't. This is a bad thing for both you and omega, but it's not something either of you can change. Omega could self-modify to become Omega+, who's just like omega except that he never lies, but he would have no way of proving to... (read more)