Suppose you say that the probability that the mugger has magical powers, and will deliver on any promise he makes, is 1 in 10^30. But then, instead of promising you quadrillions of days of extra life, the mugger promises to do an easy card trick. What's your estimate of the probability that he'll deliver? (It should be much closer to 0.8 than to 10^-30).
That's because the statement "the mugger will deliver on any promise he makes" carries with it an implied probability distribution over possible promises. If he promises to do a card trick, the probability that he delivers on it is very high; if he promises to deliver quadrillions of years of life, it's very low. When you made your initial probability estimate, you didn't know which promise he was going to make. After he reveals the details, you have new information, so you have to update your probability. And if that new information includes an astronomically large number, then your new probability estimate ought to be infinitesimally small in a way that cancels out that astronomically large number.
And if that new information includes an astronomically large number, then your new probability estimate ought to be infinitesimally small in a way that cancels out that astronomically large number.
Er, can you prove that? It doesn't seem at all obvious to me that magic power improbability and magic power utility are directly proportional. Any given computation's optimization power isn't bounded in one to one correspondence by its Kolmogorov complexity as far as I can see, because that computation can still reach into other computations and flip sign bits...
Related to: Some of the discussion going on here
In the LW version of Pascal's Mugging, a mugger threatens to simulate and torture people unless you hand over your wallet. Here, the problem is decision-theoretic: as long as you precommit to ignore all threats of blackmail and only accept positive-sum trades, the problem disappears.
However, in Nick Bostrom's version of the problem, the mugger claims to have magic powers and will give Pascal an enormous reward the following day if Pascal gives his money to the mugger. Because the utility promised by the mugger so large, it outweighs Pascal's probability that he is telling the truth. From Bostrom's essay:
As a result, says Bostrom, there is nothing from rationally preventing Pascal from taking the mugger's offer even though it seems intuitively unwise. Unlike the LW version, in this version the problem is epistemic and cannot be solved as easily.
Peter Baumann suggests that this isn't really a problem because Pascal's probability that the mugger is honest should scale with the amount of utility he is being promised. However, as we see in the excerpt above, this isn't always the case because the mugger is using the same mechanism to procure the utility, and our so our belief will be based on the probability that the mugger has access to this mechanism (in this case, magic), not the amount of utility he promises to give. As a result, I believe Baumann's solution to be false.
So, my question is this: is it possible to defuse Bostrom's formulation of Pascal's Mugging? That is, can we solve Pascal's Mugging as an epistemic problem?