I have some issues with the third experiment (URSS vs. USA). Let me try to explain them with an example.
Suppose you see a chess match up to a given point, with black to move. You are asked an estimate W1 of the probability that white can force a win. Then you see black's move, and it's a truly, unexpectedly brilliant move. You are then asked a new estimate W2 of the probability that white can force a win. If black's move is sufficiently brilliant, it's natural for W2 to be lower than the answer a "previous you" gave for W1: black has seriously undermined white's chances with his move. But the Russia vs. USA example seems to suggest that any pair of answers where W2<W1 is a fallacy. After all, the set of possible directions of play before black's move includes all possible directions of play after black's move. If white can force a win in all the former, it can also force a win in all the latter, which are a subset of the former. So one could argue that any rational analyst should always output W2 at least as high as W1.
I think the catch is that having more details made explicit, even details that we implicitly knew ("please think about the possibility of Russia invading Poland") can allow us to reason "more accurately" about the likelihood of a given, complex situation (the diplomatic relations between Russia and the USA failing). It's quite possible that, in the light of those details, our evaluation of the likelihood increases sufficiently to more than compensate for the decrease of likelihood involved in considering just a subcase (the relations failing AND an invasion of Poland). Is this a fallacy? I would rather call it a case of computation resources inadequate to the task at hand - resources that become (more) adequate with a little boost or "hint" allowing the evaluation process to run more efficiently.
In this sense, it would have been interesting to see the results if the first statement had been something along the lines:
"A complete suspension of diplomatic relations between the USA and the Soviet Union, sometime in 1983 (keeping in mind that it's not impossible that the Soviet Union invades Poland)". Or if the analysts had been given the questions, and two months of full time study of US and URSS history.
I have some issues with the third experiment (URSS vs. USA). Let me try to explain them with an example.
Suppose you see a chess match up to a given point, with black to move. You are asked an estimate W1 of the probability that white can force a win. Then you see black's move, and it's a truly, unexpectedly brilliant move. You are then asked a new estimate W2 of the probability that white can force a win. If black's move is sufficiently brilliant, it's natural for W2 to be lower than the answer a "previous you" gave for W1: black has seriously undermined white's chances with his move. But the Russia vs. USA example seems to suggest that any pair of answers where W2<W1 is a fallacy. After all, the set of possible directions of play before black's move includes all possible directions of play after black's move. If white can force a win in all the former, it can also force a win in all the latter, which are a subset of the former. So one could argue that any rational analyst should always output W2 at least as high as W1.
I think the catch is that having more details made explicit, even details that we implicitly knew ("please think about the possibility of Russia invading Poland") can allow us to reason "more accurately" about the likelihood of a given, complex situation (the diplomatic relations between Russia and the USA failing). It's quite possible that, in the light of those details, our evaluation of the likelihood increases sufficiently to more than compensate for the decrease of likelihood involved in considering just a subcase (the relations failing AND an invasion of Poland). Is this a fallacy? I would rather call it a case of computation resources inadequate to the task at hand - resources that become (more) adequate with a little boost or "hint" allowing the evaluation process to run more efficiently.
In this sense, it would have been interesting to see the results if the first statement had been something along the lines: "A complete suspension of diplomatic relations between the USA and the Soviet Union, sometime in 1983 (keeping in mind that it's not impossible that the Soviet Union invades Poland)". Or if the analysts had been given the questions, and two months of full time study of US and URSS history.