I asked Claude about this, and here are some of the things it came up with for disambiguating the different confounds like inability to think hypothetically:
• “Imagine a world where water is poisonous to humans but juice is free and everywhere. In this world, would it be a good idea to drink water?” (Anyone saying yes is failing to enter the hypothetical.)
• “Suppose scientists discovered that exercise actually makes people less healthy. In that world, should doctors recommend exercise?” (Tests whether people can override a real-world belief when told to.)
I asked Claude about this, and here are some of the things it came up with for disambiguating the different confounds like inability to think hypothetically:
• “Imagine a world where water is poisonous to humans but juice is free and everywhere. In this world, would it be a good idea to drink water?” (Anyone saying yes is failing to enter the hypothetical.)
• “Suppose scientists discovered that exercise actually makes people less healthy. In that world, should doctors recommend exercise?” (Tests whether people can override a real-world belief when told to.)
•... (read more)