-- Eliezer Yudkowsky, Making Beliefs Pay Rent (In Anticipated Experiences)
One principle of rationality is that "beliefs should pay rent in anticipated experiences." If you believe in something, what do you expect to be different as a result? What does the belief say should happen, and what does it say should not happen? If you have a verbal disagreement with someone, how does your disagreement cash out in differing expectations?
If two people try to get specific about the anticipated experiences driving their disagreement, one method for doing so is the double crux technique. The notion that beliefs are models of what we expect to experience is also one of the basic premises of predictive processing theories of how the brain works. Beliefs that do not pay rent may be related to meaningless arguments driven by coalitional instincts.
If a tree falls in a forest and no one hears it, does it make a sound? One says, “Yes it does, for it makes vibrations in the air.” Another says, “No it does not, for there is no auditory processing in any brain.” [...]
Suppose that, after a tree falls, the two arguers walk into the forest together. Will one expect to see the tree fallen to the right, and the other expect to see the tree fallen to the left? Suppose that before the tree falls, the two leave a sound recorder next to the tree. Would one, playing back the recorder, expect to hear something different from the other? Suppose they attach an electroencephalograph to any brain in the world; would one expect to see a different trace than the other?
Though the two argue, one saying “No,” and the other saying “Yes,” they do not anticipate any different experiences. The two think they have different models of the world, but they have no difference with respect to what they expect will happen to them; their maps of the world do not diverge in any sensory detail.