Previously: Testing for Rationalization

One of the red flags was "disagreeing with experts". While all the preceding tools apply here, there's a suite of special options for examining this particular scenario.

The "World is Mad" Dialectic

Back in 2015, Ozymandias wrote:

I think a formative moment for any rationalist– our “Uncle Ben shot by the mugger” moment, if you will– is the moment you go “holy shit, everyone in the world is fucking insane.”

First, you can say “holy shit, everyone in the world is fucking insane. Therefore, if I adopt the radical new policy of not being fucking insane, I can pick up these giant piles of utility everyone is leaving on the ground, and then I win.”

Second, you can say “holy shit, everyone in the world is fucking insane. However, none of them seem to realize that they’re insane. By extension, I am probably insane. I should take careful steps to minimize the damage I do.”

I want to emphasize that these are not mutually exclusive. In fact, they’re a dialectic (…okay, look, this hammer I found is really neat and I want to find some place to use it).

(I would define a "dialectic" as two superficially opposite statements, both of which are true, in such a way that resolving the apparent paradox forces you to build a usefully richer world-model. I have not run this definition past Ozy, much less Hegel.)

To which Eliezer replied in 2017:

But, speaking first to the basic dichotomy that’s being proposed, the whole point of becoming sane is that your beliefs shouldn’t reflect what sort of person you are. To the extent you’re succeeding, at least, your beliefs should just reflect how the world is.

Good News, Everyone!

I did an empirical test and found no sign of a general factor of trusting your own reasoning.

But I did find lots of disagreement. What are people deciding based on?

Types of Disagreement

Explicit

The expert actually said the opposite of your conclusion.

This is the simplest form of disagreement. The expert could still be wrong, or lying, or communicating unclearly and not actually saying what you think they said. But at least there's no complications from the form of disagreement itself.

The "communicating unclearly" hypothesis should always be high in your p...

Against Rationalization II

Against Rationalization II