[Conversation Log] Compartmentalization

The most compact explanation of compartmentalization I've come up with is:

You are shown a convincing argument for the proposition that paraquat weedkiller is an effective health tonic.
You believe the argument, you want to be healthy, and you can't think of any logical reason you shouldn't act on your belief.
Nonetheless, you fail to actually drink any paraquat weedkiller.

Sure, there was in fact a flaw somewhere in the argument. But evolution wasn't able to give us brains that are infinitely good at spotting flaws in arguments. The best it could come up with was brains that do not in fact think, say and do all the things they logically should based on their declaratively held beliefs.

[-]The Dao of Bayes14y50

That one strikes me mostly as being a subconscious standard of proof that says "it requires X level of confidence to accept an argument as convincing, but X+n confidence before I am so convinced as to do something that I intuitively expect to kill me."

It also strikes me as eminently sensible, and thus doesn't do well at illustrating compartmentalisation as a bad thing :)

[-]orthonormal14y00

It was explicitly written to show the evolutionary benefit of compartmentalization, not the problems it causes. Heuristics endure because (in the ancestral environment) they worked better on average than their alternatives.

[-]TheOtherDave14y40

While not disagreeing with the main thrust here, thinking of compartmentalization as something explicitly selected for may be misleading.

Not everything in an evolved system is explicitly selected for, after all. And it seems at least equally plausible to me that compartmentalization is a side-effect of a brain that incorporates innumerable mechanisms for deriving cognitive outputs (including beliefs, assertions, behaviors, etc.) from inputs that themselves evolved independently.

That is, if each mechanism is of selection-value to its host, and the combination of them is at least as valuable as the sum of them individually, then a genome that happens to combine those mechanisms is selected for. If the additional work to reconcile those mechanisms (in the sense we're discussing here) either isn't cost-effective, or just didn't happen to get generated in the course of random mutation, then a genome that reconciles them doesn't get selected for.

[-]CronoDAS14y50

One reduction of "compartmentalization" is "failure to notice, or act on, inconsistent beliefs and/or desires".

For example, in G.E.B., Douglas Hofstader describes a situation in which he found himself bored while driving, so he attempted to turn on his broken radio. Another time, he and his wife, when traveling in Paris, decide that they want a hotel room on the opposite side of the building from the American embassy because they're concerned about terrorism, but when asked if they want a room with a better view of the embassy gardens, accept it - and only later realize what they ended up doing.

Hofstatder goes on to state that the question "Do my beliefs imply a contradiction?" is NP-hard even in the restricted case in which all your beliefs can be expressed as statements in propositional logic; as you add more beliefs to a system, the number of ways to potentially deduce a contradiction goes up exponentially - and, therefore, some form of compartmentalization is mathematically inevitable because nothing can do the 2^zillion computations necessary to check all the beliefs in a human's head for consistency.

[+]Oscar_Cunningham14y-50

[-]Nisan14y20

User:AdeleneDawner talks like User:MBlume.

[-]shokwave14y70

Nisan talks like Clippy.

[-]AdeleneDawner14y10

That probably needs a "when talking to handoflixue" qualifier for maximum accuracy, but I'll take it as a compliment in any case. ^.^

[-]Alicorn14y00

I'm not seeing it. What prompts you to spot this resemblance?

[-]Nisan14y10

Just *nodnods* and ^^. Actually, handoflixue uses the latter idiom as well.

[-]lucidfox14y10

whereupon if I'm playing WOW, I roleplay an elf. <...> If I'm on LessWrong, I roleplay a rationalist.

Or you can roleplay a rationalist elf in WoW. :)

A long time ago, back before I quit WoW, I roleplayed an atheist draenei who refused to believe in the night elf goddess Elune. The catch here is that we players know she actually exists in the setting, because Blizzard told us so, but the characters would have no way of verifying this since she never appeared in the world in person. From my character's point of view, the magical powers that priests of Elune attributed to their goddess were actually (unknown to them) given to them by other, non-personified sources of power followed by other priests in the setting.

[This comment is no longer endorsed by its author]Reply

[-]The Dao of Bayes14y10

Heeeh, very cute! My first foray in to fantasy had a God of Atheists, who derived power from non-belief in other Gods :)

[-]Dreaded_Anomaly14y10

In the interest of not reinventing the wheel, I'll just link to Taking Ideas Seriously, which is mostly about "failure to compartmentalize" but in service of that ends up illustrating compartmentalization fairly well, too.

[-]Will_Newsome14y80

I think I did a pretty terrible job of writing that post, even if the spirit is roughly correct. Anna's followup posts were much better written. Here's the most relevant one.

[-]Dreaded_Anomaly14y10

I hadn't seen that, thanks!

LESSWRONG
LW

LESSWRONG
LW

6

[Conversation Log] Compartmentalization

6

6