Epistemic status: Some babble, help me prune.

My thoughts on the basic divide between rationalist and post-rationalists, lawful thinkers and toolbox thinkers.

Rat thinks: "I'm on board with The Great Reductionist Project, and everything can in theory be formalized."

Post-Rat hears: "I personally am going to reduce love/justice/mercy and the reduction is going to be perfect and work great."

Post-Rat thinks: "You aren't going to succeed in time / in a manner that will be useful for doing anything that matters in your life."

Rat hears: "It's fundamentally impossible to reduce love/justice/mercy and no formalism of anything will do any good."

Newcomb's Problem

Another way I see the difference is that the post-rats look at Newcomb's problem and say "Those causal rationalist losers! Just one-box! I don't care what your decision theory says, tell your self whatever story you need in order to just one-box!" The post-rats rally against people who are doing things like two-boxing because "it's optimal".

The most indignant rationalists are the one's who took the effort to create whole new formal decision theories that can one-box, and don't like that the post-rats think they'd be foolish enough to two-box just because a decision theory recommends it. While I think this gets the basic idea across, this example is also cheating. Rats can point to formalism that do one-box, and in LW circles there even seem to be people who have worked the rationality of one-boxing deep into their minds.

Hypothesis: All the best rationalists are post-rationalists, they also happen to care enough about AI Safety that they continue to work diligently on formalism.

Hazard's Shortform Feed

by Hazard 1 min read4th Feb 2018219 comments

In light of reading through Raemon's shortform feed, I'm making my own. Here will be smaller ideas that are on my mind.