This is a special post for quick takes by leogao. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
i feel like the fundamental mistake the project of rationality made was that "cognitive biases" is not in practice the right way to think about the way humans are irrational if your goal is to be very instrumentally rational. one hypothesis is the correct frame is to first deeply understand how the emotional system works, and then to think about ways to master that system to achieve rationality.
(yes, i know that buried somewhere in the sequences it says something like "humans aren't ideal intelligences with cognitive biases bolted on. we are the cognitive biases, they are just trying to approximate rationality".)
11210
If we understand "irrational" to mean something like "underperforming relative to what should be feasible", then I think one significant piece of the puzzle is the regime of "very impoverished hypothesis spaces". In this regime, Bayes (and deviations from it) is more of a peripheral conceptual frame (allowing you to understand some edge cases and some basics and some constraints) than a central guide or even that much of a useful tool. A much more important question is about hypothesis generation, aka abduction or "unupdating", i.e. expanding your hypothes...
11
I think a big part of rationality is having all the algorithms in your brain "point in the same direction". E.g. akrasia is a special case of this, where short-term "habits" and consequentialist reasoning are in conflict.
17
By the time I went to CFAR in 2019 this felt like it had already become the dominant flavor of inner-circle rationalist thinking, but then that inner circle kind of petered out in influence. The person I see carrying that torch most loudly in my current social atmosphere is Chris Lakin.
But overall rationality has been kind of quiescent imo! Ray posts good stuff, Duncan has his own thing, but it feels like we went from mid-2010s “rationalists talk a big game but don’t get anything done” to the mid-2020s most influential rationalists being too object-level busy to blog much about this metacognitive stuff.