Intended for Levels 0-1.9 Related to: Playing Video Games In Shuffle Mode
Taking the advice of talisman, I thought it would be useful to compile a list of levels of general rationalist skill. I think these levels loosely correllate to how effective one is as a rationalist, but they are not the same thing, a level 2 might still compartmentalize. This list is more a specification of the difficulty of material that a person will be able to follow, as well as providing a list of levels above mine, that I can aspire to. Of course, skill is more of a continuum than a discrete set, I've leveled up once before, but I don't remember any single moment in which this occured, just like I don't remember any single moment where I suddenly matured.
The first level, level 0, is the rationality skill of an average 20 year old with internet access, which is the skill of the vast majority of people in the world, if not higher. They've probably forgotten what math they knew beyond a little algebra and arithmetic, and haven't gotten in to the habit of checking facts. These are the kind of people who think Spock when you mention logic. This level of skill is something any rationalist has to go through, so the majority of people who arrive here without coming from OB will probably fall in to this category. With any luck, these people might discover debate forums, and seeing someone shred an argument to pieces introduces them to the idea of a logical fallacy, and begins their transition in to level 1.
The transition from level 0 to level 1 is fairly painless. With a good reading list, it should probably take about three months. Level 1 is the stage where people start self-identifying as rationalists. The usual transition to this stage comes from thinking about rationality as rhetoric, many rationalists become rationalists after a long time spent arguing with creationists and other crackpots, and slowly developing a list of silly arguments and misconcpetions and why they're so silly, until they've got a big enough list that it becomes a matter of turning on the "bullshit detector". After a bit of practice doing this, people become more willing to think hard about their beliefs after their arguments are torn to shreds in the same way that they tore other people's arguments down.
Knowing (to the core of one's being) the fundamentals of logic: that a good argument should go from shared assumptions to a conclusion with clear and explicit reasoning every step of the way - marks a level 1 or higher. People at level 1 also generally know about the scientific method and the scientific ideas that crackpots talk about, some standard fallacies and cognitive biases, and understand the law of truly large numbers: that with a big enough sample space, all sorts of weird coincidences can happen. It's much easier to apply the techniques of rationality to propositions than it is to empirical expectations, which is why most books on the subject cover those, and why you should try to translate the latter in to the former when possible. Learning how to be rational about empirical expectations is the driving force behind transitioning to level 2.
The transition from level 1 to 2 is a rarer and more lengthly process. Level 2 is also much more specialized. Whereas level 1 consists mainly of reasoning about propositions, level 2 involves higher levels of abstraction i.e. thinking about the rules that are required to reason about propositions in a particular domain. To do this without committing serious mistakes requires a great deal of care and rigor, as well as knowledge of the relevant sciences and philosophical principles. When you realize that all those papers on cognitive biases and common fallacies also apply to you, you become immediately suspicious of anything that isn't absolutely explicit and precise. People at level 2 are also much more likely to admit when they don't know something, because the pattern of defining and using a precise and explicit model is nigh impossible when you don't know what you're talking about. Hence formalization is a very important technique at this level, which requires that people at level 2 know advanced math.
Becoming a level 2 isn't easy, it requires learning an advanced model of how to think about problems in a particular domain that may clash with your naive intuitions, being able to trust the math, and more importantly do the math is an essential skill, and one that is difficult to master. Programming and teaching are good practice techniques at this level, because they both require being explicit about your unconscious reasoning. The process of using a formal model to predict and explain things tends to give people at level 2 much better intuitions about problems in their domain of knowledge, as well as a sense of aesthetics about abstracted models. This sense of aesthetics is probably very important in helping a person trying to reach level 3 and come up with their own.
Level 3 is the level where you are able to produce awesome where you not only have an inutitive understanding of the mathematical models in your field, you also created them. At level 3 you don't have a guiding textbook to help you, you are the one that writes them. Very, very few people reach this level, and those who do are heralded as geniuses, such as Newton, Darwin, Godel, Russel or Einstein. I don't know how to become a level 3, I know Eliezer has had some experience trying, but these are the people who change the world.