To continue with this metaphor, it seems what we need is a good set of problems to test our rationality (i.e., ability to resist common biases). As with any cognitive test, the details of the test can't be known in advance, or people would just memorize answers instead of developing skills. So we need to collect a rather large set of good test questions, or create a generators to create sets of them automatically. Not easy, but perhaps worth doing.

It seems that calibrating personal judgements using decision markets could in principle be a way to measure rationality. You make a guess, and if your guess is close to what the market predicts, you can be slightly more confident about yourself. Of course, for this to work, you'd have to avoid prior knowledge of the market, and skip participating in betting on topics that you want to consider unassisted first.

Great post btw.

Do we want "techniques" for finding the truth? I suppose if the truth is "the weight of this fruit for sale." But if the truth is more like "what I ought to be doing," there is something closer to the essence of personality involved. If my personality is gained as a technique at a truth dojo, then it does not feel like mine. The necessity of discovering certain truths personally rather through technical instruction is a limit on the usefulness of teaching.

Ultimately, the nature of rationality is a question that goes to essence of personality. If I am willing to receive only deductively derived formula as rational, then I have -- not only a technique of rationality -- but a particularly well-known and annoying personality. If I am willing to act on any emotional or intuitive vapor, then I have another well-known and annoying personality. We are all aware that the grounds upon which I am willing to believe and act are not just techniques of rationality -- they are part of our fundamental identities and determine the way that we form or do not form social relations with others.

Michael, you can't have a prediction market without a way to pay off the bets, and if you have that you can measure personal accuracy directly, if you just wait.

Robin, I agree that the main difficulty is figuring out how to pay off the bets, but it seems to me that - given such a measure - playing a prediction market around the measure makes the game more complex, and hopefully more of a lesson, and more socially involving and personally intriguing. In other words, it's the difference between "Guess whether it will rain tomorrow?" and "Bob is smiling evilly; are you willing to bet $50 that his probability estimate of 36.3% is too low?" Or to look at it another way, fewer people would play poker if the whole theme was just "Estimate the probability that you can fill an inside straight." I think Anissimov has a valid fun-amplifying suggestion here.

Why aren't there dojos that teach rationality?

They're called universities. My PhD adviser taught me to think rationally and caught many of the errors in my thinking, including those caused by biases. This eventually allowed me to catch many of my own errors.

No, it wasn't perfect. It was limited to one academic area and was not explicitly aimed at teaching rationality and overcoming biases. (Then again, martial-arts senseis do not teach their skills "explicitly", but by repetition and training.) Moreover, gaining subject-matter expertise took as much of the effort as learning rational thinking processes. But a PhD in a rigorous field from a good university is as close as you'll find to a "color belt" (dare I say "black belt"?) in rationality.

Joshua, the thought had occurred to me, but with all due respect to universities, that's the same sort of training-in-passing that you get from reading "Surely You're Joking, Mr. Feynman" as a kid. It's not systematic, and it's not grounded in the recent advances in cognitive psychology or probability theory. If we continue with the muscle metaphor, then I would say that - if judged by the sole criterion of improving personal skills of rationality - then studying physics is the equivalent of playing tennis. If you actually want to do physics, of course, that's a whole separate issue. But if you want to study rationality, you really have to study rationality; just as if you wanted to study physics it wouldn't do to go off and study psychology instead.

Steve: Wasn't that the claim of the sophists? "We'll teach you how to win arguments so you can prevail in politics." The problem is that the skills for winning arguments aren't necessarily the schools for rationality in general. Probably the easiest way to learn the skills for winning arguments is to go to law school and "learn to think like a lawyer."

Steve, you're probably right, but I doubt that's really a big obstacle. Rationality has plenty of uses, and if anyone wants to teach it for money, they shouldn't have problems finding ways to promote it, in the long run. Besides, if people want to learn karate to beat people up, why don't dojos advertise based on that?

But I think that the analogy to martial arts is ill-advised for other reasons. Martial arts is a mostly anachronistic practice when evaluated against its original purpose. If you're really serious about self-defense nowadays, you get weapons. (Weapons have always been an advantage, but it's only been for a few hundred years now that they've become such a decisive advantage.)

Where are the tools for giving us a rational advantage? Sure, we have plenty of them for bringing knowledge to us quickly. The internet has been pretty successful at that. But what about other prostheses that specifically target our rational acuity?

Just as one style of martial arts benefits mainly one area, say the feet, another is totally dedicated to the fists. To measure the growth in each of these forms, there are different 'belts' or levels of mastery based on what you have learned. So perhaps rationality cannot be measured in just one way; the different applications would have to have different levels, just as schools have letter grades on different subjects, and then an overall GPA. Practical usage, such as in the workplace, would have to be 'graded' differently than the same practical usage in a social setting.

Re-reading this brings to mind the history of the Logicians in ancient China. Apparently, they fell out of favor because all that their skills seemed to be good for was winning arguments, and others already had much more effective ways of winning arguments. Advertising rationality as a way to win arguments, as suggested by some of the original commenters, could cause rationality to fall into a similar trap.

I think this problem is avoided by your (later) idea of a Bayesian Conspiracy; if practitioners of beisudo were demonstrably better at the various instrumental tasks they undertook, then that might be enough advertising to attract people to rationality.

I also wonder if cargo-cult rationality would be good or bad. It seems like it could go either way.

Well, if we have a Bayesian Conspiracy in place, as it were, then I reckon the test would be derivation of hidden knowledge from available priors. This would require that the Conspiracy have access to answers reasoned out ahead of time, and also that the actual testing was kept secret. Perhaps an aspiring rationalist may only be tested once. If we have that strict a measure, though, we might consider allowing a certain amount of leniency... but then again, perhaps only those who strictly adhere to the Art deserve to move further into its ranks.

I am quite aware that my idea is based largely on a post you made about 16 months after this one; in that sense, I'm simply perpetuating a feedback loop, and may be acting in redundancy. But the more I think on this problem, the more convinced I am that it is a Good Idea: test, ceremony, Conspiracy, and all.

An AI psychologist would stand the best chance of learning human interactions and rational. An AI in that position could quickly learn how we work and understand things. The programmers would also have a great view of how the AI responds.

As for the Dojo idea, we may now have that with, "luminosity.com", possibly the world's first online mental Dojo. A local Dojo would lack the visibility needed to ensure the correct training was being applied. If it were just local it would be more like theology rather than rational learning.

I try to explore ideas and concepts that put people outside the realm or normality on my site thinkonyourown.com, I've found that challenging people's limits is a fantastic way of exercising the mental muscles.

I was arguing the other day with someone online, and I said to him " I think you're trying to throw me off your scent here... You are cornered, and trying to confuse the issue by claiming I'm arguing for this idea, and you've been arguing against it. "

The truth is, that I had not cornered him. The reason I couldn't corner him was in order to corner someone in a rational debate, they have to allow themselves to become cornered... That is, they have to explain themselves so well, what their thoughts and ideas actually are that they are "nailed down".

If their ideas are valid, from a scientific perspective, then this position of being cornered and nailed down, is actually the position of ultimate strength. It is exactly the opposite of where you would want to be in a physical fight.

What is often done, in an argument with a rationalist, who is trying to be cornered and nailed down, so that he can argue from his position of greatest strength, is that the attacker, knowing that he cannot actually attack the argument being defended, must create a strawman argument.

While this does nothing to the beliefs of the person defending the argument, it does confuse other participants or bystanders in the argument, and may succeed in impugning the integrity of a person defending a true idea... e.g., put him in an illusory box that gives others an excuse not to listen to him.

So, as a rule, one can be more successful in seeking truth, by looking for the participant in the debate who is defending and explaining their own ideas, with an honest desire to be cornered and nailed down. They will not feint, or dodge, but will stand where they are, defending their ground, until such time as they are exposed to a better idea.

-"Deliberately we decide that we want to seek only the truth; but our brains have hardwired support for rationalizing falsehoods."

Deciding that you want to seek only the truth will not give you the truth. This is because, as you say, our brains have hardwired support for rationalizing falsehoods. What I have found to be a better strategy is self-cooperation. Your mind makes its existence known on several different levels like the vocal, the subvocal but still audible, the purely silent but still internally audible, and eventually laughter and similar responses. Each of these levels has different information about the world, and may be willing to share it with the higher levels as long as it does not end up feeling betrayed. But a naive attitude of "I want the truth!" is exactly what is likely to lead to such a betrayal. To put it another way, if a friend told you a secret and then you immediately went to tell that secret to someone else, would they be likely to continue to trust you after that? Even if you said that you were sharing the secret in the name of "truth"? Self-trust works the same way.

Sounds great. Loving this so far.

I'd recommend to others to keep philosophy of science in mind. Philosophy of biology doesn't have the nicest things to say about evolutionary psychology (at least relative to other scientific disciplines). It's not about throwing evopsych out, it's about understanding its limitations in informing us about human nature.

Also, keep in mind an interesting truth I've noticed: you might feel in some cases that you're "in the truth." But that itself is qualitatively a culture like any other. If you justify a hierarchy based on, say, evopsych, you're not "in the truth", you're in yet another a culture that justifies its inequalities through a story (even if that story is scientific and truthful).

Edit: Adding the Stanford Encyclopedia of Philosophy entry on evolutionary psychology: https://plato.stanford.edu/entries/evolutionary-psychology/

Some of the machinery is optimized for evolutionary selection pressures that run directly counter to our declared goals in using it. Deliberately we decide that we want to seek only the truth; but our brains have hardwired support for rationalizing falsehoods. We can try to compensate for what we choose to regard as flaws of the machinery; but we can’t actually rewire the neural circuitry. Nor may martial artists plate titanium over their bones—not today, at any rate.

While the human brain has evolved for survival, its structure can limit our abstract reasoning abilities. One approach to overcoming these limitations is to differentiate between the learning and practicing phases when acquiring a new skill. Contrary to the common belief that "practice makes perfect," it is essential to understand that practice actually makes skills permanent, not necessarily perfect. By acknowledging this distinction, individuals can focus on perfecting their skills through learning processes, such as the Kolb cycle, before committing to practice for long-term retention and improvement.

The Kolb cycle, a well-established model in experiential learning, offers a four-stage process that includes concrete experience, reflective observation, abstract conceptualization, and active experimentation. By applying this process in the context of reasoning, we can iteratively refine our thought processes and become more adept at bypassing the evolutionary constraints of the brain.

I think that you can have a school for rationality, but the people who stick around should be those who achieve new and difficult accomplishments, and the default is that you do not advance or continue without accomplishing something difficult and great.

I suspect it is easier to teach rationality than to do rationality because it is easier to conceive of an ideal than it is to live it. A great athlete works many hours a day with focus and interest. A coach merely watches and pokes them when they're going off-course, or says some motivating words.

I would start a school by forming a set of teachers, but we would be the certifiers, not the certified. Coaches. Rationality coaches who help you in your journey, not rationality teachers who you aspire to be.

Similarly, if you have a brain, with cortical and subcortical areas in the appropriate places, you might be able to learn to use it properly. If you’re a fast learner, you might learn faster—but the art of rationality isn’t about that; it’s about training brain machinery we all have in common. And where there are systematic errors human brains tend to make—like an insensitivity to scope—rationality is about fixing those mistakes, or finding work-arounds.

This is warmly egalitarian and quite welcoming to a wide range of intellects.  

Very recently—in just the last few decades—the human species has acquired a great deal of new knowledge about human rationality.

Perhaps this is an assumption? Schools of thoughts and philosophies have existed since time immemorial to address skillsets which would require an individual to live in balance and harmony with environment and themselves (which would conceptually be similar to focus on lifestyle rather on organization challenges, which the author talks about in preface of this chapter). With this as a North Star, it would be easy to identify skill set by just emotional, mental and physical wellbeing and measurable health (as opposed to the hypothesis author proposes that it is difficult to verify skill in previous paragraph). Perhaps what the author tries to address is - 'in our current epoch defined by industrial era policies and control, brain muscles have been eroded due to deep level automations and lack of training and education?    

Egs. Patanjalis Yoga Sutras 3000 year foundational yoga text - 1.2 "Yog chitta vritti niroda" or "Praxis is the cessation of biases of the mind" 
Confucious's Analects addresses the same challenges, in a more detailed description.