Possibly because Katas aren't a very good framework for practice? Most functional martial arts that work in e.g. the UFC will suggest at the very least practicing with a partner, and gradually working up to live resistance. Martials arts that emphasize katas over live resistance tend to not be great at self-defense.
My guess is there's something similar going on with rationality. Any sort of kata that doesn't have a heavy component of "gradually working up to interacting with the real world" probably won't be very effective.
While it's true that live resistance is an essential component, it's important to note that effective martial arts do katas as well under different names. In wrestling, it's "drilling". In boxing, it's hitting the bag or pads.
I actually feel like a bunch of forecasting stuff got surprisingly close to being katas for rationality. Like "predicting one Metaculus question a day" seems like a good kata to me, though I might be misunderstanding the concept.
A kata contains a bunch of individual moves that are usually done in the same order every time and practiced together.
The idea is by practicing them that way the individual moves will go into muscle memory and executed in a fast way.
Predicting a metaculus question would be a general task that can be accomplished in a lot of different ways.
This is very true, but I think it misses a key point in what makes katas useful for actually learning a martial art in the first place. As noted in Matt Goldenberg's answer, partner work is much more important for actually learning to use a martial art. Just practicing a kata by rote may look pretty, but it won't tell you anything about how to use it. My own best teachers would teach moves and combinations and katas by rote at first, then very quickly move on to exercises that require creative application. Things like:
All of these can be modified with constraints to make them easier or harder. Easier might be "Every attack against you will be this kind of punch." Harder can be something like "Choose one part of one kata. Come up with a way to use it effectively against whatever your opponent decides to throw at you." Or "Figure out X different ways to use that same sequence of moves, with only minor variations, in different situations, then execute it and see how well it works." Hopefully you've be...
Second this. This is where the katas are.
Also seconding Matt Goldenberg's point on katas. A kata is akin to learning by rote memorization for an academic test instead of learning to approximate causal models on the fly for use in practical projects.
I think it's an issue of "inside the box" vs "open-ended" fields, that we don't have really good vocabulary to talk about. 'Katas' work great for sports that are very much inside the box. You can innovate new strategies, but the rules of the sport set up an unchanging microworld that you must stay inside of. Coincidentally, these are also areas where even current-day AIs often dominate. Established scientific disciplines with research programs are sort of half and half. You can train people in them, but they can also benefit from serious paradigm shifts and there aren't any a priori hard and fast rules about things that absolutely can't be done, like the rules of chess for the chess-playing domain.
Then there's proto-science when things haven't coalesced into a discipline yet, philosophy when it hasn't been professionalized to death, Kegan's stage 5. This is raw pattern matching, flashes of insight, original seeing, very open ended exploration of an unknown landscape. I don't think anyone has had much of an idea for how to systematically train people for this. This place is also where a lot of the actually efficient rationality practice lives.
I did wind up with some personal katas.
Calibration using search: anytime I am searching for something with a quantitative answer I have the chance to do a fermi estimate/reference class forecasting and getting feedback on how I did.
Selection effects/Straussian readings: trying to figure out what incentives drove a particular piece of information to be in front of me in this moment.
Stack trace: finding the provenance of internal maps and noticing that they are often predicated on extremely sparse data which is then overgeneralized.
Schematic thinking: An extension of narrative fallacy bias. Noticing when alternatives would be equally valid when replacing parts of arguments. The implied degrees of freedom make the proposed explanation weaker than it might otherwise seem.
Calibration using search: anytime I am searching for something with a quantitative answer I have the chance to do a fermi estimate/reference class forecasting and getting feedback on how I did.
Do you have an explicit process for this?
Why aren't there katas for diet? Because diet is about not caving to strong temptations inherent in human nature, and it's hard to practice not doing something. Maybe rationality is the same, but instead of eating bad foods, the temptation is allowing non-truth-tracking factors to influence your beliefs.
People like having superpowers and don't like obeying duties, so those who try to spread rationality are pressured to present it as a superpower instead of a duty.
Because diet is about not caving to strong temptations inherent in human nature, and it's hard to practice not doing something.
I could imagine an exercise consisting of walking amidst hundreds of delicious cakes, where you know that if you start eating them, no one will stop you...
A more cruel exercise would be having dozens of unknown meals (for example, spheres of blended matter, colored by random food colors) that you are supposed to taste, eat the healthy ones, and spit out the unhealthy ones. (To make it simple, the healthy ones are vegetables, dairy,...
Math problems are like "katas" for rationality. The difference is that, once you've solved a problem once with rationality, you can solve it again much more easily from memory without engaging your rational facilities again. Therefore you don't get the benefit from repeating the same exercises again and again.
0. You assume there's one optimal solution, and that it has already been found, i.e. no room for improvement, and that nothing changes that would affect the solution.
1. You can change the math problem. (Coming up with or finding novel new problems is also useful, but with work, an old problem can be extended. You've heard of the Monty Hall problem, but how does it change if you have more doors, and more time with doors being opened? If another person was playing a similar game, given the option, would you gain in expected value if switched with them?)
2. Yo...
I do sudokus. These are computer-generated, and of consistent difficulty. so I can't solve them from memory. Perhaps something similar could be done for math or logic problems, or story problems where cognitive biases work against the solutions.
I wonder, though - maybe there are some rational skills that do benefit from repetitive practice? Overcoming bias comes to mind - even after you recognize the bias, sometimes it still takes mental energy to resist its temptation. Maybe katas could help there?
John Wentworth and asjayan’s framing exercises seem to fit the bill, but I’m the only one consistently participating as of late…
It's my impression that John regularly provides different exercises and that there's no continues practice of the same task.
It's my impression that various rationalist groups formed what they called dojos with the intend of having a structure that's similar to a martial arts dojo.
One of the key aspects of that institution is the practice of katas. Katas are a series of moves that can be practiced together to learn a particular art.
As far as I can see we didn't find anything that fits into that spot.
Why didn't we? Is it because rationality inherently can't be broken down into that form? Is it because we approached the problem wrongly?