Overall enjoying this series and your take on CFAR-style rationality. Thanks for putting in the time to write this up.
Thanks! I mentioned on day 0 that this was one of my training regimes for rationality, but it's also training writing and posting thinking in public places, among other things. I'm glad that people are able to get something out of it.
Another example of boggling: this Quora post by Oliver Emberton.
Like millions of people I was carried to work today in a comfortable metal box by the controlled explosion of 60 million year old dinosaur juice. (You call that petrol).
I avoided unexpected traffic on my way thanks to flying machines orbiting the earth, which talked to a metal and glass supercomputer in my pocket smaller than a bar of soap. (You call that a phone).
My pocket supercomputer is – of course – wirelessly connected to the entirety of humanity’s knowledge. The entirety of humanity’s knowledge is – of course – free. And I can search all of it as fast as I can type.
None of this is even interesting to anyone anymore.
As part of exercise #2, I hereby record my pledge to carry out this training regime, by completing one article each day (including the exercises) over the next ~ 30 days. Furthermore, I plan to write a comment detailing my experiences after completing the whole regime.
(There's some relation to the sunk cost fallacy here, in the sense that theoretically you should search equally hard for understanding after you've already paid no matter how much it costed. However, human brains don't actually work like that, so I think that this extension of the concept is warranted.)
I've seen this argument, and while I acknowledge it might be true for some people, I have no reason to believe that this isn't mistaken correlation* - if you pay more for something you probably care more about it. (Though the Ikea effect seems plausible, I could see that being a) the same kind of correlation, and b) if you make something then you probably make it the way you want it.)
*Or advertising. "Our teaching program[1] sets the price high so you will learn a lot!"
Have skin in the game
There are 'teaching programs'[1] that have people pay afterward (if they get good enough results).
I'm not quite sure how to have skin in the game with respect to a sequence of blog posts, but it seems important enough to try. Some possible ways:
Here are two different ways of reading that - your readers committing to do the exercises, and you committing to publish (in a specific way). Both offer insights, and information about what you're using might be informative. At a guess, you write one every day and publish it or you wrote the whole thing in advance or you have a buffer (that's smaller than the whole thing).
Between those two methods, one seems obvious for readers (right now) - don't build up a buffer, do each one as it is released. (This is easy to do with a regular release schedule.)
(I claim to teach techniques that work for me. At CFAR, they teach rationality techniques that work for nobody.)
That is an...interesting approach.
The ability to stop doing bad things means that trying things has almost no cost and a very high benefit (with a few notable exceptions).
Dangerous activities or addiction?
Unless you're going to cease having agency soon, you should probably spend much more of your time exploring than you currently do.
Building form sounds similar to building habits.
The knowledge that the researcher didn't have before seeing the rose and did have after seeing the rose is what I am referring to as the "phenomenology" of redness with respect to that researcher.
I thought it was experience/qualia, but I'm not too invested in those words.
If I ever give an example and you’re like “well I don’t think that would be good for me”, then remember that rationality is hair style agnostic.
This could have been more strongly/explicitly stated and also seems related to Should you reverse any advice you hear?[1]
It is important to remember that rationality techniques are not supposed to be weird.
They're supposed to be real. They're supposed to work. (If you put stock in a theory and it comes up with a weird answer, see if there's a cheap experiment, or look at why that answer seems weird - general theories require may require more evidence, but a more complete understanding of why one thing works here but doesn't over there, is valuable if correct (which should be carefully established - conflict between theories highlights an area to look closer at).)
If you use a rationality technique properly, the thing that comes out of it should make sense.
To reuse an earlier example:
You try doing your hair for 30 minutes and find you enjoy this, but it doesn't makes sense.
In meditating, the point is maybe something we call "enlightenment" that can only be understood by people who have seen it.
Activities where we're not sure what the point is are an interesting class.
Being able to sit still for a very long time gets you closer to the point of meditating (might be wrong about this one).
Can meditation be done while moving?
Exercise
While the prior post was similarly abstract in subject matter, this post focused on presenting several things and was less detailed in steps.
Footnotes
[1] This is later referenced in the post under a different name.
When you're trying to learn martial arts, usually you go to a physical location and get instructed by a real live person in real time. However, there are also books on martial arts. It is unknown to me whether or not anyone can actually learn anything useful about martial arts just by reading a book, even if they practice everything they have read.
When you're learning something in meat space, the information is transferred from one mind into words and then into another mind. The other mind can confirm or deny understanding, paraphrase the content, ask for examples, etc. The first mind can then translate different parts of their knowledge into words, translate the knowledge in a different way, confirm or deny the correctness of the paraphrasing, etc. This feedback loop works to minimize the distortion of information as it becomes compressed into this insufficiency we know as language.
In contrast, when you're learning something from written works, knowledge is translated into fixed and static text, which is then absorbed by another mind. The other mind attempts to reverse-translate what they have read into some form of knowledge. If the static text contains instructions, the other mind will attempt to divine the intent behind the written instructions and perform the tasks asked of them. It is unknown to me how much distortion occurs in the absence of a tight feedback loop that limits towards proper understanding. It is my hope that the distortion is minimal enough that reading my words and trying to do my exercises causes slight positive benefits instead of active harm.
Even so, there are a number of tips and tricks that can be used when approaching this material that will make it more probable that you will benefit from it.
When I'm learning something in a traditional classroom setting, sometimes I find myself in mechanical student mode, where I just write everything down that the teacher says without even thinking about it at all. For example, the teacher will say something like "the Earth goes around the Sun" and then I'll copy down "the Earth goes around the Sun." I don't think much of this phrase because I have a vague idea of what "Earth", "Sun", and "around" mean and it seems not entirely incongruous to put them together.
It seems to me that this mode of thought contains a profound lack of something. We might call this something curiosity, a need for understanding, a desire for a complete picture, etc. To boggle at something is to explicitly discourage mechanical student mode and explicitly encourage a view where nothing quite makes enough sense and you feel like you're lost in a web of infinite knowledge.
Example 1: The teacher said that the Earth goes "around" the Sun. I thought that there was no such thing as an absolute reference frame, so what does it even mean to say that the Earth goes around something? Doesn't it mean the exact same thing to say the the Sun goes around the Earth? And what the heck is the Earth/Sun anyway?
Example 2: I've heard of these things called "post-its." They're these tiny square pieces of paper that contain a sticky part on one of the edges. You can stick them to stuff, but then you can take them off again? Like it's sticky enough to stick, but not sticky enough to not be removed? How did anyone even make this thing? And it's basically a perfect square? And there are like 100 of them stacked perfectly on top of each other? I know a tiny bit about how paper is manufactured and I think it involves something like cutting it so it's even on the side, but does that mean that there's a knife sharp enough to cut through like 100 pieces of paper at the same time? What?
Some ways to induce boggling include the phrases "whatever that means" and "somehow." The Earth(whatever that is) goes around (whatever that means) the Sun (whatever that is). This post-it (whatever that is) somehow sticks (whatever that means) to a surface (whatever that is) in a way that means you can take it off again (whatever that means).
As a side note, I claim that if you think of boggling as an instrumental move that aids understanding, you won’t get as much out of it. This is similar to how if you only want to learn some skill because you think it will make you money, you will probably end up worse at that skill and make less money than someone who was intrinsically fascinated with that skill. I’m not sure how much you can avoid boggling instrumentally just by thinking about it though.
When trying to learn something (whatever that means) I recommend that you avoid importing large amounts of past knowledge into the making of your new understanding.
One of the questions that people might ask is "why does CFAR cost so much?" The obvious answer is something like "they have operating costs and want to be able to provide financial aid for people." To my best guess, this answer is true. However, I think that, even if CFAR didn't have operating costs, charging money for the workshops is still a good idea.
An actor has "skin in the game" with respect to a certain outcome if that actor has material investment in the outcome. Traditionally, this is thought of in terms of betting, i.e. I have skin in the game of a horse race if I've bet on <generic horse name> to be the winner.
Extending this concept slightly, I think that when you pay for things, you have skin in the game with respect to getting value out of them. At CFAR, since I had paid a reasonably large amount of money to be there, I found myself more willing to do things like ask questions of the instructors. Having to pay money to be there creates a mindset of me actively trying to get everything I can out of the workshop. I suspect that if it was free, I would have, in many cases, terminated my search for understanding prematurely.
(There's some relation to the sunk cost fallacy here, in the sense that theoretically you should search equally hard for understanding after you've already paid no matter how much it costed. However, human brains don't actually work like that, so I think that this extension of the concept is warranted.)
I'm not quite sure how to have skin in the game with respect to a sequence of blog posts, but it seems important enough to try. Some possible ways:
The US air force built a control panel for a new type of aircraft. They had a lot of data on what measurements human bodies tend to have, so they did the obvious thing and built the control panel for the average measurement. The crash rate for the new type of aircraft was much higher than the crash rate for the old type of aircraft. Everyone was confused. Engineers confirmed the planes worked. Pilots claimed that their peers that crashed were highly skilled. There seemed to be no flaws anywhere in the process.
The control panel was built for the average size pilot. How many pilots were actually average sized? On average, all pilots are average sized, so surely most of the pilots are average sized. You decided to actually count. It turned out that zero of the pilots were average sized. The control panel was built for no one. All pilots were unable to use the control panel properly in some way.
The control panel was made adjustable and the crash rate went back down. Building for the average pilot was fine when the aircrafts were simpler. When the aircraft was more complicated, mistakes were much more costly.
Applied to the domain of rationality techniques, the versions of the techniques I will describe are slightly modified versions of what CFAR taught to me. In practice, this means that what I describe is a technique that works for me, perhaps slightly modified from a technique that works for the people at CFAR. This does not mean that it works for you. Importantly, it might seem to work for you when the problem is simple, but it might start breaking when the problem is complicated.
(I claim to teach techniques that work for me. At CFAR, they teach rationality techniques that work for nobody.)
Whenever a technique seems to not be working, remember to adjust your seat.
When you're lifting weights at the gym, we speak of this thing called "form." In particular, when you start to do squats, there is a very precise form that avoids possible injury that involves proper knee, hip, and chest placement.
The first time you start squatting, you start with the empty bar. You squat it a few times, think it's pretty easy, then turn to your experienced gym-going friend and say "this is pretty easy, I think I can add some weight." Then they say "no, just squat the empty bar." It turns out that while the proper form for squatting is pretty easy with the empty bar, the more weight you add the harder it becomes. It is important that the form becomes natural and thoughtless before you attempt to add more weight, else you risk form deterioration and injury.
Similarly, when learning a new rationality technique for the first time, you shouldn't try it on your biggest problems first. Failing to apply a technique properly can have bad consequences, so it is recommended that you start with small/medium sized bugs to get a feel for the technique before trying to tackle the biggest problems in your life.
When you learn a new skill, practice it on small things first to build form.
Human beings have this remarkable power where they can stop doing things that they don't like doing. If you try dancing and find you don't like it, you can stop dancing. However, if you try dancing and find that you love it, you have just acquired a fun new activity to do. The ability to stop doing bad things means that trying things has almost no cost and a very high benefit (with a few notable exceptions).
In general, this is called the explore-exploit tradeoff, where exploring is trying things and exploiting is doing things that you know to be good. Unless you're going to cease having agency soon, you should probably spend much more of your time exploring than you currently do. If you are young, you should be spending the vast majority of your time exploring, so that you have a lot of knowledge about what to exploit when you grow older.
There are certain important exceptions. There are some activities that are self-perpetuating. Trying them will change your perception of that activity to become more positive. Addictive drugs fall into this category, but this category also includes systems of belief that discourage believing other things. These things you should probably not try without thinking very carefully about the possible consequences. There are also things that are Really Bad that trying them even once is extremely dangerous. You should not try Really Bad things.
Throughout this sequence, I might recommend that you do something that you don't really think is for you. It probably costs you very little to try it and you might discover a new thing to exploit in the future.
A long time in some direction in some thought experiment far away, there is a researcher locked in a black and white room. In this thought experiment, there is complete and total understanding of the processes that underlie "seeing." The researcher has complete understanding of this understanding; they know exactly what happens in the brain when someone sees the color red. One day, the researcher is let out of the room, sees a rose and says "oh, that's what red looks like."
The knowledge that the researcher didn't have before seeing the rose and did have after seeing the rose is what I am referring to as the "phenomenology" of redness with respect to that researcher. (I might be using the word wrong and it might be more proper to say that this is the “phenomenon” of redness, but bear with me.)
(You might not think that such a thing exists, in which case this tip is not meant for you)
There are many rationality techniques that involve identifying when you feel/experience certain sensations. For these techniques, it is useful to be able to roughly capture the phenomenology of the experience. Sort of by definition, it is difficult/impossible to capture phenomenology in words. However, I claim that sometimes metaphor/analogy are tools that can be used to guide your brain into finding phenomenology.
For example, sometimes I feel defensive. To me, the phenomenology of defensiveness is something like wanting to bite something coupled with snappiness. This might not make much sense to you (or it might make a lot of sense), but to me, these words invoke a pretty clear picture of what defensiveness feels like.
When you're trying to apply a technique to an internal experience, I suggest that you try to find the phenomenology of that experience.
One of the examples one of the instructors used for one of the techniques was realizing that they were spending too much time doing their hair. They said that the technique in question let them reclaim 30 minutes in the morning that they used to spend doing their hair. One of the participants (or maybe one of the instructors [CFAR is sneaky]) asked “but would it have been okay to decide that you should actually spend longer on your hair?” to which the instructor responded “CFAR is hair style agonistic.”
This was funny at the time, but I actually think that there is wisdom in this phrase. The point of applied rationality is to actually make your life better. If you find that your life improves by spending longer doing your hair, then go for it. If you find that doing something you used to think was “irrational” makes you happy, then you should do that thing. The point of learning applied rationality is not to import a set of behaviors/habits that “rational” people do, it’s to figure out strategies to generate your own set of rational actions.
In this vein, I will not give much object level advice. I will tell you to try things, but I won’t really tell you what particular things to try. Sometimes I will say things like “I think that in-general, people don’t try things outside their perceived social strata,” but it will never be meant as something like “All people should try X thing” because there probably exists someone that shouldn’t try X.
If I ever give an example and you’re like “well I don’t think that would be good for me”, then remember that rationality is hair style agnostic.
Rationality techniques are meant to help you achieve your goals. Sometimes, you have a lot of domain knowledge about particular aspects of how to achieve your goals. Sometimes, you will apply a rationality technique and it will give an answer that is radically incongruous with your domain knowledge.
In this scenario, what you should probably not do is say something like "Oh, I guess I was being irrational before. Good thing I have this rationality technique to tell me what to do" and then change your behavior. What you should probably do is say something like "Huh, this rationality technique gave me a weird answer. I probably used it wrong or maybe it wasn't really applicable in this scenario. I wonder why that happened" and mostly keep your behavior the same.
It is important to remember that rationality techniques are not supposed to be weird. Egan's law: it all adds up to normality. If you use a rationality technique properly, the thing that comes out of it should make sense.
Remember the law of equal and opposite advice: for every person who needs a certain piece of advice, there is someone else who needs the opposite advice. Similarly, for every person that finds any technique incredibly helpful, there is a person for whom following that technique will cause disaster.
In fencing, a lot of time is spent practicing lunging. The lunge is a basic attack that can quickly cover a lot of distance. If you only observed fencing practice, you might look at the distribution of activities and conclude something like "the point of fencing is to be good at lunging." However, in fencing, lunging is not the point.
In weightlifting, people often spend a lot of time trying to lift heavy things. If you observed people at the gym, you might conclude something like "the point of weightlifting is to be able to lift the heaviest thing." For some types of people, this might be correct. However, I claim that for most people, the point of weightlifting is not to be able to lift weights.
When meditating, people spend a lot of time sitting very still. If you observed people at a meditation retreat, you might conclude something like "the point of meditating is to be able to sit still for a long time." I do not meditate that much, but it seems very clear to me that the point of meditation is not to be able to sit still for a long time.
At CFAR, people spend a lot of time learning rationality techniques. If you observed what CFAR participants tend to spend their time at the workshop doing, you might conclude that "the point of CFAR is to teach people rationality techniques.” The techniques are not the point.
If the thing you observe people to be doing isn't the point, then what is? It's complicated. In fencing, the point is to win your bouts. In weightlifting, the point is something like getting stronger, increasing bone density, etc. In meditating, the point is maybe something we call "enlightenment" that can only be understood by people who have seen it. For applied rationality, the point might be described as winning, but is also properly described as "winning" in the sense that the point of all activities is to "achieve your values."
In fencing, lunging is not the point, but being good at lunging gets you closer to the point. Being able to lift heavy things gets you closer to the point of weightlifting. Being able to sit still for a very long time gets you closer to the point of meditating (might be wrong about this one). Being able to effectively use some subset of rationality techniques extremely well gets you closer to the point of applied rationality.
The techniques are not the point, but they're not not the point either.
Attempt micro-versions of all the tips applied to your daily life.