A few years ago I had a different blog than my current one and I wrote two series of posts there. I no longer think they are especially good, but thanks to my recent linking of the second series in comments on a discussion here on LW, I have gotten feedback that they were useful to a couple people. So in the spirit of making this content a little more accessible, easily findable, and linkable, I'm reposting them here with minimal editing. I'm grouping them together in two posts to cover the two series of posts I made.
This first series is about LW rationality basics largely covered in the sequences, but I wanted to express those ideas in my own words. It was mostly useful for me to do that work of putting my understanding into words, and I'm not sure how much reading the series in this post is worthwhile to others, but the second series references it, so seems worth having it here for ease of access. The second series is probably more valuable, and I'll link it here once it's up. ETA: It's up.
let's start at the beginning. both my beginnings with rationality and the first place you should start to find the winning way.
by now the metaphor seems well worn to me, but i have to remember that there was a time when i didn't know about it. a time when i could get confused about the correspondence between my own thoughts and reality. and even a time when i had the words to separate the two but could still get things confused. so let me try to ease you in to our first attempt to break and reintegrate your thinking: the metaphor of the map and the territory.
a wise king commands a map of perfect detail be made of his kingdom. the map makers begin the project. animals in their own and neighboring kingdoms are slaughtered to make the parchment. forests are cleared to collect all their resin and giant holes are left in the ground from extracting graphite to make the ink. the army of map makers works furiously day and night until at last they tell the king the project is finished. in the morning, they say, the map shall be revealed to him.
when the king wakes up he notices it's unusually dark. looking out the royal window, he sees that the sky has turned a yellowish tan and the light is blocked out. he stares at the sky, examining the lines that seem to have replaced the sun and clouds. slowly realization dawns on him: the sky has been replaced by a map!
the king runs downstairs, still in his nightgown, calling for his map makers. the king throws open the doors of the throne room to find them gathered, nodding and smiling to themselves.
"what is the meaning of this?!?" he bellows.
"why, sire," the head map maker says, "we have completed the project as requested. you need now only look up to see a perfectly accurate map of the kingdom in the sky."
it's easy, in hindsight, to see the king's confusion. in asking for a perfectly accurate map, he got the only thing that could be perfectly accurate: an exact reflection of the kingdom. in fact, in this story, i'm disappointed the map makers didn't manage to achieve greater accuracy by not constructing a map at all and just declaring the kingdom to be the map of itself. to literally make the map the territory.
when we talk of maps what we really want are smaller versions of reality that we can carry in our pockets. or, less metaphorically speaking, thoughts we can use to model the world.
this is one of the things folks mean when they say that the map is not the territory: the map serves a different purpose than the territory, and it would be impractical to use the territory as a map. or, to again the break down the metaphor, models are useful because they are thoughts we use to reason about the state of the world. there can be no such thing as a perfect model: all models contain error and uncertainty. you cannot be 100% or 0% sure of something. probability is subjective; beliefs contain uncertainty.
inherent in the map and territory analogy is a certain conception of what 'truth' means. it relies on what some might call the simple truth or the naive perspective on reality. its points are this:
another way to say this is: the truth is out there, and it's your job to map it.
this is worth pointing out because for the last century there's been a trend in western thinking to conflate the map with the territory. it pops up in theories like "truth is relative" and "reality is made by consensus". sure enough, since maps exist they are part of the territory, but this does not mean the map is the territory. maps can be built collaboratively, and we all have our own maps, but this doesn't make truth collaborative or relative: it makes map making a shared task and maps subjective.
the following short story may help make this clearer. my goal is to cause, if it has not already happened for you, a shift in the way you understand your relationship with reality. it happened for me when i read cfai section 3.1. i hope it will or already has happened for you, because fully understanding the relationship between the map and the territory is essential to making progress in the winning way.
For generations, the Shepherd had been tasked with keeping track of the village’s flock. Every day he let the sheep out to graze and then recalled them all at night to the safety of the fence. The duty passed from father to son, requiring years of training to be able to remember all the sheep and notice if one went missing.
The last Shepherd, known to his friends as Artzain, was especially skilled. He could recognize all his sheep and knew when one was missing. He never failed to notice a missing sheep, even if he couldn’t always find the missing ones before night fell and the wolves preyed on them.
Artzain’s son, Umalusi, seemed destined to lose much of the flock. Although Artzain spent many hours every day teaching Umalusi how to recognize the sheep, to tell them apart by subtle differences, and notice if one was missing, Umalusi could no better tell apart the sheep than any other villager.
Disheartened, Artzain stopped actively teaching Umalusi. He committed himself to the idea that he would have to live long enough to train his grandson, lest Umalusi lose all the sheep and the village starve.
Left to himself, Umalusi spent his time playing games and making up new, ever more creative games to occupy his time. This seemed not very useful, but since the only job available to Umalusi was Shepherd, and Artzain had no intention of letting him mind the flock, everyone left him alone with his games.
As Umalusi entered his 16th year, his father fell suddenly ill. Artzain was unable to stand, and so could not mind the flock. He had no choice but to send out Umalusi to act as Shepherd.
When he came home on the first night, Artzain asked Umalusi if he had lost any sheep.
“No, father. They all returned to the fence,” said Umalusi.
Artzain sighed. To the untrained eye it probably did look like they had all returned, but in his heart he knew that his son had probably lost a few sheep to the wolves.
The next day he was still too weak, so Artzain sent Umalusi in his place again. And again, Umalusi came home and reported having lost no sheep, and again Artzain didn’t really believe him.
After three more days of Umalusi watching the sheep and claiming that none had been lost, Artzain regained his strength and was able to shepherd again. On his first morning back with his flock, he was astonished to find that all of the sheep were there. He hollered for Umalusi, who came running thinking something terrible had happened to his father.
“What’s wrong?” said Umalusi.
“No sheep are missing,” said Artzain. “How?”
“Stones.”
“What do you mean ‘stones’?”“You have already let the sheep out for today, so I can’t show you, but let me mind the sheep tomorrow and I’ll show you.”
The next morning, Artzain and Umalusi went to the sheep enclosure together as they had not done since Umalusi was a child. Artzain extended his arm to tell Umalusi to proceed, and Umalusi walked over to the gate.
There, he set down two large clay pots. One was filled with river pebbles, and the other was empty. He opened the gate just wide enough that one sheep could make it through at a time. As the sheep moved out of the paddock and into the pasture, he began moving stones from one clay pot to the other.
“What are you doing, my boy,” said Artzain.
“Counting sheep,” said Umalusi.
“What is counting?”
“It’s a game I made up for tracking things. See, when a sheep leaves the paddock, I move a stone from the left pot into the right.”
Umalusi paused to move some stones from the left pot to the right because some sheep had gone out of the paddock while he was talking.
“I know you like games, son, but what has that got to do with sheep?” asked Artzain.
“Imagine that each sheep is a stone. The left pot is the paddock and the right pot is the pasture. I use the stones to know where the sheep are,” said Umalusi.
“Ah, I see,” said Artzain, “you have secretly been studying magic from Shaman while I have been out with the sheep all day. You have learned some spell to control the sheep with the stones.”
“Not at all,” said Umalusi, plopping a few more stones from the left pot to the right. “I know nothing of magic.”
“Then how can you be controlling the sheep with the stones?”
“I’m not. I move the stones when the sheep move.”
Artzain chuckled heartily. “I see. So you expect me to believe, then, that the sheep have put a spell on you, and that they are making you move the stones rather than the other way around!”
“In a way, that’s true,” said Umalusi. “But the sheep have no magical power over me. I move the stones to match where the sheep are. I only win if there is one stone for each sheep in the correct pot”
“Fine, but how do you know which stone belongs to which sheep? Now you have to be able to tell apart all the sheep and all the stones.”
“Nope. To me, all the sheep look the same. All the stones look the same, too. So it doesn’t matter to me which sheep or which stone I move, so long as there are as many stones as there are sheep.”
“But no two sheep are the same,” said Artzain.
“And no two stones are the same,” said Umalusi.“
So how can you possibly say it doesn’t matter so long as you have the right number of stones in the right pots?”
“Because although all sheep are different and all stones are different, I can treat them all the same in my game. So to me one sheep in the paddock is the same as any other sheep in the paddock, so one stone in the left pot is the same as any other stone in the left pot.”
Artzain stood silent and thought for a few minutes. Finally he said, “I think I understand, but to me it seems strange to ignore details in order to be able to do something you couldn’t do with them. Yet I can’t deny the fact that you’ve managed not to lose any sheep, even though you can’t tell them apart, by doing this.”
“I admit, I also found it a bit odd at first, and if I’m honest I’ve been secretly testing this method out from a distance while you watched the flock. When you got sick, it was my first chance to try it for real.”
“Well, I’m glad you succeeded. Now I can retire. You are now Shepard.”
“Thanks, but I’m not sure I want to stay Shepard. I have bigger games to play.”
when i talk about the map and the territory, i talk almost exclusively about the map, and when i do mention the territory it's as some undifferentiated soup that exists 'out there'. turns about, that's because we literally cannot talk about anything other than maps.
there is a famous painting of a pipe with the caption "this is not a pipe". on the surface this may seem odd: you are most definitely looking at a pipe. but really the thing you see is not a pipe, but rather a representation of one. that is to say, a map.
let that sink in a bit. we say 'the map is not the territory', but here we are being quite explicit about it. the map of a thing is not the thing itself.
once you understand this, you are led to the inevitable conclusion that you have never directly known reality in your life.
since pipes are relatively rare now, consider the humble spoon. you may object that you've seen spoons in real life, held them in your hand, maybe even used them to eat. surely that was the territory. but consider, how did you even know that thing was a spoon, or even a thing separate from other things? how could you tell that some small part of the universe was distinct from the rest of it in a way that you might call that part of it a spoon? because you have a map that allows you to literally make sense of things: to tell stuff in the universe apart.
to put it another way, there is no spoon until you create it. as far as the universe is concerned (and it's not concerned with anything at all in such a fundamental way you can't even really say the universe is or is not concerned), spoons do not exist. the territory is just stuff that has particular properties. although the stuff you will soon call a spoon exists independent of your understanding of it, it does not really become a spoon until it is mapped into one. the map makes the meaning, and it is only in the map that the spoon appears out of reality's confounded state.
in fact, there is no territory without the map, because the separation of map and territory is itself part of the map. without the map, there would be nothing to call territory in contrast to it: it would simply be everything, with all its complexities and yet none of its differentiation. it is the map that makes the meaning because the territory is confused. not in the usual sense of 'confused', but in the literal sense based on it's etymology: all mixed together as to be indistinguishable. the territory is confused, and you will be confused without the map to help you make meaning of it.
if you have ever used a geographical map to help you navigate roads or the wilds or the ocean, you have inevitably encountered disagreement between the map and the territory you are in: the road you are looking for isn't there, a boulder is blocking your way, or the water is much shallower than reported. in these ways maps can be wrong, and yet we wouldn't say the map is all wrong: if nothing else it was right enough that you made it as far as finding the map's errors. when we say 'the map is wrong' what we really mean is 'it has errors'. unsurprisingly, this is also true of our metaphorical maps of the territory of reality.
because the map is not the territory, it must contain errors. this is supported by our knowledge of thermodynamics and its mathematical abstraction, information theory, with assistance from special relativity to prevent anything happening instantaneously. and don't try any quantum trickery: at most it just shows we can move bits around before creating them. so if we cannot have an error-free map, we should consider how to live with those errors.
although i like to talk about maps, in everyday language people talk about beliefs, i.e. thoughts about the state of the world. and the naive model of belief common in the occident is that beliefs are either true (perfectly predict reality) or false. this leads some folks to talk of rejecting all beliefs, especially because 'belief' is often a word associated with religion. while this position is understandable given a true or false model of belief, from a technical standpoint, belief is the right term, so what we need is a better theory of belief.
that is to say, we need a more useful map of the map.
the best model i know of is to treat beliefs as subjective and probabilistic. if you think of a belief as a point on the map, each belief is made up of at least two parts: a statement about reality and a likelihood of correctness. this expands the notion of correctness assumed by the naive model of belief by adding a mechanism for assessing the maybe-ness of a belief. with this richer model of belief, we can understand the map as something that may be mostly right, but not in the sense of mostly made up of true things, but in the sense of mostly made up of things that are mostly likely to be true. the burden of truth shifts from the map to the territory, where it belongs, and leaves the map with the responsibility of assessing its likelihood of reflecting reality.
all maps contain errors, but since error cannot be eliminated, the next best thing is to reduce the magnitude of our errors, and in so doing, perhaps also the count. now that we know what the map is and what it is for, we are ready to consider the process of map making.
the key techniques for improving your map making skills over your natural abilities are well known and studied, but they are also scattered and difficult to implement. the map making skills of your ancestors beget your natural mapping skills, but they only had to be good enough map makers to produce you. that still makes them excellent map makers, ergo also you, but it does not imply you have reached the limits of your map making abilities. sure, you can already make maps that are good enough, but our goal is not a good enough map, but the best map we can make!
since the skills of better map making are already well explored by others, let's just take a tour (with links) of what you should know so that you can start your journey to better map making.
the first thing is to have a good map of the map. this cannot be underestimated, because your map of the map constrains your thoughts about the map, and if you want to make better maps, you need a strong process for reasoning about the map that allows you to improve upon it. without a good model of the map many map making techniques, commonly referred to as critical thinking and rationality, will seem confusing. this doesn't mean you can't start learning techniques before you have your model of the map figured out: it just means you will face confusion and difficulty as you figure out how to reason about the map in a way that makes some of the most powerful techniques useful. that's why i like to write more about the map and less about map making.
so assuming you have a good enough map of the map, let's highlight the core knowledge and techniques humans have accumulated to help them make better maps.
of course this list is incomplete. map making is a skill that requires learning a million little things and then putting them all into play at once largely without rising to the level of consciousness (or forcing such a rise when appropriate). and in truth it is a skill you never stop working on because, if nothing else, constant vigilance is required to avoid slipping back into easier, natural methods of map making. we'll return frequently to the topic of map making, but for now let this serve as a start on your journey to make better maps.