I’m not asking if you know how to use a toilet. I sure hope you know how to use a toilet. I’m just asking if you know how a toilet works.
I thought I did, at least until recently. After all, I use toilets all the time. I’ve even taken apart toilets myself, when the damn things stopped working, and sometimes I’d even manage to fix them. How could I not know how a toilet works? Then I read The Illusion of Explanatory Depth by the Decision Lab, and now I’m questioning everything I know.
Imagine this: a friendly alien comes to Earth and asks you how a toilet works. No, seriously, imagine it. What would you say to it? And how far would you get into your explanation before it starts to fall apart?
And yet, as the alien takes a seat to listen, you realize you can tell him about the button or the lever you press to flush the toilet, but you can’t explain much about what’s going on inside the toilet. How does the dirty water leave and the clean stuff arrive? What mechanisms or forces are working behind the scenes? And what about the little thing that rises and falls inside the… what’s it called… the bit at the back…
Perhaps you can partially answer one or two of these specific questions (or even more if you happen to be a plumber), but surely the alien will have even more questions you can’t answer. To think that a toilet is such a simple, everyday item, and yet you actually know much less than you’d predicted. This puzzles you greatly.
It’s true. I don’t know what my plumbing looks like, much less how to build it; I don’t even know the names of all of my toilet’s constituent parts, even those of the toilets I’ve personally taken apart. I guess I don’t really know how a toilet works.
What if that’s a good thing?
II.
I’ve been thinking about this idea for a couple months now, and I’m finally ready to write. Below are three thought-provoking excerpts from the posts that inspired it. (All emphasis mine.)
First, the aforementioned Decision Lab article. It’s a comprehensive explainer post that goes much deeper into the specifics of the illusion of explanatory depth (IOED). Here are the most relevant bits.
Your trouble explaining a toilet to the alien is because of the illusion of explanatory depth: sometimes, having to explain your knowledge brings you to realize how limited it is in reality. You could replace the example of the toilet with many other everyday items, such as locks, car engines, or light bulbs.
The illusion has two parts. First, the “explanatory” part refers to our belief that we can provide a clear, detailed explanation of how something works. The “depth” part reflects our assumption that the explanation will be thorough and complex enough to convey what we’re trying to explain. In reality, however, when we try to dig into the details, we often find that our explanatory knowledge is much shallower than we initially thought. What we actually end up with is the reality of explanatory shallowness.
Both parts are important to the illusion. That is, it’s only when we explain something that the gaps in our knowledge become apparent to us. Before that, we might still believe in our minds that we possess a depth of knowledge.
The illusion is far stronger for explanatory knowledge than other domains of knowledge, such as facts, procedures, or narratives. And it doesn’t apply if you know nothing about a topic and are happy to admit it. It only works when we inaccurately overestimate our understanding or knowledge of a certain concept or topic.
Second, Harari vs. Henrich by Joseph Heath. This excellent post chiefly focuses on The Secret of Our Success by Joseph Heinrich, and how its explanation of human development conflicts with (and arguably repudiates) that of Yuval Noah Harari in Sapiens. However, one paragraph in particular caught my eye—a paragraph about the benefits of mindless imitative copying.
Human culture is made possible by a particular style of social learning, and so the original adaptation – the “tweak” to the basic primate capacities – was the emergence of imitativeness. Human infants got really good, not just at copying what other people do, but at mindlessly copying others – repeating the exact behaviours that they observe. This created the possibility for more complex culture to evolve, somewhat ironically, by freeing it from the bottleneck imposed by individual intelligence. Chimpanzees learn from observing others, but they tend to view the behaviour as a source of inspiration, then use their own intelligence to reproduce the performance. They observe the wheel and then, in a sense, reinvent it for themselves. Human infants, by contrast, are disposed to copy others even when they don’t understand what they are seeing. This means that over time, something like a tool-making procedure can become more and more complex, as people make little improvements to it, yet the next generation of humans is still able to reproduce it, precisely because they do not need to figure out how it works in order to copy it.
Finally, we have the classic blog post, Reality has a surprising amount of detail by John Salvatier. “Reality has a surprising amount of detail” is another way of pointing at the IOED. My favorite part is his description of boiling water.
Consider the boiling of water. That’s straightforward, water boils at 100 °C, right?
Put yourself in the shoes of someone at the start of the 1800’s, with only a crude, unmarked mercury thermometer, trying to figure out the physics of temperature.
Go to your stove, put some water in a pot, start heating some water, and pay attention as it heats.
(I suggest actually doing this)
The first thing you’ll probably notice is a lot of small bubbles gathering on the surface of the pot. Is that boiling? The water’s not that hot yet; you can still even stick your finger in. Then the bubbles will appear faster and start rising, but they somehow seem ‘unboiling’. Then you’ll start to see little bubble storms in patches, and you start to hear a hissing noise. Is that Boiling? Sort of? It doesn’t really look like boiling. The bubble storms grow larger and start releasing bigger bubbles. Eventually the bubbles get big and the surface of the water grows turbulent as the bubbles begin to make it to the surface. Finally we seem to have reached real boiling. I guess this is the boiling point? That seems kind of weird, what were the things that happened earlier if not boiling.
To make matters worse, if you’d used a glass pot instead of a metal one, the water would boil at a higher temperature. If you cleaned the glass vessel with sulfuric acid, to remove any residue, you’d find that you can heat water substantially more before it boils and when it does boil it boils in little explosions of boiling and the temperature fluctuates unstably.
Worse still, if you trap a drop of water between two other liquids and heat it, you can raise the temperature to at least 300 °C with nothing happening. That kind of makes a mockery of the statement ‘water boils at 100 °C’.
It turns out that ‘boiling’ is a lot more complicated than you thought.
This surprising amount of detail is not limited to “human” or “complicated” domains, it is a near universal property of everything from space travel to sewing, to your internal experience of your own mind.
I love this example for two reasons.
First, John’s explanation was generally surprising to me. Not in its entirety; I’ve had a couple of these thoughts myself, and I do think something along the lines of “um, Is that Boiling?” every time I make pasta. But just like I never thought to find my toilet’s internal schematics on Google, I also never thought to test how container material affected boiling point; nor did I wonder if pot residue would affect the boiling process; nor did I even stop to consider whether putting a lid on a pot would actually affect the boiling rate—I kinda just assumed that it would prevent heat loss by trapping steam and thus be faster. I guess I don’t really know how water boils.
Second, none of this knowledge is necessary to boil water.
III.
Let’s think a little more about these two claims.
First, the process of boiling water obviously includes John’s entire description. If someone told me they understood how to boil water, I would expect them to know all of the above, AND I would expect them to know more—because of course, it would be a further mistake to assume that John has described everything that happens when water boils. In fact, John didn’t even mention the two main factors that determine the boiling point of water, which are atmospheric pressure (lower air pressure = lower boiling point) and dissolved impurities (more salt/minerals/other stuff = higher boiling point). If you thought you understood how water boils after reading John’s explanation alone, you’d be wrong.
I’m not exempt from this mistake. I knew about both of these factors from high school chemistry and physics, but the IOED is so strong that I forgot them after reading John’s explanation; I went out of my way to Google “factors that affect water boiling point” in order to remember those two factors, and almost fell for it again immediately after. There are many more molecular shenanigans surrounding water that I have either forgotten or never learned, like van der Waals interactions, or the Maxwell–Boltzmann distribution that describes particle speeds in ideal gases, or how the Maxwell-Boltzmann distribution only sort of applies to liquids (oops, my bad), and there are probably even more shenanigans that as of yet remain undiscovered.
But let’s not get bogged down in this uncomfortably warm pond. You don’t understand how water boils, and neither do I.
Second, boiling water is incredibly easy. All you have to do is grab a pot, put some water in it, and apply heat to the pot until the water starts to boil. Humans have been boiling water for thousands of years, possibly even as early as 30,000 years ago.
So here lies our paradox. You don’t understand how water boils, but you do understand how to boil water. And if you want to teach someone how to boil water, you don’t have to teach them how water boils.
IV.
This problem is not limited to laymen.
In 1894, Albert A. Michelson—the Michelson half of the Michelson–Morley experiment that struck down the luminiferous aether, also the first American to receive the Nobel Prize in the sciences—famously said:
While it is never safe to affirm that the future of Physical Science has no marvels in store even more astonishing than those of the past, it seems probable that most of the grand underlying principles have been firmly established and that further advances are to be sought chiefly in the rigorous application of these principles to all the phenomena which come under our notice. It is here that the science of measurement shows its importance — where quantitative work is more to be desired than qualitative work. An eminent physicist remarked that the future truths of physical science are to be looked for in the sixth place of decimals.
Michelson may have been one of Physics’ greatest minds, but he was dead wrong about this one. His illusion unraveled within his lifetime. 11 years later, Einstein would discover special relativity; 11 years after SR, he would discover general relativity. Both theories upended Physics as we know it.[1]
We even have a name for complex phenomena that are separate from their constituent parts, at least insofar that making predictions about the complex phenomenon in question does not require understanding its constituent parts. It’s called emergence. I think it’s pretty cool—I wrote a whole post about it—but for now, I will limit myself to this simple conclusion; the laws that govern reality, along with the heuristics we use to interpret them, are scale-dependent and fractal. “Seemingly-endless complexity” is reality’s favorite calling card. When it comes to invention, replication, or other processes that require you to actually build what you use, the IOED is a limiter on your imagination and must be dispelled.
But, when it comes to everyday life, the IOED is a blessing. Blind imitation isn’t a failure of intelligence, but our unique human superpower—because imitation is the engine of culture. And, as Joe Heath writes in his post, culture is ultimately the driving force behind prosocial behavior, and thus society as a whole.
The big idea – which I’m not going to be able to do justice here – is that conformist social learning increases homogeneity within social groups, thereby potentiating group selection as a force in human cultural evolution. The mechanism of between-group conflict, which is a weak force supporting cooperativeness in biological evolution, becomes a strong force favoring cooperativeness in cultural evolution. Groups with more prosocial behavioural norms are more likely to establish cultural dominance over others. Because of this, human cultures became more cooperative over time.
Apart from driving prosocial behavior, the IOED also allows us to use tools that we don’t really understand—which, in today’s day and age, is basically all of them. Imagine building a kettle every time you wanted to boil water, or assembling a toilet every time you needed to relieve yourself. And let’s not even talk about the word processor I’m using to write this article. Trying to truly and deeply understand everything you use would be impossibly overwhelming; if we didn’t possess the capacity for blind imitation, we’d have to reinvent the proverbial (or literal) wheel every time we wanted to use one, and our civilization would fall apart (or more likely never have spawned in the first place).
Thus, if you are to do anything at all, you must eventually accept that there is more going on than you will ever understand—AND you must choose to act in spite of that lack of understanding. Furthermore, you must choose to interact with emergent phenomena as if they are complete, and disregard the fact that you have no idea what is actually causing them.
This whole song and dance reminds me of Zeno’s old yarn about Achilles and the tortoise. When considered logically, it seems impossible that Achilles would ever be able to overtake his competitor—Zeno himself thought it was a mistake to suppose Achilles could run at all—but of course, if Achilles wants to win his race, he can simply blow past the lazy chelonian and plead his ignorance of infinity later. The same principle applies to the man who seeks to boil a pot of water.
V.
So our paradox of boiling water isn’t really a paradox. Of course we don’t understand how water boils. Of course we understand how to boil water. The illusion of explanatory depth, that strange gap between execution and explanation, is the very Secret of Our Success. And in a very real sense, it’s a huge part of what makes us human.
If you accept this to be true, there is only one mistake left for you to make, and it’s to believe that you do understand it all. And, as the wisest man in Athens once taught us, correcting this mistake is the simplest thing in the world. All you have to say is this: “I know that I know nothing.”
Interestingly, Michelson was not alone in his dismissal of new paradigms. 10 years after general relativity, the field of quantum mechanics was born, and Einstein himself became one of its harshest critics. At the time, this stance was pretty reasonable, but I can’t help but mention that his famous quote about how the universe is fundamentally deterministic—that “God plays dice”—has since been proven wrong.
what a harmful lie. they don't learn "do these actions", but rather "think these thoughts (which lead to these actions)". quite the opposite of mindless.
I promise this image is relevant.
I.
Reader, do you know how a toilet works?
I’m not asking if you know how to use a toilet. I sure hope you know how to use a toilet. I’m just asking if you know how a toilet works.
I thought I did, at least until recently. After all, I use toilets all the time. I’ve even taken apart toilets myself, when the damn things stopped working, and sometimes I’d even manage to fix them. How could I not know how a toilet works? Then I read The Illusion of Explanatory Depth by the Decision Lab, and now I’m questioning everything I know.
Imagine this: a friendly alien comes to Earth and asks you how a toilet works. No, seriously, imagine it. What would you say to it? And how far would you get into your explanation before it starts to fall apart?
It’s true. I don’t know what my plumbing looks like, much less how to build it; I don’t even know the names of all of my toilet’s constituent parts, even those of the toilets I’ve personally taken apart. I guess I don’t really know how a toilet works.
What if that’s a good thing?
II.
I’ve been thinking about this idea for a couple months now, and I’m finally ready to write. Below are three thought-provoking excerpts from the posts that inspired it. (All emphasis mine.)
First, the aforementioned Decision Lab article. It’s a comprehensive explainer post that goes much deeper into the specifics of the illusion of explanatory depth (IOED). Here are the most relevant bits.
Second, Harari vs. Henrich by Joseph Heath. This excellent post chiefly focuses on The Secret of Our Success by Joseph Heinrich, and how its explanation of human development conflicts with (and arguably repudiates) that of Yuval Noah Harari in Sapiens. However, one paragraph in particular caught my eye—a paragraph about the benefits of mindless imitative copying.
Finally, we have the classic blog post, Reality has a surprising amount of detail by John Salvatier. “Reality has a surprising amount of detail” is another way of pointing at the IOED. My favorite part is his description of boiling water.
I love this example for two reasons.
III.
Let’s think a little more about these two claims.
First, the process of boiling water obviously includes John’s entire description. If someone told me they understood how to boil water, I would expect them to know all of the above, AND I would expect them to know more—because of course, it would be a further mistake to assume that John has described everything that happens when water boils. In fact, John didn’t even mention the two main factors that determine the boiling point of water, which are atmospheric pressure (lower air pressure = lower boiling point) and dissolved impurities (more salt/minerals/other stuff = higher boiling point). If you thought you understood how water boils after reading John’s explanation alone, you’d be wrong.
I’m not exempt from this mistake. I knew about both of these factors from high school chemistry and physics, but the IOED is so strong that I forgot them after reading John’s explanation; I went out of my way to Google “factors that affect water boiling point” in order to remember those two factors, and almost fell for it again immediately after. There are many more molecular shenanigans surrounding water that I have either forgotten or never learned, like van der Waals interactions, or the Maxwell–Boltzmann distribution that describes particle speeds in ideal gases, or how the Maxwell-Boltzmann distribution only sort of applies to liquids (oops, my bad), and there are probably even more shenanigans that as of yet remain undiscovered.
But let’s not get bogged down in this uncomfortably warm pond. You don’t understand how water boils, and neither do I.
Second, boiling water is incredibly easy. All you have to do is grab a pot, put some water in it, and apply heat to the pot until the water starts to boil. Humans have been boiling water for thousands of years, possibly even as early as 30,000 years ago.
So here lies our paradox. You don’t understand how water boils, but you do understand how to boil water. And if you want to teach someone how to boil water, you don’t have to teach them how water boils.
IV.
This problem is not limited to laymen.
In 1894, Albert A. Michelson—the Michelson half of the Michelson–Morley experiment that struck down the luminiferous aether, also the first American to receive the Nobel Prize in the sciences—famously said:
Michelson may have been one of Physics’ greatest minds, but he was dead wrong about this one. His illusion unraveled within his lifetime. 11 years later, Einstein would discover special relativity; 11 years after SR, he would discover general relativity. Both theories upended Physics as we know it.[1]
We even have a name for complex phenomena that are separate from their constituent parts, at least insofar that making predictions about the complex phenomenon in question does not require understanding its constituent parts. It’s called emergence. I think it’s pretty cool—I wrote a whole post about it—but for now, I will limit myself to this simple conclusion; the laws that govern reality, along with the heuristics we use to interpret them, are scale-dependent and fractal. “Seemingly-endless complexity” is reality’s favorite calling card. When it comes to invention, replication, or other processes that require you to actually build what you use, the IOED is a limiter on your imagination and must be dispelled.
But, when it comes to everyday life, the IOED is a blessing. Blind imitation isn’t a failure of intelligence, but our unique human superpower—because imitation is the engine of culture. And, as Joe Heath writes in his post, culture is ultimately the driving force behind prosocial behavior, and thus society as a whole.
Apart from driving prosocial behavior, the IOED also allows us to use tools that we don’t really understand—which, in today’s day and age, is basically all of them. Imagine building a kettle every time you wanted to boil water, or assembling a toilet every time you needed to relieve yourself. And let’s not even talk about the word processor I’m using to write this article. Trying to truly and deeply understand everything you use would be impossibly overwhelming; if we didn’t possess the capacity for blind imitation, we’d have to reinvent the proverbial (or literal) wheel every time we wanted to use one, and our civilization would fall apart (or more likely never have spawned in the first place).
Thus, if you are to do anything at all, you must eventually accept that there is more going on than you will ever understand—AND you must choose to act in spite of that lack of understanding. Furthermore, you must choose to interact with emergent phenomena as if they are complete, and disregard the fact that you have no idea what is actually causing them.
This whole song and dance reminds me of Zeno’s old yarn about Achilles and the tortoise. When considered logically, it seems impossible that Achilles would ever be able to overtake his competitor—Zeno himself thought it was a mistake to suppose Achilles could run at all—but of course, if Achilles wants to win his race, he can simply blow past the lazy chelonian and plead his ignorance of infinity later. The same principle applies to the man who seeks to boil a pot of water.
V.
So our paradox of boiling water isn’t really a paradox. Of course we don’t understand how water boils. Of course we understand how to boil water. The illusion of explanatory depth, that strange gap between execution and explanation, is the very Secret of Our Success. And in a very real sense, it’s a huge part of what makes us human.
The mystery of life isn’t a problem to solve, but a reality to experience. We’ll never understand how it all works, and our lives are all the richer for it. What endless mysteries have we to uncover!
If you accept this to be true, there is only one mistake left for you to make, and it’s to believe that you do understand it all. And, as the wisest man in Athens once taught us, correcting this mistake is the simplest thing in the world. All you have to say is this: “I know that I know nothing.”
Interestingly, Michelson was not alone in his dismissal of new paradigms. 10 years after general relativity, the field of quantum mechanics was born, and Einstein himself became one of its harshest critics. At the time, this stance was pretty reasonable, but I can’t help but mention that his famous quote about how the universe is fundamentally deterministic—that “God plays dice”—has since been proven wrong.
Or so we think…