A large chunk - plausibly the majority - of real-world expertise seems to be in the form of illegible skills: skills/knowledge which are hard to transmit by direct explanation. They’re not necessarily things which a teacher would even notice enough to consider important - just background skills or knowledge which is so ingrained that it becomes invisible.
I’ve recently noticed a certain common type of illegible skill which I think might account for the majority of illegible-skill-value across a wide variety of domains.
Here are a few examples of the type of skill I have in mind:
- While operating a machine, track an estimate of its internal state.
- While talking to a person, track an estimate of their internal mental state - emotions, engagement, thoughts/worries, true motivations, etc.
- While writing an algorithm, track a Fermi estimate of runtime.
- While reading or writing math, track a prototypical example of what the math is talking about.
- While playing a competitive game, track an estimate of the other players’ plans, intentions and private information
- While writing, track an estimate of the mental state of a future reader - confusion, excitement, eyes glossing over, etc.
- While reasoning through a difficult search/optimization problem, track an estimate of which constraints are most taut.
- While working on math, physics, or a program, track types/units
- While working on math, physics, or a program, track asymptotic behavior
- While in conversation, track ambiguous tokenization for potential jokes.
- While presenting to a crowd, track engagement level.
- While absorbing claims/information, track an estimate of the physical process which produced the information, and how that process entangles the information with physical reality.
The common pattern among all these is that, while performing a task, the expert tracks some extra information/estimate in their head. Usually the extra information is an estimate of some not-directly-observed aspect of the system of interest. From outside, watching the expert work, that extra tracking is largely invisible; the expert may not even be aware of it themselves. Rarely are these mental tracking skills explicitly taught. And yet, based on personal experience, each of these is a central piece of performing the task well - arguably the central piece, in most cases.
Let’s assume that this sort of extra-information-tracking is, indeed, the main component of illegible-skill-value across a wide variety of domains. (I won’t defend that claim much; this post is about highlighting and exploring the hypothesis, not proving it.) What strategies does this suggest for learning, teaching, and self-improvement? What else does it suggest about the world?
Pay Attention To Extra Information Tracking
I had a scheme, which I still use today when somebody is explaining something that I’m trying to understand: I keep making up examples. For instance, the mathematicians would come in with a terrific theorem, and they’re all excited. As they’re telling me the conditions of the theorem, I construct something which fits all the conditions. You know, you have a set (one ball) – disjoint (two balls). Then the balls turn colors, grow hairs, or whatever, in my head as they put more conditions on. Finally they state the theorem, which is some dumb thing about the ball which isn’t true for my hairy green ball thing, so I say, ‘False!’
- Feynman
A lot of people have heard Feynman’s “hairy green ball thing” quote. It probably sounds like a maybe-useful technique to practice, but not obviously more valuable than any of a dozen other things.
The hypothesis that extra-information-tracking is the main component of illegible-skill-value shines a giant spotlight on things like Feynman’s examples technique. It suggests that a good comparison point for the value of tracking a prototypical example while reading/writing math is, for instance, the value of tracking the probable contents of opponents’ hands while playing poker.
More generally: my guess is that most people reading this post looked at the list of examples, noticed a few familiar cases, and thought “Oh yeah, I do that! And it is indeed super important!”. On the other hand, I’d also guess that most people also saw some unfamiliar cases, and thought “Yeah, I’ve heard people suggest that before, and it sounds vaguely useful, but I don’t know if it’s that huge a value-add.”.
The first and most important takeaway from this post is the hypothesis that the unfamiliar examples are about as important to their use-cases as the familiar examples. Take a look at those unfamiliar examples, and imagine that they’re as important to their use-cases as the examples you already use.
Ask “What Are You Tracking In Your Head?”
Imagine that I’m a student studying under Feynman. I know that he’s one of the great minds of his generation, but it’s hard to tell which things I need to pick up. His internal thoughts are not very visible. In conversation with mathematicians, I see him easily catch errors in their claims, but I don’t know how he does it. I could just ask him how he does it, but he might not know; a young Richard Feynman probably just implicitly assumes that everyone pictures examples in their head, and has no idea why most people are unable to easily catch errors in the claims of mathematicians!
But if I ask him “what were you tracking in your head, while talking to those mathematicians?” then he’s immediately prompted to tell me about his hairy green ball thing.
More generally: for purposes of learning/teaching, the key question to ask of a mentor is “what are you tracking in your head?”; the key question for a mentor to ask of themselves is “what am I tracking in my head?”. These extra-information-tracking skills are illegible mainly because people don’t usually know to pay attention to them. They’re not externally-visible. But they’re not actually that hard to figure out, once you look for them. People do have quite a bit of introspective access into what extra information they’re tracking. We just have to ask.
Returns to Excess Cognitive Capacity
Mentally tracking extra information is exactly the sort of technique you’d expect to benefit a lot from excess cognitive capacity, i.e. high g-factor. Someone who can barely follow what’s going on already isn’t going to have the capacity to track a bunch of other stuff in parallel.
… which suggests that extra-information-tracking techniques are particularly useful investments for people with unusually high g. (Hint: this post is on LW, so “unusually high g” probably describes you!) They’re a way to get good returns out of excess cognitive capacity.
The same argument also suggests a reason that teaching methods aren’t already more focused on mentally tracking extra information: such techniques are probably more limited for the median person. On the other hand, if your goal is to train the great minds of the next generation, then figuring out the right places to invest excess cognitive capacity is likely to have high returns.
Other Examples?
Finally, the obvious question: what extra information do you mentally track, which is crucial to performing some task well? If the hypothesis is right, there’s probably high-value mental-tracking techniques which some, but not all, people reading this already use. Please share!
This post is probably right that illegible skills rely on tracking non-obvious bits of information. But I don't think that discovering that info is as simple as asking "What Are You Tracking in Your Head". Remember that there's a lot of inferential distance between the you and an expert, and they've likely forgotten all that you don't know.
... (read more)Thankfully the problem of getting tacit knowledge out of someone has a growing literature on it that is quite useful. The field of Naturalistic Decision Making developed some techniques to do this, one of which is fairly simple. It is called Applied Cognitive Task Analysis. Here's a summary of it from CommonCog [1]:
I was thinking you had all of mine already, since they're mostly about explaining and coding. But there's a big one: When using tools, I'm tracking something like "what if the knife slips?". When I introspect, it's represented internally as a kind of cloud-like spatial 3D (4D?) probability distribution over knife locations, roughly co-extentional with "if the material suddenly gave or the knife suddenly slipped at this exact moment, what's the space of locations the blade could get to before my body noticed and brought it to a stop?". As I apply more force this cloud extends out, and I notice when it intersects with something I don't want to get cut. (Mutatis mutandis for other tools of course. I bet people experienced with firearms are always tracking a kind of "if this gun goes off at this moment, where does the bullet go" spatial mental object)
I notice I'm tracking this mostly because I also track it for other people and I sometimes notice them not tracking it. But that doesn't feel like "Hey you're using bad technique", it feels like "Whoah your knife probability cloud is clean through your hand and out the other side!"
Seems at least partially related to cognitive apprenticeship, a type of teaching that aims at explicitly making the teacher's "thinking visible", so that the pupils can find out what it is that the teacher is tracking at any moment when solving the problem. For instance, they might carry out an assignment in front of pupils and try to explicitly speak out loud their thoughts while doing it.
For instance, when writing an essay:
... (read more)I tried to follow my own thoughts on the polynomial example. They were pretty brief; the whole problem took only a few seconds. Basically:
... so I guess +1 point for the "bag of tricks" model of expertise.
Kinda surprised you didn't mention purpose-tracking, for while you're trying to do a thing--any thing. Arguably the most important skill I acquired from the Sequences, and that's a high bar.
In resource management games, I typically have a set of coefficients in my head for the current relative marginal values of different resources, and my primary heuristic is usually maximizing the weighted sum of my resources according to these coefficients.
In combat strategy games, I usually try to maximize (my rate of damage) x (maximum damage I can sustain before I lose) / (enemy rate of damage) x (damage I need to cause before I win).
These don't seem especially profound to me. But I've noticed a surprising number of video games that make it distressingly hard to track these things; for instance, by making it so that the data you need to calculate them is split across three different UI screens, or by failing to disclose the key mathematical relationships between the public variables and the heuristics I'm trying to track. ("You can choose +5 armor or +10 accuracy. No, we're not planning to tell you the mathematical relationship between armor or accuracy and observable game outcomes, why do you ask?")
It's always felt odd to me that there isn't widespread griping about such games.
As a result of reading this post, I have started explicitly tracking two hypotheses ... (read more)
Another example from Feynman: Besides the object level of what the math or physics described symbolically, he was tracking what that meant in real life. Not as obvious as you'd think. See, e.g., the anecdote about Brewster's angle. The most common form is Guessing the Teacher's Password - which happens if there is no spare capacity to track what all these symbols mean in real life. Tracking the symbols is difficult enough if you are at the limits of your ability (though it might also result from investing as little effort as possible to pass).
An all too common folly.
Nice post!
One of my fears is that the True List is super long, because most things-being-tracked are products of expertise in a particular field and there are just so many different fields.
Nevertheless:
I could imagine a website full of such lists, categorized by task or field. Could imagine getting lost in there for hours...
This may be true if you write a scientific paper, an essay or a non-fiction book. As a professional writer, when I write a novel, I usually don't think about the reader at all (maybe because, in a way, I am the reader). Instead, I track a mental state of the character I'm writing about. This leads to interesting situations when a character "decides" to do something completely different from what I intended her to do, as if she had her own will. I have heard other writers describe the same thing, so it seems to be a common phenomenon. In this situation, I have two options: I can follow the lead of the character (my tracking of her mental state) and change my outline or even ditch it completely, or I can force her to do what the outline says she's supposed to do. The second choice inevitably leads to a bad story, so tracking the mental state of your characters indeed seems to be essential to writing good fiction.
I assume that readers do a similar thing, so if a character in a book does something that doesn't fit the mental model they have in mind, they often find it "unbelievable" or "unrealistic", which is one of the reasons while "listen to your characters" seems to be good advice while writing.
Not type tracking seems to be the thing that makes people susceptible to bad philosophical intuition pumps. In particular, that the referent of a token shifts between a proposition being used at different points in the problem.
In Advanced Driving courses a key component was (and may still be -it’s been awhile) commentary driving. You sit next to an instructor and listen to them give a commentary on everything they are tracking, for instance other road users, pedestrians, road signs, bends, obstacles, occluders of vision etc; and how these observations affect their decision making, as they drive. Then you practice doing the same, out loud, and, ideally, develop the discipline to continue practising this after the course. I found this was a very effective way of learning from an expert, and I’m sure my driving became the safer because of it.
I have a couple frameworks that seem to fit into this:
One is the Greek word "kairos," which means... something kinda like "right now," but in the context of rhetoric means something more like "the present environment and mood." A public speaker, when giving a speech, should consider the kairos and tailor their speech accordingly. This cashes out in stuff like bands yelling out, "How are you tonight, Houston?!" or a comedian riffing off of a heckler. It's the thing that makes a good public speaker feel like they're not just delivering a canned speech they'v... (read more)
My comment here is a bit narrow, but re
A lot of people get surprised at how quickly and easily I intuit linear algebra things, and I think a major factor is this. For linear algebra, you can think of vector spaces as being the equivalent of types/units. E.g. the rotation map for a diagonalization maps between the original vector space and the direct sum of the eigenvectors. Sort of.
It's always the first question I ask when I see a matrix or a tensor - what spaces does it map between?
Tracking an estimate of how warm food is while being cooked and how its consistency changes (ex. what the bottom of a slice of eggplant looks like without flipping it over).
Precisely estimating the note I will sing before singing it. I'm never totally accurate, but I find it extremely helpful in order to become more accurate.
Tracking an estimate of how my/another's body will feel after I massage it in an area/move it in a way.
Tracking an estimate of the risk of germs on my hands.
Tracking an estimate of the line that is at the center of my body's mass and r... (read more)
Curated. I think figuring out how to transfer hidden/illegible skills is a major bottleneck and like this post for digging into that.
I do agree with Algon's comment that simply asking an expert what they're tracking may not be good enough – some expert's brains seem nicely legible and accessible to themselves, sometimes they're tracking so much stuff it's hard to articulate.
There is an interview technique called Experiential Array which is designed to pull out this sort of information (and some other stuff too). Matt Goldenberg conducted this type of interview on me on the topic of designing and running events. This experience gave me the ability to communicate the invisible parts of event design.
Read here for more details
Great post. Would add as an example: "While thinking about something and trying to figure out your viewpoint on it, track internal feelings of cognitive dissonance and confusion"
As a complement or intro to this technique, I find it helpful to create checklists. This helps me identify the most important items to track. I can either do it in my head, or externally via the checklist. It's often easy to come up with a reasonable checklist if you can define the topic specifically enough. Once I've created a checklist, and worked with it enough to commit it to memory, I find that new relevant information is easy to synthesize. If I encounter new information with no checklist, on the other hand, it's very hard for me to remember or make sense of it.
The world is full of scale-free regularities that pop up across topics not unlike 2+2=4 does. Ever since I learned how common and useful this is, I've been in the habit of tracking cross-domain generalizations. That bit you read about biology, or psychology, or economics, just to name a few, is likely to apply to the others in some fashion.
ETA: I think I'm also tracking the meta of which domains seem to cross-generalize well. Translation is not always obvious but it's a learnable skill.
I'm a pretty good poet. I usually don't share my poetry except with close friends, but take my word for it, my poetry is good enough that I think the majority of people who heard it would like it. What am I tracking when I write a poem?
Well, it happens almost automatically - if I have inspiration, the poem just comes out; if I don't have inspiration, I can sort of try to write something but it doesn't work. So it's a partly subconscious process already and not necessarily something that can be analyzed like this; but I know at the very least I am tracking ... (read more)
Hmm, I don't think this kind of tacit knowledge and skills is at all obvious to the holder. In most cases it's like asking a centipede how exactly it walks. Feynman was unusually introspective about this, not an easy example to follow for mere mortals.
A lot of items in your list are about modeling other agents and yourself. In an embedded agency abstraction hierarchy it would be close to the top (model the general environment, model other agents, model self), so probably a recent evolutionary development, not very well entrenched into the genome, that's why we have trouble "just doing it" and need to introspect to make it make sense.
I track the age of data.
Here are a couple examples of how this is helpful:
- Wikipedia has case counts by country for the 2022 monkeypox outbreak. Portugal was one of the leading countries for a while in number of confirmed cases, but it has since been surpassed by others. However, on closer inspection, the numbers for Portugal haven't been updated since the 7th of July, 5 days ago. In the context of exponential growth, that matters a lot
... (read more)Can you give an example?
An "isthmus" and a "bottleneck" are opposites. An isthmus provides a narrow but essential connection between two things (landmass, associations, causal chains). A bottleneck is the same except the connection is held back by its limited bandwidth. In the case of a bottleneck, increasing its bandwidth is top priority. In the case of an isthmus, keeping it open or discovering it in the first place is top priority.
I have a habit of making up pretty words for myself to remember important concepts, so I'm calling it an "isthmus variable" when it's the thing you ... (read more)
I track my confidence in a given step of a hypothesised chain of mathematical reasoning via a heuristic along the lines of “number of failed attempts at coming up with a counterexample”.
Another immediate question is "for what tasks you don't have to do that?", partly because then one can ask the following question of why. For example, I think now that one doesn't have to track extra information when feeding livestock. (A not-too-variable time-consuming routine.) But I haven't yet really tracked what I do when I do that.
This post reminded me of the exercises in Calibrating with Cards, a post which very nicely advises what to pay attention to during magic practice.
minor feedback...
note that as one practices tracking something, the effort needed to track it goes down.
i don't think it makes sense to think of it like needing excess cognitive capacity to track things. i think our skill improves to the point of needing little to no excess cognitive capacity. so we only need excess cognitive capacity for new things we want to track.
It may be critical to note that tracking estimates of the internal states of other entities often feels like just having a clue about what's going on. If someone asks us how we came to our intuitions, without careful introspection, we might answer with "I just know" or "I pay attention is how!" or similar.
To unpack a mundane example, here's a somewhat rambly account of some of what I'm tracking in my head while I operate a motor vehicle:
When I'm driving, I don't actively scan through all the sounds and smells and tactile events and compare them with past e... (read more)
I started writing down things I am tracking.
I actually never realized I am tracking so many things.
The problem and issue is, I rarely remember or know what to do with the tracked information.
Lets say I am trying to be engaging and have a discussion.
There could be a number of things to track, from motives, meanings, or specific reasons something is said.
Other thing to track is filling in the gaps. Lets say someone says something incomplete, one should when engaged fill in the gaps and ask question or find a way to follow up.
Another thing is to know yo... (read more)
This reminds me of dual N-back training. Under this frame, dual N-back would improve your ability to track extra things. It's still unclear to me whether training it actually improves mental skills in other domains.
When thinking about a physics problem or physical process or device, I track which constraints are most important at each step. This includes generic constraints taught in physics classes like conservation laws, as well as things like "the heat has to go somewhere" or "the thing isn't falling over, so the net torque on it must be small".
Another thing I track is what everything means in real, physical terms. If there's a magnetic field, that usually means there's an electric current or permanent magnet somewhere. If there's a huge magnetic field, that usual... (read more)
The extent to which you can benefit from asking what someone is tracking in their head, and the degree to which they can usefully explain it to you, will depend critically on how much information, basic to the topic at hand, you two of you already share.
You can learn more from a master, using this technique, the more you already knew.
If “cognitive capacity” is the amount of information useful to some specific domain of problem solving one has in one’s head, then everyone on Earth has more or less the same cognitive capacity (excepting only people with some... (read more)
When programming, I track a mixed bag of things, top of which is readability: Will me-6-months-from-now be able to efficiently reconstruct the intention of this code, track down the inevitable bugs, etc.?
"Finally, the obvious question: what extra information do you mentally track, which is crucial to performing some task well?"
When I try to cook something complicated by recipe, I go over each line of the recipe and previsualize all the corresponding physical actions.
I previsualize the state, amount, location and the transitions for each object. Objects = {pots, pans, ingredients, oil, condiments, package, piece of trash, volume of water, stove, task-completion times, hands, free seconds/minutes for cleaning during the cook, towel, tissue paper...}.
Th... (read more)
One answer to the question for me:
While writing, something close to "how does this 'sound' in my head naturally, when read, in an aesthetic sense?"
I've thought for a while that "writing quality" largely boils down to whether the writer has an intuitively salient and accurate intuition about how the words they're writing come across when read.
Now that I read this, I notice that I automatically do this when i'm in school, and that it's much more automatic and frequent in subjects I find easy (I wonder whether it's the tracking that makes it easy, or whether less effort frees up brain space to track?).
In history class, I always keep a mental map of when something happened, why it happened, and what resulted from it. I was very surprised when I found out none of my friends do anything similar, because it's such an obvious tool for seeing the bigger picture and remembering how things fit together f... (read more)
The example that immediately popped into my head was the difficulty I had training lawyers to conduct a jury trial.
You spend so much time building up a persuasive story to tell, but at trial most of your mental effort is elsewhere.
You have the jury, individual jurors, the judge, the witnesses, and opposing counsel, your story, the opposing story, and rules of evidence and procedure. All of these things are moving and changing at the same time.
It felt unteachable. I've said things like "it takes extreme observation".
In the main, isn’t this post mostly representing the idea of tacit knowledge, albeit under a different name?
https://en.m.wikipedia.org/wiki/Tacit_knowledge
This sounds a lot like mindfulness. :-)
Cool post :)
This is essentially what machine learning, especially ANN, is.