# 5

Intelligence seems related to one's ability to handle high-level abstractions. A minimum intelligence level is necessary to understand certain kinds of things. This post attempts to order concepts by how much intelligence is required to understand them.

I had an employee who could do #1-2 but struggled with #3. I know several people who can do #1-4 just fine but struggle with #5 or cannot do #5 at all. I know people who can do #1-5 but cannot do #6 at all. I know an engineer who can do #1-6 but cannot do #7. I know another engineer who can cruise through #1-8 but struggles with #9.

I have never[1] observed someone go up a level in this hierarchy. Someone who does not grasp a level quickly tends never to grasp it at all. I think your potential to climb this hierarchy is limited by your fluid intelligence.

### 1. Can imitate simple behavior

Anyone reading this post passes this threshold.

### 2. Can follow simple verbal instructions

Anyone reading this post passes this threshold.

### 3. Can follow simple written instructions

Anyone reading this post passes this threshold.

Can solve questions of the following form: "Given a $100,000 initial investment with 4% compounding interest, estimate your profit after 10 years." The testee is allowed to use a calculator. ### 5. Can write Python scripts Anyone who has written original computer code in a mainstream programming language passes this threshold. ### 6. Understands pointers Anyone who can implement a pointer-based algorithm in C passes this threshold. ### 7. Understands recursion Can implement the Fowler–Noll–Vo hashing algorithm recursively and explain why one would do so. ### 8. Understands information density Can crack all three of these problems in an hour or less. Confidently asserting an incorrect answer to the first question fails this test instantly. The testee may use the Internet to look up details of the Java language spec and how the Enigma machine works. This research does not count towards the hour. Otherwise, using the Internet is forbidden. ### 9. Thinks in terms of entropy Can invent unbiased small data[2] algorithms on-the-fly. People with an intuitive grasp of hypothesis space Kolmogorov complexity tend to make good Wall Street quants. ### 10. ??? Here be dragons. 1. I managed to tutor a computer science student from #5 to #6—but I do not think it counts. Ostensibly, she hired me to teach her how pointers work. The real problem was that her college had cancelled the prerequisite class that was supposed to teach her C syntax. She fired me as soon as I had finished explaining the syntax. ↩︎ 2. Edit: There used to be a hyperlink here to a sequence of posts that explained what "small data" is. This hyperlink is now broken. ↩︎ # 5 22 comments, sorted by Highlighting new comments since New Comment The title is 'A Hierarchy of Abstraction' but the article focuses on levels of intelligence. The article claims that intelligence positively correlates with the ability to handle high level abstractions, but it does not talk about actual hierarchies of abstraction. For example, I'd expect a hierarchy of abstraction to contain things like: concrete objects, imagined concrete objects, classes of concrete objects, concrete processes, simulated processes, etc. A more accurate title might be 'The Ability to Understand and Use Abstractions in Computer Science as a Measure of Intelligence.' The article lays out a way of measuring fluid intelligence but does not decouple the crystallized intelligence requirements from the fluid ones. For example, 'Understands Recursion' specifies needing to implement a specific algorithm recursively as a requirement. There are plenty of people who understand and use recursion regularly who do not know that algorithm. (take me) Let's say you test them and they fail. Did they fail because of their fluid intelligence? Did they fail because of a lack of crystallized knowledge related to that specific problem? Did they fail because of abstraction capability requirements in that specific problem, but not recursion in general? What about recursion as a concept makes it hard for people to understand? I would recommend trying to generalize the requirements more. I would recommend exploring other possible attributions of failure other then fluid intelligence. If the article examined the components of recursion it would be more interesting and compelling. What are the components? Drilling down into the components of any of these tests will reveal a lot of context and crystallized knowledge that the article may be taking for granted. (curse of knowledge bias) You might be seeing someone struggle with recursion, and the problem isn't that they can't understand recursion, its that they don't have crystallized knowledge of a building block. As someone who understands recursion to a reasonable level, I'd like to see the article point at the key idea behind recursion that people have trouble grasping. Are there a sequence of words that the article can specify where someone understands what each word means, but finds the overall sentence ineffable? Or perhaps they can parrot it back, but they can't apply it to a novel problem. A requirement of this hypothesis is that someone has all prerequisite crystallized knowledge, but still cannot solve the problem. Otherwise these are not 'hard' boundaries of fluid intelligence. I guess you primarily deal with computers and programming. One way to try and generalize this quickly would be to compare notes across disciplines and identify the pattern. Is there a 'cannot learn pointers' in chemistry for example? I understand that you are trying to share the gist of an idea, but I think these are things that should be further examined if you want other people to take on this mental model. Much more needs to be said and examined in an article that lays out 10 specific levels with specific tests. I'd also be wary of the possibility this entire framework / system looks good because it positions your tribe as superior (computer programmers) and possibly you somewhere comfortably towards the top. This article triggered me emotionally because I think one of the things that prevents people from learning things is the belief that they can't. I wouldn't want anyone to take away from this article that because they didn't understand pointers or recursion at some point in there life, it was because they are dumb and should stop trying. As I pointed out in my other comments, these levels are about potential, not ability. The question are not really about "Can you answer ?" They are more along the lines of "Can you easily learn to answer ?" I believe this decouples "the crystallized intelligence requirements from the fluid ones". What about recursion as a concept makes it hard for people to understand?…I'd like to see the article point at the key idea behind recursion that people have trouble grasping. Recursion requires a person to hold two layers of abstraction in zeir mind simultaneously. This could require twice as much working memory. Working memory, unlike crystallized intelligence, is something we cannot do much to improve. Is there a 'cannot learn pointers' in chemistry for example? I have never tutored chemistry. But I have tutored physics for years. I have never run into anything like "cannot learn pointers" in standard undergraduate physics. The only time I encounter anything like "cannot learn pointers" is when I discuss my weird crackpot theories of relational quantum gravity and entropic time. The ideas involved in undergraduate computer science are simpler than the ideas involved in undergraduate physics. The difference between undergraduate computer science and undergraduate physics is that undergraduate physics is a set of solutions to a set of problems. It requires little creativity. You can (relatively speaking) learn everything by rote. While fluid intelligence helps you learn rote knowledge faster, fluid intelligence tends not to put a hard cap on total rote knowledge acquisition. On the other hand, computer science requires a person to solve novel (albeit simpler) problems all the time. It makes sense one's ability to do this could be limited by fluid intelligence. The other difference between computer science and physics is that in physics you never have to hold two layers of abstraction in your head at the same time. It makes sense that holding two layers of abstraction in your head at the same time could be limited by working memory. One's ability to solve novel problems is the definition of fluid intelligence. Fluid intelligence is correlated with working memory. My brief forays into chemistry suggest chemistry involves more rote knowledge and less "juggling multiple layers of abstraction" than computer science does. Chemistry, like all subjects, is limited by intelligence at some level. But I would expect the floor is lower than physics and computer science. I'd also be wary of the possibility this entire framework / system looks good because it positions your tribe as superior (computer programmers) and possibly you somewhere comfortably towards the top. Really? I do not know a single computer programmer who meets #9. I know only one who passes #8. Most of my programmer friends do not even pass #7. This framework puts quants on top. Not computer programmers. I do not think anyone would doubt that quants are smart people—at least in the sense relevant to this post. Computer programmers tend to be superior at computer programming (and perhaps related fields). This article triggered me emotionally because I think one of the things that prevents people from learning things is the belief that they can't. I wouldn't want anyone to take away from this article that because they didn't understand pointers or recursion at some point in there life, it was because they are dumb and should stop trying. Among the most important things I have learned from Less Wrong is the value of epistemic rationality over instrumental rationality. It breaks my heart that certain people seem unable to perform certain tasks I consider trivial. The greater cruelty is blaming someone for their failure to achieve a goal zey lack the potential to accomplish. You seem to be making a few claims: (1) that these skills require an increasing amount of 1-dimensional intelligence (2) that one cannot do lower-indexed things without doing higher-indexed ones and (3) that there is something fundamental about this. You obviously do not mean this literally, for there are plenty of people who understand recursion but not pointers (i.e.: intro Python students), and plenty of programmers who have never touched Python. First, what is an abstraction? As someone who recently wrote a paper involving Cousot-style abstract interpretation, this is a question that interests me. We in programming-languages research have our answer: that an abstraction is a relation between two spaces such that every behavior in the larger ("concrete") space has a corresponding behavior in the smaller "abstract" space. In this definition, there is a sensible meaning of a hierarchy of abstraction, but it's not what most people think: tracking arbitrary sets of integers is less abstract than tracking intervals of integers, which are in turn less abstract than tracking the possible sign(s) of a number. I am less familiar with abstraction as it is used by AI researchers; my understanding is that it is similar, but without the strict requirements, e.g.: they'll be willing to analyze poker rounding all bets to multiples of$5, and say that two bets of something that rounds to $5 = 1 bet of something that rounds to$10 bet, rather than "something that rounds to $5,$10, or \$15", as would the PL researcher.

Most software engineers seem to use "more abstract" to mean "can be implemented using" or "can be defined using," e.g.: the home screen on my phone is an abstraction built on top of the file system, which is an abstraction built on top of the hardware. This seems to be the closest to what you mean. In that sense, I cannot see what makes these examples a hierarchy, or fundamental in any sense. E.g.: recursion clearly does not require arithmetic. The lambda calculus has recursion but not arithmetic.

The best I can make of this post is that these tasks have something akin to a hard-floor in g-factor required, which is an extraordinary claim in need of extraordinary evidence.

I appreciate your genuine attempt to understand this post. Python is indeed a stand-in for any programming language of comparable complexity. Pointers and recursion could indeed be flipped around. I think we are on the same page concerning these details.

You seem to be making a few claims: (1) that these skills require an increasing amount of 1-dimensional intelligence (2) that one cannot do lower-indexed things without doing higher-indexed ones and (3) that there is something fundamental about this.

1. Yes.
2. Not quite. There can be a little slippage between close-together indices, especially #6 & #7. More importantly, this hierarchy is about talent and potential, not ability. If someone could do #5 with a little bit of study then that counts as doing #5. It is easy to imagine a mathematician who understands recursion but has never written a line of computer code. But I would be surprised if such a person could not quickly learn to write simple Python scripts.
3. This is technically an implication, not a claim. But…yeah.

First, what is an abstraction?

By "abstraction", I mean a simplification of the universe—a model. By "higher-level abstraction", I mean a model built on top of another model.[1]

I am not referring to how these models are built upon one another within mathematics. I am trying to organize them based on how they are constructed inside a typical human brain. For example, a human being learns to write before learning arithmetic and learns arithmetic before learning calculus. It is possible to construct calculus from the Peano axioms while skipping over all of arithmetic. But that is not how a human child learns calculus.

To put it another way, a human being learning a complex skill begins by learning to perform the skill consciously. With practice, the skill becomes unconscious. This frees up the conscious mind to think on a higher (what I call more "abstract") level. Rinse and repeat. To use an example from Brandon Sanderson's creative writing lectures, a novice author is worried about how to put words down on a page while a skilled writer is thinking with plot structure. An author cannot afford to think about higher level activities until the lower level activities are automatic.

I hope this clarifies exactly what I mean by "abstraction".

The best I can make of this post is that these tasks have something akin to a hard-floor in g-factor required…

Yes. That is the point of this post.

…which is an extraordinary claim in need of extraordinary evidence.

Really? What makes you think this is an extraordinary claim? My prior is 50%/50% credence without evidence. Here are some observations I have used to subsequently update those priors.

• I am not the first person to point out that there is a hard floor on the intelligence required to understand pointers. Joel Spolsky wrote about the basic idea concerning pointers back in 2005. My college computer science teacher pointed out something similar in 2011.
• On the low end of the spectrum, I have cared for severely retarded adults who appeared to have -related ceilings. This is a sensitive example and I do not want to get too deep into it. But it should be clear that someone without enough to hold a coherent conversation is unlikely ever to perform arithmetic at the level specified in this post.
• Similarly, an adult who has trouble remembering how to press 3 buttons on a smartphone is unlikely ever to write computer code.
• On the high end of the spectrum, quants are regularly hired from physics departments. These people can apparently pick up finance faster than students trained in economics. The difference seems to be mathematical aptitude. When I talk to quants I can get away with explaining much less than when I talk to, say, a Silicon Valley CTO. At least one friend of mine has confirmed this evaluation.
• I have spent countless hours tutoring physics, computer science and other subjects. The levels in my post comes from my personal experience. When it comes to specific subjects it often feels like there's a block and my interlocutor cannot fit all of the relevant information in zeir head at once.

In the scientific literature it is established that traits like fluid intelligence and working memory are basically capped. If the ability to reason at certain levels of abstraction is capped by fluid intelligence or working memory then that would explain the phenomenon I have observed.

1. This might seem to contradict my claim about talent and potential. You can get around this contradiction by supposing the existence of horizontal alternatives for specific benchmarks. If these alternatives require similar potential to pass then they do not invalidate the ordering. ↩︎

Thanks for the explanation. I accept your usage of "abstraction" as congruent with the common use among software engineers (although I have other issues with that usage)). Confusingly, your hierarchy is a hierarchy in g required, not a hierarchy in the abstractions themselves.

I am well-read in Joel Spoelsky, and my personal experience matches the anecdotes you share. On the other hand, I have also tutored some struggling programmers to a high level. I still find the claim of a g-floor incredible. This kind of inference feels like claiming the insolubility of the quintic because I solved a couple quintics numerically and the numbers look very weird.

Sidenote: I find your example discussion of human learning funny because I learned arithmetic before writing.

the insolubility of the quintic

It's that the general form is unsolvable, not specific examples, without better tools than the usual ones: +, -, *, /, sqrt, ^, etc. I've heard that with hypergeometric functions it's doable, but the same issue reappears for polynomials of higher degree there as well.

This kind of inference feels like claiming the insolubility of the quintic because I solved a couple quintics numerically and the numbers look very weird.

I think it is more like the irreversibility of the trapdoor functions we use in cryptography. We are unable to prove mathematically they are secure. But an army of experts failing to break them is Bayesian evidence.

Sidenote: Lol.

Was just reading through my journal, and found that I had copied this quote. I think you'll find it to be of interest re: teaching recursion.

-------------------------------------

From “Computing Science: Achievements and Challenges” (1999):

“I learned a second lesson in the 60s, when I taught a course on programming to sophomores, and discovered to my surprise that 10% of my audience had the greatest difficulty in coping with the concept of re, cursive procedures. I was surprised because I knew that the concept of recursion was not difficult. Walking with my five-year old son through Eindhoven, he suddenly said "Dad not every boat has a life-boat, has it?" '"'How come?" I said. "Well, the life-boat could have a smaller life-boat, but then that would be without one." It turned out that the students with problems were those who had had prior exposure to FORTRAN, and the source of their difficulties was not that FORTRAN did not permit recursion, but that they had not been taught to distinguish between the definition of a programming language and its implementation and that their only handle on the semantics was trying to visualize what happened during program execution. Their only way of "understanding" recursion was to implement it, something of course they could not do. Their way of thinking was so thoroughly operational that, because they did not see how to implement recursion, they could not understand it. The inability to think about programs in an implementation-independent way still afflicts large sections of the computing community, and FORTRAN played a major role in establishing that regrettable tradition”

Significantly, your examples are all within the domain of analytical—and I would suggest, reductive—mathematics-science-coding. In my experience, it is often the case that one who has ascended this ladder is quite blind to, and unable to conceive of the importance of, context and perspective to meaning-making.

Apropos, there was a period during my childhood when I tried to question my elders regarding my observation that entropy, probability, and meaning were inherently subjective—meaningless without reference to an observer. Similar to the misnomer of Shannon "information" theory when it is actually about the transfer of data, rather than information through a lossy channel. In virtually every case, they could not understand my questions.

People generally fail to understand that "subjective" does not imply "arbitrary", but rather, perspectival and necessarily within context. And of course when it comes to agreement, we share a great deal of (hierarchical) perspective.

One concrete example: Back when I was a technical manager in the field of analytical instruments, I tried to convey to our support team that the key metric of success was customer satisfaction, followed, if necessary, by whether the instrument met standards for precision, sensitivity, stability, etc. Of the members of that team, the one who I knew and relied on to be the most rigorously analytical could not accept that definition of success, and was therefore weaker (overall) in terms of achieving customer satisfaction.

As for your potential item #10—remaining within the analytical paradigm you have established—I would suggest "the ability to think in terms of evolutionary processes and strategies, especially in regard to production of meaningful novelty."

Similar ladders exist in other domains—drawing, for example. But I have not observed such hard ceilings on one's performance in drawing. Effective training seems far more important. The same goes for routine physics, entrepreneurship and foreign languages.

I like where you are going with #10. Perhaps it could condensed into "informatic creativity".

Yes, I think evolutionary processes are the only generator of meaningful novelty, and this is also key to the nearly always neglected question of "hypothesis generation" in discussions of "the" scientific method.

If one person is talking analytically and the other is talking about meaning-making, then you're each trying to have different conversations.  One of you is talking about how to do something and the other is talking about how to motivate people to do something.  If at all possible you should let the first person lead; if they're diligently working on the problem then they're motivated enough.

To consider your support team example: they seem to be assuming that if their product works well, customers will be satisfied.  That's not a terrible strategy, and it puts the focus on something they can control (the product).  If you could point to something else about the customer experience that's causing customer dissatisfaction, they would probably understand the problem and deal with it.  But if there's nothing specific that needs addressing otherwise, it's probably best just to let them focus on getting the instrument to work as well as possible.

And of course, maximizing customer satisfaction is itself a strategy toward achieving your real goal, which is profit.  Companies don't give their flagship products away for free*, no matter how much it would please the customers.

*With the exception of some loss leaders that are carefully calculated to grow revenues over the long term.

[-][anonymous]1y 3

Secondary to the hierarchy itself or the claim about g floors which Darmani has already commented on: the tests you refer to for various levels don’t seem like reliable measures of the claims you’re making. As one example, I definitely understand pointers (I’ve written a memory management subsystem in C for a widely-used open source project) but I can’t make heads or tails of your pointer puzzle. I have similar quibbles for 4, 7, and 8.

The pointer puzzle was unfair. I have removed it. What quibble do you have with #4?

[-][anonymous]1y 1

Compound interest is a thing in the world people might not know about despite knowing arithmetic. It’s also underspecified (presumably compounds annually?) and it’s not clear how close your estimate has to be to “pass”.

Just in general, while actual pass/fail tests like that may be indicative, I find them too easily confounded by other factors to be strong solo evidence of whether somebody has a given piece of knowledge or ability. “Can write code in a mainstream language” is obviously a lot fuzzier, but for examples like this post I think that’s a pro, not a con.

The levels are about potential, not ability. The tests are not pass/fail (except for #8). They are pass/null tests. Ability implies potential. Lack of ability does not imply lack of potential. (Except nominally in the Bayesian sense.)

This is interesting.  I'm curious how it relates to "time taken to master each understanding."  For instance, how do you rate someone who has taken a year to understand recursion, vs. someone who bounced off after failing a single CS class.

I have never met someone who took a year to understand recursion. If it is not grasped quickly then the student never reaches mastery at all. Many programmers read this website and so far not a single one has contested this claim.

Incidentally, I bounced off of CS for a couple years after getting a bad grade in my first CS class. But I bounced off because the class was too easy and therefore not worth time. Not because it was too hard.

I have never met someone who took a year to understand recursion.

I probably took much more, but that's because I first heard about the concept (in Karel) when I was 10 years old. I kinda-understood the most simple implementation, but anything beyond that was too abstract for me. Then I didn't need the concept for a few years. And then, maybe five years later, I finally understood it, but it still felt uncomfortable.

These days it feels obvious, and I guess the trick is that before I start writing the code, I write down an exact specification of what the function does -- and that makes it easy to decide when calling the function with a smaller parameter is the right thing to do, and when something extra needs to be done. (Plus now I also think about tail recursion, which is usually the right thing to do.)

General intelligence is still increasing at 10 years old. Simply getting older is sufficient to explain your experience.

I feel like I have personally experienced of banging against a math concept for months before it clicks (that is, I'm not just guessing based on vague teacher's passwords) but I can't remember a specific example so perhaps you're right.

I was 12 or so when I first studied pointers. I did not get them at all back then.