Before I make my main point, I want to acknowledge that curriculum development is hard. It's even harder when you're trying to teach the unteachable. And it's even harder when you're in the process of bootstrapping. I am aware of the Kahneman inside/outside curriculum design story. And, I myself have taught 200+ hours of my own computer science curricula to middle-school students. So this "open letter," is not some sort of criticism of CFAR's curriculum; It's a "Hey, check out this cool stuff eventually when you have time," letter. I just wanted to put all this out there, to possibly influence the next five years of CFAR.

Curriculum development is hard.

So, anyway, I don't personally know any of the people involved in CFAR, but I do know you're all great. 


A case for developmental thinking

The point of this post is to make a case for CFAR to become "developmentally aware." Massive amounts of quality research has gone into describing the differences between 1) children, 2) adults, and 3) expert or developmentally advanced adults. I haven't (yet?) seen any evidence of awareness of this research in CFAR's materials. (I haven't attended a CFAR workshop, but I've flipped through some of the more recent stuff.)

Developmental thinking is a different approach than, e.g., cataloguing biases, promoting real-time awareness of them, and having a toolbox of de-biasing strategies and algorithms. Developmental literature gives clues to the precise cognitive operations that are painstakingly acquired over an entire lifetime, in a more fine-grained way than is possible when studying, say, already-expert performers or cognitive bias literature. I think developmental thinking goes deeper than "toolbox thinking" (straw!) and is an angle of approach for teaching the unteachable

Below is an annotated bibliography of some of my personal touchstones in the development literature, books that are foundational or books that synthesize decades of research about the developmental aspects of entrepreneurial, executive, educational, and scientific thinking, as well as the developmental aspects of emotion and cognition. Note that this is personal, idiosyncratic, non-exhaustive list.

And, to qualify, I have epistemological and ontological issues with plenty of the stuff below. But some of these authors are brilliant, and the rest are smart, meticulous, and values-driven. Lots of these authors deeply care about empirically identifying, targeting, accelerating, and stabilizing skills ahead of schedule or helping skills manifest when they wouldn't have otherwise appeared at all. Quibbles and double-takes aside, there is lots of signal, here, even if it's not seated in a modern framework (which would of course increase the value and accessibility of what's below).

There are clues or even neon signs, here, for isolating fine-grained, trainable stuff to be incorporated into curricula. Even if an intervention was designed for kids, a lot of adults still won't perform consistently prior to said intervention. And these researchers have spent thousands of collective hours thinking about how to structure assessments, interventions, and validations which may be extendable to more advanced scenarios.

So all the material below is not only useful for thinking about remedial or grade-school situations, and is not just for adding more tools to a cognitive toolbox, but could be useful for radically transforming a person's thinking style at a deep level.


child:adult :: adult: ? 

This has everything to do with the "Outside the Box" Box. Really. One author below has been collecting data for decades to attempt to describe individuals that may represent far less than one percent of the population.


0. Protocol analysis

Everyone knows that people are poor reporters of what goes on in their heads. But this is a straw. A tremendous amount of research has gone into understanding what conditions, tasks, types of cognitive routines, and types of cognitive objects foster reliable introspective reporting. Introspective reporting can be reliable and useful. Grandaddy Herbert Simon (who coined the term "bounded rationality") devotes an entire book to it. The preface (I think) is a great overview. I wanted to mention this, first, because lots of the researchers below use verbal reports in their work.


1. Developmental aspects of scientific thinking

Deanna Kuhn and colleagues develop and test fine-grained interventions to promote transfer of various aspects of causal inquiry and reasoning in middle school students. In her words, she wants to "[develop] students' meta-level awareness and management of their intellectual processes." Kuhn believes that inquiry and argumentation skills, carefully defined and empirically backed, should be emphasized over specific content in public education. That sounds like vague and fluffy marketing-speak, but if you drill down to the specifics of what she's doing, her work is anything but. (That goes for all of these 50,000 foot summaries. These people are awesome.)


David Klahr and colleagues emphasize how children and adults compare in coordinated searches of a hypothesis space and experiment space. He believes that scientific thinking is not different in kind than everyday thinking. Klahr gives an integrated account of all the current approaches to studying scientific thinking. Herbert Simon was Klahr's dissertation advisor.


2. Developmental aspects of executive or instrumental thinking

Ok, I'll say it: Elliot Jacques was a psychoanalyst, among other things. And the guy makes weird analogies between thinking styles and truth tables. But his methods are rigorous. He has found possible discontinuities in how adults process information in order to achieve goals and how these differences relate to an individuals "time horizon," or maximum time length over which an individual can comfortably execute a goal. Additionally, he has explored how these factors predictably change over a lifespan.


3. Developmental aspects of entrepreneurial thinking

Saras Sarasvathy and colleagues study the difference between novice entrepreneurs and expert entrepreneurs. Sarasvathy wants to know how people function under conditions of goal ambiguity ("We don't know the exact form of what we want"), environmental isotropy ("The levers to affect the world, in our concrete situation, are non-obvious"), and enaction ("When we act we change the world"). Herbert Simon was her advisor. Her thinking predates and goes beyond the lean startup movement.

"What effectuation is not"


4. General Cognitive Development

Jane Loevinger and colleagues' work have inspired scores of studies. Loevinger discovered potentially stepwise changes in "ego level" over a lifespan. Ego level is an archaic-sounding term that might be defined as one's ontological, epistemological, and metacognitive stance towards self and world. Loevinger's methods are rigorous, with good inter-rater reliability, bayesian scoring rules incorporating base rates, and so forth.

Here is a woo-woo description of the ego levels, but note that these descriptions are based on decades of experience and have a repeatedly validated empirical core. The author of this document, Susanne Cook-Greuter, received her doctorate from Harvard by extending Loevinger's model, and it's well worth reading all the way through:

Here is a recent look at the field:

By the way, having explicit cognitive goals predicts an increase in ego level, three years later, but not an increase in subjective well-being. (Only the highest ego levels are discontinuously associated with increased wellbeing.) Socio-emotional goals do predict an increase in subjective well-being, three years later. Great study:

Bauer, Jack J., and Dan P. McAdams. "Eudaimonic growth: Narrative growth goals predict increases in ego development and subjective well-being 3 years later." Developmental Psychology 46.4 (2010): 761.


5. Bridging symbolic and non-symbolic cognition


Eugene Gendlin and colleagues developed a "[...] theory of personality change [...] which involved a fundamental shift from looking at content [to] process [...]. From examining hundreds of transcripts and hours of taped psychotherapy interviews, Gendlin and Zimring formulated the Experiencing Level variable. [...]"

The "focusing" technique was designed as a trainable intervention to influence an individual's Experiencing Level.

Marion N. Hendricks reviews 89 studies, concluding that [I quote]:

  • Clients who process in a High Experiencing manner or focus do better in therapy according to client, therapist and objective outcome measures.
  • Clients and therapists judge sessions in which focusing takes place as more successful.
  • Successful short term therapy clients focus in every session.
  • Some clients focus immediately in therapy; Others require training.
  • Clients who process in a Low Experiencing manner can be taught to focus and increase in Experiencing manner, either in therapy or in a separate training.
  • Therapist responses deepen or flatten client Experiencing. Therapists who focus effectively help their clients do so.
  • Successful training in focusing is best maintained by those clients who are the strongest focusers during training. [IFS is very similar to focusing] [more references, similar to focusing] [favorite book of all time, by the way]


6. Rigorous Instructional Design

Siegfried Engelmann ( and colleagues are dedicated to dramatically accelerating cognitive skill acquisition in disadvantaged children. In addition to his peer-reviewed research, he specializes in unambiguously decomposing cognitive learning tasks and designing curricula. Engelmann's methods were validated as part of Project Follow Through, the "largest and most expensive experiment in education funded by the U.S. federal government that has ever been conducted," according to Wikipedia. Engelmann contends that the data show that Direct Instruction outperformed all other methods:

Here, he systematically eviscerates an example of educational material that doesn't meet his standards:

And this is his instructional design philosophy:



In conclusion, lots of scientists have cared for decades about describing the cognitive differences between children, adults, and expert or developmentally advanced adults. And lots of scientists care about making those differences happen ahead of schedule or happen when they wouldn't have otherwise happened at all. This is a valuable and complementary perspective to what seems to be CFAR's current approach. I hope CFAR will eventually consider digging into this line of thinking, though maybe they're already on top of it or up to something even better.

New to LessWrong?

New Comment
6 comments, sorted by Click to highlight new comments since: Today at 5:04 AM

Eugene Gendlin and colleagues developed ... The "focusing" technique

Just want to mention that this isn't just important for therapy, it's foundational to any sort of mindhacking or self-improvement technique that's directed at internal self-modification (as opposed to environmental intervention). Indeed, Gendlin's "focusing" is basically the same thing I've previously written about as "RMI", or that Byron Katie describes as "thinking with the heart", or that NLP calls "transderivational search" and many other people refer to as "the small, still voice". It can also be considered as akin to fractionation in self-hypnosis, where you're dropping in and out of a trance-like state, though it's exactly the same kind of trance you go into when you're say, trying to remember where you left your keys.

As Gendlin and co report, many people do this automatically and naturally, others have to be trained. And it makes a big difference in whether a given class of cognitive techniques will produce genuine behavioral change, vs. just intellectual insights and superficial agreement that one ought to act differently.

I think Byron Katie is quite able to distinguish the heart from the belly and when she says heart she actually means it. In Gendlin's focusing you are welcome to think with all parts of your body.

I'm not sure where you draw the relationship to fractionation. I do know that from a hypnosis context but I wouldn't use it with focusing.

When it comes to "the small, still voice", people can have that without being connected to the felt sense.

I don't/didn't have time to assemble a coherent section on this, but I also want to point people towards Acceptance and Commitment Therapy. Related buzzphrases: Experiential avoidance, rule-governed behavior, environmentally contingent behavior, repertoire narrowing, relational frame theory... (Stephen C Hayes and colleagues) There's a developmental line or skillset in there that relates to ugh fields, original seeing, cached thoughts, and more, in LessWrong jargon. See also "Compassion Focused Therapy: Distinctive Features" for more references and exercises for influencing "socioemotional" brain systems.

I experienced an invigorating degree of cognitive shear reading your linked summary of ego levels. I suspect there was a bit of Forer effect going on, as I "recognized" aspects of myself in every level described. I would guess that I am probably mostly somewhere in the Autonomous domain, with temporary swings into the higher and lower levels based on mood and environment.

And I find it fairly unconvincing that I should want to develop to a so-called higher level. It seems that levels higher than Autonomous place ethical concerns into an "it's all relative, man" perspective, which would reduce the degree to which one finds ethical concerns to be motivating, which I see as a bad thing.

It's amusing in an ironic way that, since I am apparently still an ego dominated person, I read the linked document with an attitude of "how can I use this, what good is this to me." Ultimately I came away with the thought, "Psh, inner peace, bliss, and a holistic sense of oneness and compassion? That's not going to make me more productive or help me to win dominance contests against my peers." I'm fully capable of recognizing the paradox even while thinking it - loosely, this is about a person's utility function (or frame of reference for assigning value, or whatever) changing over time due to supposedly greater levels of internal cognitive integration, and at any given level you don't want to modify your utility function because you don't want to stop wanting what you want.

Anyway, you've successfully made me curious.


The later stages of that document are heavily influenced by Western meditation styles. Note that Ken Wilber, whose model heavily influences it, is a well-known figure in the Western Buddhism community. I don't think Ken Wilber would disagree with me in saying that reaching the "Unitive stage" described therein is only possible through contemplative practice (i.e. meditation), since it basically describes in academic terms a very conventional modern Buddhist view of enlightenment.

I am a very regular meditation practitioner, and it strikes me as somewhat odd to put it on a continuum with stages of developmental psychology, although perhaps my opinion will change once I'm enlightened. Based on my understanding, it seems orthogonal. For example, a characteristic described of stage 4 is "Truth: Can be found through appropriate scientific methods if not now later." This seems like a simple statement of philosophy based on intellectual arguments that would be unlikely to change just because one practices enough to make their mind experience non-duality between observer and observed.

Meditation is just a way of practicing with your mind to experience reality in a specific way. That experience often does change how people think reality works, but I contend that it shouldn't, as it's more of a trick of your mental perception than anything else. I also think that there is self-selection for the people that are changed in such a way. People that want to view things through a unitive, universal, new age, "cosmic" perspective are more likely to see the experienced non-duality of enlightenment as validation of their perspective.

If meditation is the pathway to reach the highest level of psychological development, shouldn't it in principle be possible to move people from the earlier stages to the middle stages with the same method?


I don't have any references that I can reach for, and I'm not sure how good the studies actually are, or what they actually say, but my mild belief is that meditation does generally accelerate cognitive development at all stages.

Also, the "unitive" stage isn't just found in advanced meditators: "enlightenment is an accident, meditation makes you accident prone," as the saying goes. Not that I want to get into a discussion about classical Buddhist enlightenment. Also, Wilber does say that that meditative attainment and cognitive development can be orthogonal (see "Wilber-Combs lattice" concept).

Finally, "Meditation is just a way of practicing with your mind to experience reality in a specific way," is a really broad statement. There's lots of different kinds of meditation. The types of meditation that are probably most relevant here are not "state achievement and stabilization" types, but meditation techniques that make what your mind is already doing more transparent over time.

I think "mind functioning transparency" and associated metacognitive skill increases are a key part of how all this hangs together (cf. the so-called "construct aware" stage, for example). So, the above class of meditation protocols could accelerate the "ego level" line, and use of which would be correlated with but not necessary for hitting the so-called "unitive" stage.

But I do agree that descriptions of the "unitive" stage (by both the experiencers and the researchers) can be colored by all sorts of "spiritual" language and assumptions, and these assumptions have certainly influenced the path and interpretation of this research (and that influence is a mixed bag). But there is methodologically sound signal in there, and I think it's likely that there's an underlying structure to the unfolding of the higher (and lower) stages that can be independent of culture, meditation, and Buddhism. In any case, the nature/nurture/culture influence at each stage and the "ontological status" of these stages is still an open question, as are their utility and malleability.