Looking a little ahead into the future, I think LLMs are going to stop being focused on knowledgeable, articulate^[1] chatbots, but instead be more efficient models that are weaker in these areas than current models, but relatively stronger at reasoning, a pure-reasoner model. The rest will be bolted on via tool-use and other scaffolding^[2].

Retrospective Edit: A few weeks after this was published, Karpathy posted a very similar idea under the name "cognitive core", which seems to be catching on as popular phrase for this concept.

The Current Idiom is Inefficient

Chatbots make a great replacement for Google searches because they know a lot about everything. But that knowledge comes at a price. The majority of an LLM's parameters are thought to be spent on storing factual knowledge. Parameters are the key determinant of training and inference costs. It likely has a significant cost on data efficiency, and it's possible these extra parameters just hurt generalisation overall.

Memorisation is just not a feature that we need models to have when there are likely more efficient ways to get nearly the same capabilities.

We keep building bigger models which memorise more and more data largely because we don't know how not to. Every sentence of pre-training simultaneously encourages the model to memorise more factual information, and ekes it towards better high-level thinking and eventually reasoning ability.

But I think that technicality will fall soon, driving towards sleeker agents that fit inside a larger framework^[3].

The Technical Path to Pure Reason

The main blocker at the moment is that we don't know how to train models that have good capability without also training them to be great memorisers. There's also a lesser blocker of ensuring that such a model is reliably supplied in context with any information it needs, to compensate for not being able to furnish it itself.

Decoupling Reasoning Capability from Memorisation

While I cannot predict exactly what techniques are likely to work, it's clear there are several productive areas of current research that all lead in the same direction.

Reasoning RL. While I've been using "reasoning" in the colloquial sense, the biggest recent advance in reasoning capabilities comes from reinforcement learning techniques that encourage a reasoning style chain-of-thoughts. This technique is still new enough that there are likely remaining gains^[4]. 

Reasoning models often have very long thinking traces, which makes all those parameters dedicated to memorisation even more expensive.

Mixture of Experts can be viewed as an alternate take on pure reasoners. They cut the parameter count in use at once, bringing similar efficiency gains to separating memorisation. It's unclear at present if MoE can be fairly described as a separation, though.

Distillation allows smaller models to be trained from a larger model. The smaller model often loses some performance - it's not hard to imagine a training regime which preserves reasoning capabilities while losing memorisation capabilities.

Supplying the Pure Reasoning Model with Needed Knowledge

I hardly feel I need to spell this one out. Retrieval Augmented Generation, tool use and subagents are all standards of a modern LLM-based application. These techniques will probably improve over time.

Simultaneously, there is also furious work on architectural improvements to attention. As models get better at attending accurately in large context windows, in-context techniques like the above become relatively stronger compared with parameter-based memorisation.

Once you have pure-reasoning models, it will also make sense for those models to call out to other LLMs that are not as smart, but do have the wide knowledge base. Or possibly these will both be the same model, as a mixture-of-experts where the majority of the model isn't active most of the time.

Implications for Safety

I think this development will be good news, at least for AI control, not AI alignment. Current reasoners show chains of thought that are very useful for inspection. If only we could rely on them being a faithful, complete representation of thought.

If instead we have very smart models that are completely dependent on their scaffolding for providing the facts, then we essentially gain an additional human-readable window into what the LLMs are currently recalling (and thus, thinking about). And unlike chain of thought, it is much harder for pure reasoners to fake - they need this information loaded in order to perform the task.

1. ^{^}

   Most of this essay focuses on knowledge and memorisation rather than language and argumentation skills. It's plausible these and other skills will suffer the same fate.

2. ^{^}

   For example, Search-R1.

3. ^{^}

   The bitter lesson suggests that progress will come more from increases in compute than architectural changes like my suggestion. I'm not contradicting that. I just think we've reached the point where models are large enough that there is also value in trimming the fat, so we can spend that compute (and data) on the areas that are most useful.

4. ^{^}

   I've seen it suggested that RL only elicits existing capabilities. If true, then the ability to improve base models to be better reasoners will be limited to some fixed amount.