I disagree, see some relevant discussion here (me) and here (Dwarkesh).
40 gigabytes of text is about how much a person would say if they spoke nonstop for 8 hours a day, 7 days a week, for 250 years.
Preschoolers learn language much faster than that.
You can argue that vision transfers to language, but blind preschoolers also learn language much faster than that.
You can argue that hearing-bird-songs-or-whatever transfers to language but (1) really? (2) that doesn’t add much.
You can argue that the full data-heavy audio waveform (as opposed to just the words per se) makes the language learning task dramatically easier, but (1) that seems like the exact opposite of what we should expect, i.e. I’d assume that one of the hardest computational parts of language acquisition is learning to ignore the countless aspects of the auditory waveform over which the linguistic content is invariant, (2) deaf-blind preschoolers also learn language much faster than that.
In model-size terms, how large is a preschooler?
I’m not sure why you’re bringing up parameter count in a post about data efficiency. You can argue that LLMs are more data-efficient than brains, or you can argue that brains are more data-efficient than LLMs because brains have a higher parameter count, but it’s weird how you seem to be arguing both at once.
(My own opinion, stated without justification, is that brains are more data-efficient than LLMs for reasons unrelated to “higher parameter count”.)
Mmm, hadn't read your reply on Dissolving the Deep Learning Sample Efficiency Gap before. I'd been wondering about this.
The thing about deaf people is quite interesting.
I don't have a strong belief about any of this right now, but flagging a lot of the brain's data is kinesthetic/proprioception. (also, a ton of the brain's data is presumably "managing stuff inside the body", I'm not sure whether that is relevant to learning or not)
I'd guess there's some basic "ability to learn and combine concepts, at all" which is a key building block for visually recognizing mama, papa, recognizing the word "mama" and "pape", one-shotting the concept of "dog", learning to count, etc. How much data do humans take before they grok that?
Do there exist blind + deaf + nonproprioceptive people who seem smart (or, survive at all?)
Helen Keller looks like she lost sight/hearing at 19 months old which seems after my hypothesized "generalized concept-grokking" would form.
a lot of the brain's data is kinesthetic/proprioception
Sure! A proper accounting of “how many sensory bits” should certainly include all the senses, including proprioception, tactile, interoception, and so on. (But those are OOMs fewer bits than vision, if I recall correctly.)
Do there exist blind + deaf + nonproprioceptive people who seem smart (or, survive at all?)
Lacking proprioception is super-rare. The most famous example is Ian Waterman, who lost all proprioception at age 19 (they think he got a random minor infection, and by terrible luck, the immune system response happened to also false-positive upon his own proprioceptive nerves, or something like that). There was a neat little BBC documentary about him.
On a quick search, it seems like a congenital absence of proprioception is something that’s happened in like two documented cases ever, and those two were not blind.
Putting proprioception aside: I think congenital deaf-blindness is a thing (I heard that congenital rubella syndrome can cause it) and such people can indeed acquire language in childhood.
Putting proprioception aside: I think congenital deaf-blindness is a thing (I heard that congenital rubella syndrome can cause it) and such people can indeed acquire language in childhood.
The current theory I want to distinguish is "children get a fuck-ton-of-bits, in the first ~2 years, of some kind, which is when they get a generalized-concept-wrangling skill, which makes all subsequent learning much easier"
from
"children do that, but, don't require that many bits" (either because it comes hardwired somehow, or because they are just very good at learning)
from
"idk everything else."
For purposes of figuring out "apart from not having continual learning, how much less sample-efficient are LLMs than humans".
The "deaf/blind people still learn" thing does nudge me towards #2 over #1, but seems plausible that there's still a ton of proprioception bits that contribute substantively.
Also, I vaguely recall people missing a sense end up repurposing the brain that usually goes towards that sense to doing other stuff. Might be something like "the congenital deaf/blind children turn the spare brainpower towards eeking out more effective learning from proprioception."
(Not sure if this is framed right)
Children don’t learn language from reading a bunch of text. They learn it from people interactively teaching them, pointing at things, explaining words on demand, etc. Non-textual information like intonation and gestures is certainly useful for that process.
Let’s try an (thought) experiment where we lock a child in a small room and pass in sheets of paper covered in English text. Occasionally he can go to the backyard and experience nature. How long before he will read and write about nature?
Yet, an LLM can do this, even without direct experience.
I think that western culture vastly overstates the importance of deliberately teaching things to young children. There are cultures where “everyone waits until the child can talk sensibly before acknowledging its existence”, and the kids still learn to talk just fine. See Notes on "The Anthropology of Childhood".
I don’t think your thought experiment is quite right: if I understand correctly, LLMs get pretraining data of different modalities coupled together; they don’t have to discover on their own a correspondence between a text corpus and a completely disconnected vision corpus.
Anyway, it’s definitely hard to pin down an apples-to-apples comparison between human learning and LLM pretraining, because human learning is just a totally different thing from LLM pretraining. See “LLM pretraining magically transmutes observations into behavior, in a way that is profoundly disanalogous to how brains work”. LLMs have pretraining data, humans are more like RL agents in that there’s a training environment. A fun example is Nicaraguan Sign Language, or pidgins turning into creoles, where a group of kids who have never seen a grammatical language in their life simply create one. What would be the “sample efficiency” of that? They’re learning grammatical language from exactly zero samples!
As of today, I strongly believe that there is no known learning algorithm that can start from random initialization, spend the equivalent of 2 waking years roaming around and doing stuff in various virtual environments like VR & Minecraft, occasionally happening upon people who are talking to each other about whatever, and/or watching YouTube videos, such that the model winds up at the end with the linguistic and other competences of a human 4yo. You sure can’t train an LLM that way. How would it ever take intelligent action?
Maybe you’ll say: Well, we raise humans in an appropriate environment built for humans, and we “raise” LLMs in an appropriate environment built for LLMs. Each would do badly if put in the shoes of the other. And my response would be: Yeah, sure, whatever, I think it’s obvious that LLMs can do things that humans can’t and vice-versa, and I’d like to avoid a getting into a vague and pointless argument about which of them is “more impressive”. Those arguments tend to be divorced from any concrete question and unhelpful for understanding.
I wouldn’t say it’s about the “appropriate environment“ being different, I would say LLMs can actually learn language in a way that humans probably would not be able to do from scratch.
Do you think that a blind child who everyone ignored would learn to speak fluently? Intuitively I doubt it, though I’m not that confident.
But that is how LLMs learned to speak, until the introduction of multi-modal training. I believe that GPT-3 was trained on text only. The problem isn’t just establishing the right correspondences, it is that establishing correspondences would make learning language much easier by breaking harder-to-learn concepts down into easily grounded ones.
I also think that the evidence you’re raising about people in other cultures giving children less attention is pretty weak. It seems to mean relative to western parenting, versus “not at all” or “sparsely” in some statistical sense. Also I don’t want to dig through that entire post, when it seems maybe one paragraph is relevant.
It really doesn’t appear to me that there’s a straightforward comparison for language learning efficiency of LLMs versus humans, and probably both are better in certain senses.
learning problems can become drastically easier or harder based on representation or interaction protocol. Generally, I have the impression that you are too cavalier about brushing things under the rug as constant factor differences or lower order corrections that seem potentially quite important to me. i often find your conclusions plausible but your arguments not very convincing. perhaps in this case also, there is some underlying conclusion-generator that is stronger than the arguments given.
I would say LLMs can actually learn language in a way that humans probably would not be able to do from scratch.
Yeah I probably agree.
It really doesn’t appear to me that there’s a straightforward comparison for language learning efficiency of LLMs versus humans, and probably both are better in certain senses.
I agree here too. Like I said, it’s hard to pin down an apples-to-apples comparison between human learning and LLM pretraining, because human learning is just a totally different thing from LLM pretraining. Best to ask more specific questions.
(But I stand by my belief that the gist of the OP is mostly wrong and misleading.)
I also think that the evidence you’re raising about people in other cultures giving children less attention is pretty weak. It seems to mean relative to western parenting, versus “not at all” or “sparsely” in some statistical sense. Also I don’t want to dig through that entire post, when it seems maybe one paragraph is relevant.
Yeah my claim was “western culture vastly overstates…” which I stand by. The relevant sections of Notes on "The Anthropology of Childhood" include “How do children learn?” and “How children are expected to speak”.
It seems like (in those cultures) 1yo toddlers might often be supervised by somewhat-older siblings, peers, etc., and I’m a bit unclear on what kinds of interactions they’d have. (I doubt it’s “deliberately teaching”, except for things like yelling at toddlers not to wander off in dangerous areas (cf. the “Chewong” excerpt). But maybe there’s other kinds of playful interactions between toddlers and older kids?)
Anyway, I put in that summary quote (“everyone waits until the child can talk sensibly before acknowledging its existence”) because it’s from the author and I figure he would know. His tone throughout the book is pretty careful as opposed to sensationalizing and exaggerating, IMO.
(I read the whole book, but that was a decade ago so my memory is a bit hazy. Julia’s excerpts are good.)
Do you think that a blind child who everyone ignored would learn to speak fluently? Intuitively I doubt it, though I’m not that confident.
It’s a bit pointless to argue since we don’t know the answer. Oh well: My guess is yes (assuming they survive), because I think little blind kids, just like little sighted kids, are generally capable of learning how the world works, and what’s what, and learning intuitive physics and intuitive biology and so on. It’s restricted to tactile and auditory for the blind kids, and for all I know it might take somewhat longer, but they still get there. And so they’re still in a position to connect overheard words to relevant objects and situations, just like the sighted kids are. (Again, maybe it would take somewhat longer, but I’d guess not overwhelmingly longer.)
If people kept ignoring the kid, they would sure have a lot of unanswered questions, e.g. about how to identify some plant by touch if everyone else is talking about its color. But that would be after the kid is fluently talking about more basic things like rain and beds and inside and mud and fighting etc., that blind kids can easily develop an intuitive understanding of by autonomous nonverbal exploration and sensing.
What about testing the ability of neuralese/brainlike "LLMs" to learn? Current AIs, even including the presumably looped transformers like GPT-6 Astra, are CoT-based in the same sense as described in the section "Neuralese recurrence and memory" of AI-2027. I conjecture that a neuralese/brainlike LLM learns faster by virtue of being free from such constraints. The alternate explanation is that the human's genes bias the human brain towards learning some tasks, which I find unlikely since the human genes are hard to change and carry around 4 gigabits of information.
The text to speech conversion is a good point which I should have considered. I would still argue that humans have an easier time because they bootstrap a world model and then learn language on top of it, but this leans very far into conjecture.
I also still think it's incredible how effective transformers are considering their tiny size, but yes, that is unrelated to data efficiency. I framed the post as about data inefficiency because that is what most of the bad hominid metaphors focus on. My main goal was to attack those metaphors in general, because suggesting model capabilities are limited using hominid metaphors has rarely done any good for anyone.
Thanks!
If it’s true that “humans have an easier time because they bootstrap a world model and then learn language on top of it”, why would you contextualize that as, like, “…and so humans aren’t really data-efficient, that doesn’t count, it’s cheating”, rather than as “…and perhaps this is part of the explanation of how humans manage to be so data-efficient”?
suggesting model capabilities are limited using hominid metaphors has rarely done any good for anyone
I think it’s valid and useful for people to point out things that human brains can do that AI engineers can’t (yet) do, because human brains provide a convenient existence proof that the thing in possible at all. For example, (copying from a different comment) as of today, I strongly believe that there is no known learning algorithm that can start from random initialization, spend the equivalent of 2 waking years roaming around and doing stuff in various virtual environments like VR & Minecraft, occasionally happening upon people who are talking to each other about whatever, and/or watching YouTube videos, such that the model winds up at the end with the linguistic and other competences of a human 4yo. You sure can’t train an LLM that way. LLMs need (pre)training data, humans instead need a training environment. It’s very different.
But when (as above) someone points out a thing that human brains can do that AI engineers can’t (yet) do, I feel like you’re not treating that as an neutral observation about AI-algorithm-possibility-space, but rather as a front in the war over what we actually care about, which is whether we should be more impressed by human brains versus LLMs. And you’re fighting for the yay-LLMs side and they’re fighting for the yay-brains side. And indeed fighting this war is a popular activity on both sides. But that’s a waste of time, because this is a “meaningless argument” in the Scott Alexander sense. Really we should all be very impressed by both human brains and LLMs in their own ways, and then we should move on to arguing about more concrete and specific things.
On further reflection, the point I was originally trying to make with this post is that text token language modeling is more "data efficient" than objectives using world models/games/video data. My basis for this is that language modeling enabled the scaling era we are now in and still remains the only method with clear potential of achieving human-level intelligence. Furthermore it shows evidence of being able to adapt to robotic control and visual design and other non-text native domains better than trying to bootstrap on those modalities.
It's still possible that language modeling hits a wall, and that something more "human" ends up being more efficient for achieving AGI.
I think that my favorable comparisons of LLMs to humans were initially intended for rhetorical effect, but at some point I think I let myself slip into defending them as a position in a meaningless argument.
Trying to count the "human experience" naively as TB of footage ingested is a major mistake.
All footage is massively redundant - because images are spatially redundant, and videos are both spatially and temporally redundant. It's what video compression exploits. But even practical video compression is very far from information-theoretic limits. It only squeezes out the shallow redundancy patterns, ones that are easy enough to exploit with traditional algorithms.
AI training gets closer to the limit. Videos are nowhere near the "training signal per byte" of tokenized text, as a rule. It's a big part of why incorporating video data into pre-training in a useful fashion is so hard.
"Human experience" is even more redundant than that. Typical pre-training dataset footage is likely to cover different places, subjects and topics. Humans over time tend to inhabit self-similar places, perform self-similar routines and engage in self-similar interactions.
The "poverty of the stimulus" is very much in play. Humans get far more done with far less training signal. That's just true.
If you only consider pre-training, that is.
Now, what about in-context learning? The part that happens after the AI is already pre-trained? Or context distillation techniques that effectively feed in-context learning back into the weights?
AI closes the sample efficiency gap massively.
Human brain is best thought of as a partially pre-trained artifact. Nature did its own training run - millions of years of highly parallel search and optimization process. As such, human learning is always closer to "in-context learning + context distillation" than it is to "pre-training". We're only beginning to figure out what nature did there and how.
I agree with the first part of your comment, but about "in-context learning" - this is just really shallow to the point that I don't believe it's worth comparing to a bona fide "learning algorithm" at all. It's just what transformers do; every time they generate a new token, influences are propagated from prior generated tokens according to pathways learned by the model during training. If you provide the model a list of instructions, the attention mechanism will impart that information into later latent vectors, which condition the output tokens in such a way that they are likely to obey those instructions. But that only works because these signals slot into a specific pre-existing conceptual model encoded into the weights that is sufficient to interpret those instructions and retrieve them at precisely the right times. When a human learns how to navigate the world, or masters a discipline, or even works on a specific long-horizon project, they build new conceptual models that enable them to organize and use information effectively. LLM in-context learning does not achieve this. It resembles what we do when we recollect what we had for lunch, or consult an instruction manual. The only ways to build deeper knowledge into an LLM are pretraining and to a lesser extent, RL. Both of these are tremendously less efficient for LLMS than for humans, even when there's no reason humans should have any kind of evolutionary prior (for example the data efficiency for learning math or code).
I also disagree with the analogy between pretraining and evolution. Our DNA just doesn't contain that much information - less than 2 gigs! Almost of all that has no relevance for the brain at all. There's not really any room in that to customize how our brains function beyond overall structure and cell types. Certainly there's no way that could wire in any approximation to our faculties, which modern neural nets need billions of custom set parameters to emulate. That stuff has to come from learning after birth.
These concepts are discussed eloquently in many places by Steven Byrnes (who comments elsewhere on this page), and in this post by Dwarkesh.
Disagree on most of it. But the point on "DNA as a data bottleneck" is fascinating. Used to buy into that argument myself, but, not anymore.
ICL is an implicit learning algorithm that leverages a big, fat, nasty stack of priors (contained in trained model weights) to adjust transformer behavior in a very sample-efficient fashion.
Whether the "learned approximation of Solomonoff induction" framing is correct I don't know, but the power of the method is hard to deny. When relevant priors are available in weights, ICL kicks ass.
Now, it's not an optimizer that you can plug in place of AdamW and sic on any common ML task, no. But that would be a very narrow definition of what a "learning algorithm" is.
The stack of "in-context learning + context distillation" is one of the most powerful methods of injecting both knowledge and behaviors into LLMs. Yes, it's limited by the trajectories that you can get out of a pre-trainied policy - but in practice, so is RL. No one RLVRs off random weights. What makes context distillation a more niche technique is that it requires obtaining the context that would elicit the desirable trajectory - there is no cheap generalized "find the right context to make the LLM complete the task" operator. It's still used widely for things like constitutional alignment.
For LLMs, you have to pair ICL with CD to make the gains durable. But, in a way, the brain's inability to fully split inference and learning makes this kind of "inference-augmented learning" extremely natural.
Our DNA just doesn't contain that much information - less than 2 gigs! Almost of all that has no relevance for the brain at all.
Now that's the fun part!
Let's say that's true: brain-relevant DNA is under 200MB of information. And let's say that's also true: human brain works in an ICL-like priors-driven learning regime from day 0.
This is not a contradiction because low Kolmogorov complexity priors are a thing.
The priors brain uses to scaffold its learning process are compact enough to fit into DNA, but powerful enough to extract massive sample efficiency gains in its learning regime. The brain "hardcodes" priors that are broadly applicable, or give important domain-specific gains, or are hard to learn from the environment[1], and uses them first to dredge more knowledge up from the data available to it. Which then becomes the next layer of priors - and so on, and so on.
Do we know for sure if priors like this are real? Both useful and compact? Yes actually. The earliest work proving that is FractalDB - using synthetic fractal images generated by compact algorithms to inject useful structure into ResNet CNNs before applying any real data. This was explored further under the name of "Formula-Driven Supervised Learning" in image domain. They even compacted it further down in some papers.
For LM domain, the most striking paper would probably be "Training Language Models via Neural Cellular Automata", which applies a very similar setup to LLMs directly.
Note that those operate via synthetic training data, while what the brain does is likely to lean more into architecture, initialization, and - the one ML-inaccessible edge - its ability to shape its own data acquisition process. But priors are priors, and them being passed into data is just ML doing what's more convenient for ML.
Finding useful priors like this is incredibly hard, but evolution didn't skimp on the scale of its search process.
- ^
See: Baldwin effect
It seems that we disagree about how powerful ICL actually is.
I agree with what you're saying about ICL+distillation not being that different from RL in spirit. I just also believe that both of these are pretty limited in terms of what they can actually teach an LLM to do without appropriate priors encoded in the base model. This doesn't apply to humans! It seems really important that as we generate a trajectory this backreacts on our priors in a way that reshapes the future of that trajectory. This means that the trajectories we generate are way less limited by our priors. Rather than needing a bunch of external signal to select between parallel rollouts that comes from a verifier or a human selecting context, we can bootstrap ourselves into learning complicated concepts (including ones that nobody has figured out before).
I also agree with your model that we come prewired with exactly the right information that provides us with a learning algorithm that then stacks knowledge recursively. But I don't think that it's useful to compare this to ICL resulting from pretraining. For one thing whatever this algorithm is isn't buried in a deluge of other subject-specific priors. More importantly, the most powerful aspect of this algorithm is the fact that it can achieve sample-efficient distillation and generalization. This is exactly the part that needs to be tacked onto ICL with distillation over many rollouts!
Thanks for sharing the cellular automata paper! It's interesting, but I find the argument you're making to be unconvincing. If we take that paper at face value then what it suggests is that training on general syntactic rule-based data creates general priors related to downstream syntactic transmission i.e. induction heads. I don't dispute this, but the crucial advantage of human learning is the ability to build intricate semantic representations on the fly in a sample-efficient way. ICL makes use of specific pre-existing semantic representations, as well as at least somewhat general pre-existing syntactic representations, but I maintain that it cannot actually form new semantic structures. As a test case, I do not believe that you can remove all knowledge of e.g. quantum mechanics removed from the training data of an LLM, leave it with a quantum mechanics textbook and unlimited inference compute, and expect it to master quantum mechanics to the level of a capable human. I do not believe that you can do this even with in-context distillation or RL on quantum mechanics problems. I expect that this requires a vast amount of knowledge of quantum mechanics to be integrated into pretraining. This experiment is hard to run with commercial models, but it may be possible to do this with one of those vintage LLMs.
This doesn't apply to humans!
I think you overestimate the power of human learning in "inhuman" regimes. And underestimate LLMs. But mostly just overestimate humans.
In domains where "human priors" don't apply, sample efficiency of human learning takes a dive. See "Investigating Human Priors for Playing Video Games" for an instructive example - removing just the "surface level visual cues" from the games without changing the game logic dropped human performance by an order of magnitude. Goes to show that a lot of "human advantage" was resting on well made assumptions.
Even your previous examples of humans tackling the unnatural tasks of "math and coding" are overwhelmingly likely to benefit from priors for language, spatial reasoning and general abstract thinking. Like LLMs that were never exposed to code in training data can learn basic coding via ICL, seemingly by transfer from language priors. Which is remarkably similar to how books like "programming for children" try to teach them to humans.
If we take that paper at face value then what it suggests is that training on general syntactic rule-based data creates general priors related to downstream syntactic transmission i.e. induction heads. I don't dispute this, but the crucial advantage of human learning is the ability to build intricate semantic representations on the fly in a sample-efficient way.
That doesn't follow. You're trying to make a forced distinction that the LLMs themselves don't make.
It's a classic blunder. Humans try to slot things into neat little boxes, but ML-generated solutions are shaped more like "it's both, and also..."
Case in point: LLMs have no hard split between "syntactic" and "semantic" data. The same architectural units handle both - data types and abstraction levels are implicit and learned. Attention mechanism doesn't care - vectors are vectors. The same types of attention primitives can be employed by different heads at different depths, targeting different distances and operating on different "types" of data.
It's likely that the NCA runs from the paper manage to instill some "generally useful attention over sequence" primitives. Including some of the primitives that back ICL.
As a test case, I do not believe that you can remove all knowledge of e.g. quantum mechanics removed from the training data of an LLM, leave it with a quantum mechanics textbook and unlimited inference compute, and expect it to master quantum mechanics to the level of a capable human. I do not believe that you can do this even with in-context distillation or RL on quantum mechanics problems.
Now that is a very funny thing to say.
Talkie-1930, a "vintage LLM" trained on up-to-1930 data, is capable of writing some Python code via ICL out of the box. It also becomes more Python capable with RL.
There was also a line of experiments aimed at training GPT-1900 for better physics performance, via SFT on period-relevant physics documents and RL for better physics reasoning. Apparently, it does meaningfully improve its ability to make sense of post-1900 observations or discoveries.
And those are small models trained on small datasets. They're nowhere near the frontier juggernauts like whatever it is that OpenAI has sicced on research math. They can't possibly capture the entire breadth of human priors. They aren't trained for reasoning or agentic behavior, they aren't trained to work in groups. They still manage to push past their training dataset with ICL and RL.
Now, there is a possibility of data contamination. But given the nature of the datasets, LLMs would have to learn very sample efficiently for meaningful contamination to happen.
In domains where "human priors" don't apply, sample efficiency of human learning takes a dive. See "Investigating Human Priors for Playing Video Games" for an instructive example - removing just the "surface level visual cues" from the games without changing the game logic dropped human performance by an order of magnitude. Goes to show that a lot of "human advantage" was resting on well made assumptions.
This paper is really not the takedown of human sample efficiency that you're making it out to be. The conclusion is that when you deliberately confuse the visual language of a video game it takes humans ... twenty minutes to solve it consistently? That's nothing! Of course if you mess with the visual cues you make the challenge more time-consuming, since now the player has to learn more information in order to be able to solve it. The humans still do it far more sample efficiently than RL agents. Now of course the human advantage in terms of spatial priors is still pretty significant for this kind of game. Instead one can consider board games where the model is provided with hardcoded models of the board state and movement rules. Humans are still more sample efficient at learning these games by a significant margin, when measured in terms of games encountered during training (simulated or otherwise).
Even your previous examples of humans tackling the unnatural tasks of "math and coding" are overwhelmingly likely to benefit from priors for language, spatial reasoning and general abstract thinking.
Of course, but this benefit comes in layers. At the first layer, yeah, our priors (especially linguistic priors) allow rapid acclimation to the language of these domains. This is pretty similar to the advantage conferred by a familiar visual language in the video games experiment, and to what I model ICL as achieving for LLMs. At the next layer, we can apply our general capacity for "abstract thinking" (which you refer to as a prior, but I would refer to as an innate ability) to build rich internal models of the domain. This is a deliberative process that takes place over longer timescales, but it is still sample-efficient. I think it should be clear from experience what this is like when e.g. learning a subject in advanced math.
That doesn't follow. You're trying to make a forced distinction that the LLMs themselves don't make.
It's a classic blunder. Humans try to slot things into neat little boxes, but ML-generated solutions are shaped more like "it's both, and also..."
Case in point: LLMs have no hard split between "syntactic" and "semantic" data. The same architectural units handle both - data types and abstraction levels are implicit and learned. Attention mechanism doesn't care - vectors are vectors. The same types of attention primitives can be employed by different heads at different depths, targeting different distances and operating on different "types" of data.
You could easily say that the brain doesn't care about the difference between syntactic and semantic data either, because "everything is neuron activations". It would be missing the point, because there is a difference in the difficulty of organizing different kinds of information. It's much more nontrivial to work with novel semantic data than syntactic data because of the degree of specific abstract structure involved. Obviously these aren't neatly separated boxes, but there can still be a vast gap in difficulty between different tasks. At one extreme, with some general priors on language it's easy to recognize patterns in sequences like "the cat jumped over the moon, the dog jumped over the stars" and extend the pattern to, say, "the fox jumped over the sun". Attention mechanisms can trivially handle problems like this using general representations of grammatical patterns and categorical relationships between words. Towards the other end consider the problem of digesting the concepts of parallel transport and the Riemann curvature tensor. This involves the construction of a highly specific and abstract map connecting different ideas, which in humans is a product of what I called Layer 2 learning. Suppose you try to get an LLM to do this in-context by dropping in exposition on these ideas and having the model work from there. When generating each token in its reasoning trajectory, the model needs to route the proper information to the proper points through its attention mechanism in such a way as to accurately orient itself and figure out what to generate next. What priors could possibly be baked into the model weights that would permit this, if no information about these concepts was presented during training? The weights would have to implement a general system for taking in complicated unfamiliar ideas, organizing them, coordinating with existing ingrained knowledge, and drawing inferences within a single forward pass. And I see absolutely no reason to believe that even the most advanced LLMs do anything resembling this. Nobody seems to believe that in practice Claude Code is effectively a continual learner, and anecdotally OOD learning/adaptation remains very bad in interactions that I/other people I know have with these models.
Talkie-1930, a "vintage LLM" trained on up-to-1930 data, is capable of writing some Python code via ICL out of the box. It also becomes more Python capable with RL.
There was also a line of experiments aimed at training GPT-1900 for better physics performance, via SFT on period-relevant physics documents and RL for better physics reasoning. Apparently, it does meaningfully improve its ability to make sense of post-1900 observations or discoveries.
I've seen the Talkie experiment. Its ability to write python was extremely limited, and entirely explicable by Layer 1 learning. I have no complaints about the ability of LLMs to follow explicit instructions or make simple syntactic inferences (as the examples of the model editing code demonstrated). There is no indication that this can be pushed to any kind of significant software engineering capability. Likewise GPT-1900 is just not that impressive (especially given the prompts that it was spoonfed).
As far as I can tell the status quo in model training is in fact that to get results out of RL you need heavy bootstrapping from expensive mid-training because otherwise the model trajectories will just not be able to get anywhere useful. Beren Millidge's writing on this seems informative.
I don't think you're splitting complexity correctly. ICLing a high level abstract is not inherently more complex than ICLing a highly complex syntactic rule.
And the OOM loss from visual priors is just a demonstration of the principle. If the priors weren't extremely important for sample efficient learning, there will be no drop. If a way to get one OOM of drop was found by attacking the most obvious priors, there's likely at least one more OOM somewhere in there.
Towards the other end consider the problem of digesting the concepts of parallel transport and the Riemann curvature tensor. This involves the construction of a highly specific and abstract map connecting different ideas, which in humans is a product of what I called Layer 2 learning. Suppose you try to get an LLM to do this in-context by dropping in exposition on these ideas and having the model work from there. When generating each token in its reasoning trajectory, the model needs to route the proper information to the proper points through its attention mechanism in such a way as to accurately orient itself and figure out what to generate next. What priors could possibly be baked into the model weights that would permit this, if no information about these concepts was presented during training? The weights would have to implement a general system for taking in complicated unfamiliar ideas, organizing them, coordinating with existing ingrained knowledge, and drawing inferences within a single forward pass.
Sure. LLMs do that. They do that routinely.
Humans love doing "let me introduce a concept, and then explain how it works, and then use it a whole bunch" in their texts. So if ICL is able to pick up a concept from text and figure out how to connect it to adjacent concepts, and to what concepts it best connects to, it would reduce loss on any text that contains patterns like this.
Sometimes those concepts repeat across texts and can be reliably memorized, but sometimes they're one-offs from obscure science papers or works of fiction or blog posts. And sometimes different authors understand different concepts differently, so memorization of a concept is not a full substitute for being able to ICL it. So it's a generally useful LM capability - one that different LLMs develop, to different degrees. The fit will not be perfect, but it only ever needs to be good enough.
"A single forward pass" you're just technically wrong about. The LLM in pre-training is depth-limited but not width-limited, because of how the data flows layer to layer. And then autoregressive reasoning at inference time allows for a partial release from the depth limit too.
As far as I can tell the status quo in model training is in fact that to get results out of RL you need heavy bootstrapping from expensive mid-training because otherwise the model trajectories will just not be able to get anywhere useful.
There's a difference between what's possible and what's practical. If you had infinite RL compute, it's possible that you could get to today's RL destinations with just RL, RL and more RL. Just steering the trajectories in a useful direction one RL step at a time. But if feeding pointed SFT data into the pipe slashes compute spend by a factor of 10? Yeah, SFT em.
It's a big part of why Chinese labs try so hard to distill from frontier models. That's their "expensive mid-training".
Nobody seems to believe that in practice Claude Code is effectively a continual learner, and anecdotally OOD learning/adaptation remains very bad in interactions that I/other people I know have with these models.
Claude Code is absolutely a continual learner, with a twist. ICL is an approximation of continuous learning. It's how models explore solution spaces, make discoveries, etc. It's what powers things like OpenAI's math monsters. If those models weren't able to pile new abstractions upon new abstractions within a context window, they wouldn't go far in research math.
Don't underestimate the expressive power of KV vectors and attention primitives. Attention can do a lot. You can even "train" KV cache entries - "prefix tuning" - and often to similar ends as full model SFT.
But the "twist" is that without weight updates, the learning depth is still bounded by model's own prior knowledge, context length and attention routing capacity. It's continual learning tied to a post with a leash.
The "ICL + CD" chain has CD in it for a reason - you want a mechanism to feed the ICL'd knowledge into a durable medium. You want the new data to be able to leave the attention system and extend the priors. But we don't know how to stabilize a formulation like this in a general way - even if we were willing to pay for it in compute.
I don't think you're splitting complexity correctly. ICLing a high level abstract is not inherently more complex than ICLing a highly complex syntactic rule.
I was talking about difficulty, not complexity. These are not the same thing. Syntactic rules can in principle be given arbitrary complexity, but one has a clear set of instructions to follow in order to implement them. As long as an LLM can implement the steps of a rule, it should in principle be able to compose them sequentially by tracking the sequence with attention. Whether or not it can do so in practice for arbitrarily complex rules is another question, but let's suppose that it can. This does not imply that it has the power to ICL and apply flexible abstract structures the same way. The difference is there are far fewer indicators to predict how the information needs to be routed through the model, when and how to generalize it or integrate with other information, etc. The model's weights were not engraved with the abstract relationships needed to do so.
And the OOM loss from visual priors is just a demonstration of the principle. If the priors weren't extremely important for sample efficient learning, there will be no drop. If a way to get one OOM of drop was found by attacking the most obvious priors, there's likely at least one more OOM somewhere in there.
I object to the framing of this paper as a disruption of the "sample efficiency" of learning. Messing with the visual language of the game just increases the scope of things that need to be learned in order to master the game, so it's obviously going to take longer. That doesn't mean that it actually affects the sample efficiency of the learning process.
So if ICL is able to pick up a concept from text and figure out how to connect it to adjacent concepts, and to what concepts it best connects to, it would reduce loss on any text that contains patterns like this.
That doesn't mean it's an ability that is usefully learned by the training process! I think part of our disagreement is about the degree of novelty required to serve as evidence of such a robust learning ability. This is muddled by the sheer amount of information and associations that show up in the training data. But I still come across instances where the advanced models fail to internalize and make use of novel concepts that show up in paper drafts, when experts in my field have no difficulty doing so.
Claude Code is absolutely a continual learner, with a twist. ICL is an approximation of continuous learning. It's how models explore solution spaces, make discoveries, etc. It's what powers things like OpenAI's math monsters. If those models weren't able to pile new abstractions upon new abstractions within a context window, they wouldn't go far in research math.
The main thing that the models are widely agreed upon to not be good at doing is precisely what you're describing, by the name of "theory building". Now I have my gripes with this framing, but the creation of new abstract perspectives just isn't what is powering the models' problem solving. The correct characterization seems to be more like "stochastic walk where individual steps are directed by prior-generated human abstractions". On the other hand I think the lack of abstraction is in many ways responsible for how garbage the presentation of these results is. Even when new insights can be inferred from the output, the model rarely seems to have had any reflective access to them.
"A single forward pass" you're just technically wrong about. The LLM in pre-training is depth-limited but not width-limited, because of how the data flows layer to layer. And then autoregressive reasoning at inference time allows for a partial release from the depth limit too.
I'm aware that LLMs are only width-limited by the length of the sequence. It's still a single forward pass! CoT (or looped layers or whatnot) do not alleviate the fundamental issue, which is that at each token the model needs to look at everything that came before, including whatever it wrote down, internalize the structure of all of it, and decide what direction to go in before it writes down the next token. I do agree that Claude Code, with all harnessing and internal reasoning tokens, is an "approximation" to a continual learner. The approximation is just lossy in critical ways. If you gave it a concept-learning task like the one I suggested, I believe that the agent would at first be able to follow along, take notes, and navigate the more superficial aspects of the conceptual structure (the kinds of things that can be converted without difficulty into instructions). But if it is set to work on more difficult applications and generalizations, there will be some points where the agent will need to pivot the CoT into a new direction. Maybe it needs to translate the setup of a question into a form where the concepts it was supposed to learn apply, maybe it needs to question some aspect of the framework. How can the agent figure out what this direction should be? Well, at the relevant token, the agent will read all previous tokens and process them with attention heads and MLP blocks. It to internally organize this information into the right picture, reflect on it, and intuit the right direction to go in. My hypothesis is that without enough specific knowledge encoded in its weights, the agent will generally get lost and fail to orient itself correctly at such junctures. These failures will compound and it will go off the rails or in circles, and it will not be able to achieve the kind of fluid mastery that a capable human would be able to. I think these failure modes are pretty well-known at this point (albeit not in perfectly controlled conditions), and I myself see them pretty often.
Tl;dr Ok we've written a lot about this back and forth. Your contention is, as I understand it, that ICL is powerful but constrained by the inability to get the juicy stuff back into the weights. Context distillation can fix this in principle, but we don't know how to do this well enough to turn it into an unbounded process. My contention is that actually ICL is additionally constrained by lack of a robust reflection mechanism that can isolate that juicy stuff and make sense of it fluidly. So improved context distillation isn't going to be enough on its own to get a human-like learner. That would require a yet-to-be-seen ability to reflect while thinking, token-by-token.
I would contest "X is a very special magical ability that LLMs fundamentally don't have, and not yet another thing that LLMs already do, and are getting better at" for just about any given X.
Your "new concepts" stuff isn't that special. I don't see a gap there that would require a "fundamental new ability" to cover.
If the dataflow graph doesn't expressly forbid it, LLMs can learn to do it. If LLMs can learn to do it, and it helps next token prediction some, they probably did learn it already. This happened far too many times - "world models", "introspection" and more. And a lot of it was driven by people underestimating the expressiveness of attention mechanisms.
The gap that I actually see is continuous learning itself - but even that could be papered over with things like increased ICL capabilities, extended context window and agentic context management, or agentic self-fine-tune.
I would say that I do see a gap in what attention mechanisms are capable of, and that this is indeed caused by limitations of the data flow structure. I also think that the examples of LLMs seeming to develop capabilities that people did not expect them to develop are largely explained by a vast increase in subject specific data incorporated into training, rather than actually developing general capabilities. The failure modes of LLMs have not really changed in character. It's true that they occur less frequently, especially in the domains that have been targeted most heavily by the corporations. But they haven't gone away, and are still prevalent in many disciplines.
I don't know enough to articulate a rigorous positive view, but I think important things are missing here. My sketch:
One thing is to learn that data at all, the transformer has to learn in parallel in massive batches. It does not filter out the noise otherwise. I do think locally it's more sample efficient than humans, one thing I'd bring up is in-context learning, where, IMO it's been long more sample efficient than the average human being (recall, say, your time in school, how much drilling students needed in order to learn algebra, how many times they needed to review material to memorize simple transferable facts that easily fit in their working memory, requiring no additional chunking.) But then it runs out of the 1M context window, and it's done, and not even the bitter lesson with this setup can come up with an algorithm to make it work, there is no "sleep" or whatever is the process that contextually weighs information for novelty, filters it for noise, and moves it to the persistent slow weights. The setup doesn't have a way to do that, the scale and data needed for it to meta-learn continual learning by autocompacting on 100M context with AdamW + GRPO might just not be feasible. Our optimization objective is not the long-term coherence of an intelligent animal, but performance in a ton of noisy and isolated settings all averaged out to rip out the signal.
The post makes much more sense if you interpret "data efficiency" to mean "how many parameters does it take to get to a given level of effectiveness". This is of course not what anyone else means by data efficiency. I suggest the author has simply misunderstood what the word means, and has proceeded to argue eloquently against a proposition nobody in fact holds.
That's a good take, and you're being charitable in giving a preschooler only 27 terabytes of data, counting video alone. You could have accounted for all the raw input from every sensory receptor. You could have accounted for the "pretraining" we get from evolution, through our genes.
But you could even have given up on counting altogether, because what exactly are we supposed to count? Raw data? Even if we could estimate it, would it be meaningful? Text, taken not as an image but as a sequence of symbols, isn't raw data : it's encoded, compressed information, the residue of cognitive work done by other minds. It's comparing apples to oranges.
So what metric, then? Kolmogorov complexity? Bennett's logical depth, which might capture how much prior computation a text embeds ? The "pretraining" we get from evolution has also, arguably, an enormous logical depth.
I think your conclusion is the wise one : we don't know, and we must be careful when comparing data efficiency between humans and AIs.
(Edit 2026-10-15: The main point I was trying to make is that discrete token language modeling is a very alien objective, and it is the only objective that has allowed us to achieve current intelligence levels with current levels of compute. World model/game based approaches that are more intuitively "human" in some ways have not yielded comparable results. Therefore discrete token language modeling may be very "data efficient" in some sense.
This is a comparison of ML objectives rather than architectures or algorithms. If you compare current ML architectures to humans on "data efficiency" for learning from language, ML loses sorely.
I think that trying to draw implications about models based on comparisons to humans is very tricky; this post intentionally does this at points for rhetoric, which may have been more trouble than it was worth.)
Humans are really bad at comparing themselves to models.
Take, for example, Leopold's preschooler graph:
Set aside, for a moment, concerns of scaling and RSI, and meditate: what characteristics does GPT-2 share with a preschooler?
I can't come up with anything else, because these two entities are almost completely disjoint. Is GPT-2 capable of bipedal locomotion, recognizing its mother's voice, or naming people by face? Does a preschooler learn from eight million scraped web pages sourced from Reddit?
What exactly is a preschooler "trained" on? Thousands of hours of "multimodal data". Assuming that a preschooler sees at 720p, and is awake for 12,000 hours by the age of 3 (about 11 hours a day), they have consumed at least 27 terabytes of video data at streaming-quality compression, or about 3.6 petabytes uncompressed, not to mention audio and sensorimotor data.[1]
In model-size terms, how large is a preschooler? Trillions of parameters, maybe. Beren Millidge's estimate, which assumes only ~1,000 synapses per neuron, puts the whole brain at an effective 10-30 trillion parameters.
So surely GPT-2 is much more "data efficient" than humans, wielding "preschooler"-level control of language with a measly 1.5 billion parameters and 40 gigabytes of text! And what about GPT-3, which, with only 175 billion parameters and about 1 terabyte of text, was capable of prose like:
Only a truly prodigious "elementary schooler" would be able to continue Janus' thought so eloquently. And yet this notion of LLM data inefficiency persists. "Frontier data efficiency lab" Flapping Airplanes, on their front page, writes:
Does this make any sense when you consider that most humans in the history of the species lived never "seeing" a single "text token" by adulthood, or in their entire lives? What do most human "samples" actually consist of? Yet a thesis like this is enough pretense for a $180M seed round and an estimated valuation of $5B.
Even OpenAI's Dan Selsam indulges this notion of data inefficiency, in an otherwise lucid tweet on AI risk:
It's natural that Dan, a capabilities researcher watching the marginal gains from text data go down, would say this. Also, nothing rules out enormous algorithmic advances that squeeze more out of the same quantities of data and compute.
But we really don't understand how humans scale, at all, and so it's absurd to claim that models are "data inefficient" compared to humans. At small scale, the discrete text token-based transformer architecture seems more efficient than anything we've ever seen.[2]
What about at large scale? As Dan concludes, as far as dangerous capabilities are concerned, it's efficient enough.
Yann LeCun does a similar calculation for a 4-year-old's optic nerve and lands at 10^14–10^15 bytes.
It's almost as if human language is a "fully general highly compressed mathematical isomorphism of human cognition". In that case Yann is also wrong, because language is the shortcut to human-level intelligence at current levels of compute; you can just slap robot capabilities and visual intuition on later.