In domains where "human priors" don't apply, sample efficiency of human learning takes a dive. See "Investigating Human Priors for Playing Video Games" for an instructive example - removing just the "surface level visual cues" from the games without changing the game logic dropped human performance by an order of magnitude. Goes to show that a lot of "human advantage" was resting on well made assumptions.
This paper is really not the takedown of human sample efficiency that you're making it out to be. The conclusion is that when you deliberately confuse the visual language of a video game it takes humans ... twenty minutes to solve it consistently? That's nothing! Of course if you mess with the visual cues you make the challenge more time-consuming, since now the player has to learn more information in order to be able to solve it. The humans still do it far more sample efficiently than RL agents. Now of course the human advantage in terms of spatial priors is still pretty significant for this kind of game. Instead one can consider board games where the model is provided with hardcoded models of the board state and movement rules. Humans are still more sample efficient at learning these games by a significant margin, when measured in terms of games encountered during training (simulated or otherwise).
Even your previous examples of humans tackling the unnatural tasks of "math and coding" are overwhelmingly likely to benefit from priors for language, spatial reasoning and general abstract thinking.
Of course, but this benefit comes in layers. At the first layer, yeah, our priors (especially linguistic priors) allow rapid acclimation to the language of these domains. This is pretty similar to the advantage conferred by a familiar visual language in the video games experiment, and to what I model ICL as achieving for LLMs. At the next layer, we can apply our general capacity for "abstract thinking" (which you refer to as a prior, but I would refer to as an innate ability) to build rich internal models of the domain. This is a deliberative process that takes place over longer timescales, but it is still sample-efficient. I think it should be clear from experience what this is like when e.g. learning a subject in advanced math.
That doesn't follow. You're trying to make a forced distinction that the LLMs themselves don't make.
It's a classic blunder. Humans try to slot things into neat little boxes, but ML-generated solutions are shaped more like "it's both, and also..."
Case in point: LLMs have no hard split between "syntactic" and "semantic" data. The same architectural units handle both - data types and abstraction levels are implicit and learned. Attention mechanism doesn't care - vectors are vectors. The same types of attention primitives can be employed by different heads at different depths, targeting different distances and operating on different "types" of data.
You could easily say that the brain doesn't care about the difference between syntactic and semantic data either, because "everything is neuron activations". It would be missing the point, because there is a difference in the difficulty of organizing different kinds of information. It's much more nontrivial to work with novel semantic data than syntactic data because of the degree of specific abstract structure involved. Obviously these aren't neatly separated boxes, but there can still be a vast gap in difficulty between different tasks. At one extreme, with some general priors on language it's easy to recognize patterns in sequences like "the cat jumped over the moon, the dog jumped over the stars" and extend the pattern to, say, "the fox jumped over the sun". Attention mechanisms can trivially handle problems like this using general representations of grammatical patterns and categorical relationships between words. Towards the other end consider the problem of digesting the concepts of parallel transport and the Riemann curvature tensor. This involves the construction of a highly specific and abstract map connecting different ideas, which in humans is a product of what I called Layer 2 learning. Suppose you try to get an LLM to do this in-context by dropping in exposition on these ideas and having the model work from there. When generating each token in its reasoning trajectory, the model needs to route the proper information to the proper points through its attention mechanism in such a way as to accurately orient itself and figure out what to generate next. What priors could possibly be baked into the model weights that would permit this, if no information about these concepts was presented during training? The weights would have to implement a general system for taking in complicated unfamiliar ideas, organizing them, coordinating with existing ingrained knowledge, and drawing inferences within a single forward pass. And I see absolutely no reason to believe that even the most advanced LLMs do anything resembling this. Nobody seems to believe that in practice Claude Code is effectively a continual learner, and anecdotally OOD learning/adaptation remains very bad in interactions that I/other people I know have with these models.
Talkie-1930, a "vintage LLM" trained on up-to-1930 data, is capable of writing some Python code via ICL out of the box. It also becomes more Python capable with RL.
There was also a line of experiments aimed at training GPT-1900 for better physics performance, via SFT on period-relevant physics documents and RL for better physics reasoning. Apparently, it does meaningfully improve its ability to make sense of post-1900 observations or discoveries.
I've seen the Talkie experiment. Its ability to write python was extremely limited, and entirely explicable by Layer 1 learning. I have no complaints about the ability of LLMs to follow explicit instructions or make simple syntactic inferences (as the examples of the model editing code demonstrated). There is no indication that this can be pushed to any kind of significant software engineering capability. Likewise GPT-1900 is just not that impressive (especially given the prompts that it was spoonfed).
As far as I can tell the status quo in model training is in fact that to get results out of RL you need heavy bootstrapping from expensive mid-training because otherwise the model trajectories will just not be able to get anywhere useful. Beren Millidge's writing on this seems informative.
(Edit 2026-10-15: The main point I was trying to make is that discrete token language modeling is a very alien objective, and it is the only objective that has allowed us to achieve current intelligence levels with current levels of compute. World model/game based approaches that are more intuitively "human" in some ways have not yielded comparable results. Therefore discrete token language modeling may be very "data efficient" in some sense.
This is a comparison of ML objectives rather than architectures or algorithms. If you compare current ML architectures to humans on "data efficiency" for learning from language, ML loses sorely.
I think that trying to draw implications about models based on comparisons to humans is very tricky; this post intentionally does this at points for rhetoric, which may have been more trouble than it was worth.)
Humans are really bad at comparing themselves to models.
Take, for example, Leopold's preschooler graph:
Set aside, for a moment, concerns of scaling and RSI, and meditate: what characteristics does GPT-2 share with a preschooler?
I can't come up with anything else, because these two entities are almost completely disjoint. Is GPT-2 capable of bipedal locomotion, recognizing its mother's voice, or naming people by face? Does a preschooler learn from eight million scraped web pages sourced from Reddit?
What exactly is a preschooler "trained" on? Thousands of hours of "multimodal data". Assuming that a preschooler sees at 720p, and is awake for 12,000 hours by the age of 3 (about 11 hours a day), they have consumed at least 27 terabytes of video data at streaming-quality compression, or about 3.6 petabytes uncompressed, not to mention audio and sensorimotor data.[1]
In model-size terms, how large is a preschooler? Trillions of parameters, maybe. Beren Millidge's estimate, which assumes only ~1,000 synapses per neuron, puts the whole brain at an effective 10-30 trillion parameters.
So surely GPT-2 is much more "data efficient" than humans, wielding "preschooler"-level control of language with a measly 1.5 billion parameters and 40 gigabytes of text! And what about GPT-3, which, with only 175 billion parameters and about 1 terabyte of text, was capable of prose like:
Only a truly prodigious "elementary schooler" would be able to continue Janus' thought so eloquently. And yet this notion of LLM data inefficiency persists. "Frontier data efficiency lab" Flapping Airplanes, on their front page, writes:
Does this make any sense when you consider that most humans in the history of the species lived never "seeing" a single "text token" by adulthood, or in their entire lives? What do most human "samples" actually consist of? Yet a thesis like this is enough pretense for a $180M seed round and an estimated valuation of $5B.
Even OpenAI's Dan Selsam indulges this notion of data inefficiency, in an otherwise lucid tweet on AI risk:
It's natural that Dan, a capabilities researcher watching the marginal gains from text data go down, would say this. Also, nothing rules out enormous algorithmic advances that squeeze more out of the same quantities of data and compute.
But we really don't understand how humans scale, at all, and so it's absurd to claim that models are "data inefficient" compared to humans. At small scale, the discrete text token-based transformer architecture seems more efficient than anything we've ever seen.[2]
What about at large scale? As Dan concludes, as far as dangerous capabilities are concerned, it's efficient enough.
Yann LeCun does a similar calculation for a 4-year-old's optic nerve and lands at 10^14–10^15 bytes.
It's almost as if human language is a "fully general highly compressed mathematical isomorphism of human cognition". In that case Yann is also wrong, because language is the shortcut to human-level intelligence at current levels of compute; you can just slap robot capabilities and visual intuition on later.