When people outside the AI field hear about synthetic data for the first time, they often have one of two opposite intuitions. Either they assume that AI-generated data should be totally useless for training, or they imagine it should be possible to bootstrap all the way to superintelligence that way. Why neither intuition is correct is surprisingly subtle.

If the naive approach to synthetic data (i.e., just telling GPT-5 to generate a bunch of text and then training GPT-6 on it) worked, we would have general superintelligence already, much like the narrow superintelligence we have in board games like chess and Go. The problem is that, in order to be useful, training data has to encode information about patterns in the real world that the model doesn't know yet. If the model doesn't know those patterns yet, it can't generally put them into synthetic data that it creates.

So the big challenge for AI labs is to figure out ingenious ways of making synthetic data work even when it seemingly shouldn't. It's sort of analogous to how aeronautical engineering isn't antigravity—it's all about finding clever ways of making heavier-than-air objects fly, even though a simplistic understanding of gravity suggests they can't. In essence, generating useful synthetic data requires leveraging some kind of asymmetry between the information a model understands and the information content of what it can create. If the model knows enough to (with the right kind of highly specific engineering) put information in its outputs, but doesn't yet deeply understand that information, training on those outputs can distill that information into a form the model can grasp more thoroughly.

One major asymmetry stems from the fact that it's usually easier to verify the quality of outputs than to generate them. Another is that if a model spends a lot more compute on reasoning, it can act much smarter than its base-level capabilities allow. A third is that random exploration of potential solutions sometimes finds lucky insights. Together, these factors mean that it's often feasible to have an AI produce a lot of data, curate the very best samples, and then train the next version on those. For example, mathematical proofs can be written in the Lean language, which lets them be cheaply and automatically verified. By scaling up test-time compute for reasoning quality and stochastically attempting many possible proof strategies, a model might generate 990 proof attempts that fail and 10 that succeed. The verifier system converts the winners into training signal that distills those insights back into the base model.

Humans experience something very similar with talk therapy or Socratic teaching. Sometimes you know things on some level, but can't quite put them into words in a way that lets you reason about them. A skilled therapist or teacher can elicit that information from you. And then, when you hear what you just said in the session or the seminar (i.e., your synthetic data), you're able to make better use of the insight that was hidden inside you. Once you have mastery over that insight, you may gain dim intuitions about harder questions that can subsequently be elicited to help you master those as well.

These analogies also suggest why synthetic data isn't sufficient to create broadly superintelligent AI. Verbalizing ideas can help you crystallize vague insights that were already in your brain, but it can't teach you brand-new information about the outside world. No Socratic professor can draw out of you the capital of Kyrgyzstan or the mass of the top quark if you haven't first learned those facts independently. In the same way, synthetic data can help AI digest and master information, but training frontier-level models still requires ongoing contact with external reality.

The discourse about whether LLMs are the right paradigm for AGI is getting hung up on the wrong question. Scientists like Gary Marcus and Yann LeCun argue that pure scaling of today's models won't get to AGI. I think they're right. But so do all the frontier labs. Leaders at every frontier lab are on the record saying they expect we're 1-2 major architectural or algorithmic breakthroughs away. This is mainly a vibes war with people talking past each other.

So everyone agrees that we need more than pure scaling. The interesting question is: will the first AGI be directly descended from today's models, or will a radically new paradigm be needed? 

I think the labs are correct that the first AGI will likely be something transformer-like plus a couple other things bolted on, as opposed to something in a whole different clade from the AI of 2026.

This implies that the research happening right now, including safety and alignment work, is likely building directly toward AGI. And that although regular AI safety will have important differences from AGI safety, progress on one can feed into the other.

Aeronautics offers an analogy. The two dominant early paradigms for high-altitude flight were lighter-than-air balloons and rockets. And as late as the 1930s, balloons had reached greater altitudes than rockets ever had.

Imagine someone back then had naively proposed that one day humans could fly to the moon by building an absolutely enormous hydrogen-filled dirigible—pure scaling. The correct answer would have been: "Sorry, buddy, you're barking up the wrong tree. The laws of physics don't allow that. It's just not a matter of how big the airship is."

Now imagine someone else back then proposing that we'll get to the moon with a single-stage rocket. He'd be just as wrong as the balloonist about whether his exact idea will reach lunar orbit. But we wouldn't say he was wasting his time. Many of the lessons learned by single-stage rocket pioneers during the '30s and '40s directly contributed to the multistage rockets that eventually got humans to the moon. 

My expectation is that 2026 AI research isn't a balloon-style dead end, but more like single-stage rocketry on the way to the multistage rockets that will fly all the way to the moon of AGI.

Many people assume that the hard part of interpreting AI/ML papers is following the formulas, proofs, or other mathematical formalisms. 

Yes, the symbols are often opaque, but there are great free resources online that can bring you up to speed quickly and clarify how all the parts of a Greek-letter sandwich fit together.

And absent that, if you plug a paper into an LLM, it can usually walk you through exactly what's happening and translate the math into surprisingly accessible natural-language intuitions (or even interactive visual demos!).

The much harder and subtler task is judging whether a paper is using the right formalism—whether the quantitative framework is pointing accurately at the real-world problem the abstract advertises. 

Most scientists can figure out if the math makes internal sense, but the harder question ("Is this formalism optimal for the problem?") often requires deep knowledge of the tools of a given subfield or sub-subfield. And there's not an AI scientist alive who can keep up with the details of every sub-subfield that might someday be relevant to their own work. 

In addition, the harder question often requires a lot of zoomed-out thinking. Sometimes this takes deep theory—a motivating logic that connects the paper's problem to higher or lower levels of abstraction. A lot of it is commonsense reasoning about the real-world problem the paper is trying to solve and whether the authors properly understood it and accurately characterized it.  

For almost two years after AI did a decent job checking formulas, it was pretty terrible at this harder question. 

But for the first time with Mythos/Fable class models, it often does a very good job. Not yet better overall than the best human scientists on its own, but good enough to be a valuable research partner.

This will be a significant accelerator of AI progress, because models can digest and synthesize more research and more diverse research than any human scientist could, and thus facilitate faster diffusion of good ideas to the people who can use them best.