A stated goal of many of the frontier AI labs is to automate science, or at least large portions of it. AlphaFold’s architects won the Nobel Prize in 2024 for enormous advances in automated solutions to protein folding. That was a system tailored to a specific domain. The holy grail is a generalized system that performs the entire cycle of scientific discovery across any domain, largely or entirely unsupervised.

Heavyweight Jeff Dean just exited Google to found Discovery Loop to focus entirely on this problem.

But is it doable? Maybe that’s a few different questions rolled into one. Is it doable with current and near-future extensions of LLMs and agents? Is it doable without embodiment? That is, do we need generally intelligent robots working in physical labs in order to automate science? Or maybe what we should be asking is what can we accomplish in pursuit of science without that dependency?

Dean’s vision is a propose-run-evaluate loop, thousands of experiments at a time. More experiments, run faster. Billions of dollars are flowing to that vision. I think there’s at least one other fruitful path.

I’ve seen several articles over the last few months poo-pooing the idea that LLM-based technology can make any real substantive progress in any area of science.

An article on Chaotropy last week called "Non-Physical Intelligence Has A Ceiling" argues that AI without a sensory or motor interface to reality can only work off a pile of static recorded measurements and can’t run experiments. The subtitle kind of says it all: “Reasoning alone cannot predict the chaotic physical world. Without a sensory and motor interface to reality, non-physical AI will not deliver the scientific and technological breakthroughs we expect.”

Tom Zahavy at DeepMind has a paper called "LLMs can't jump." His claim is that LLMs lack the sensory grounding needed for manipulative abduction, and without that you can't do the step where new axioms get generated. So you get optimization within a framework somebody else built, but never a new framework. He’s careful not to claim categorically that embodiment is the only answer, but he strongly suggests that’s the best way forward:

To build an AI capable of true invention, we must therefore move beyond systems that merely read scientific literature to systems that can perceive the physical world. The emergence of physically consistent World Models offers a pathway to a synthetic laboratory. By enabling agents to run counterfactual simulations—to experience the physical consequences of a thought experiment—we may finally mechanize the feedback loop between intuition and logic.

I’m going to argue against these views. I’m going to argue that there’s likely a lot of scientific mileage to be gotten out of existing data, and I’m going to use my favorite scientist and favorite theory to illustrate why.

Charles Darwin changed the way we view life and its origins forever with the book-length explanation of the theory of natural selection in 1859.

How did he get there?

I’m going to focus on two very large influences on his ideas: uniformitarianism and selective breeding.

Darwin was influenced by the work of geologist Charles Lyell, who advocated a view of geology known as uniformitarianism. For most of human history, the prevailing view was some variant of the idea that one or more supernatural beings had created the earth and all its geological features (mountains, valleys, etc.) pretty much as they were, and not too long ago in time. Lyell thought the evidence pointed to a different conclusion. The earth was very old, much older than just about everyone thought. And things like caves, diamonds, and plateaus weren’t designed and instantiated whole cloth. They were formed via gradual change through natural processes acting over very long stretches of time.

Darwin was a naturalist. He sought to explain the origins and diversity of life. Up until then, the prevailing view was that species had been designed and created by one or more supernatural beings pretty much as they were, and not too long ago. He wasn’t very satisfied with this explanation, so began from the assumption that maybe there was a perfectly sensible explanation that didn’t involve gods. In other words, a scientific explanation. And he was inspired by Lyell’s views on geology, which included the concepts of:

• Deep Time, spans far longer than human minds are generally capable of contending with
• Gradual Change, alterations so small and subtle they are virtually imperceptible to humans
• Naturalistic Mechanisms, physical forces and rules that work slowly across time to guide incremental change

In other words, Darwin drew analogies between geological formations and living things. A species of tree was like a mountain. All the complexity at work in the machinery of an oak, from its growth from a single acorn to photosynthesis, was potentially explained by a set of straightforward rules acting incrementally over vast periods of time.

Another enormous influence on Darwin’s formation of the theory was selective breeding, or what he called artificial selection. Since prehistory, humans have made deliberate choices about the differential reproduction and survival of other species: dogs, pigeons, corn, rice, etc. That process has generated enormous diversity in much shorter spans than geological time.

Dogs are one of the clearest examples. All dogs descend from an extinct population of grey wolves. A single species spans the morphological range from chihuahuas to mastiffs, encompassing all sorts of shapes, colors, and sizes of features.

How the hell did that happen? Well, Darwin surmised that there were rules governing such modifications over time, and by analogy came to the conclusion that the very same mechanics at work in selective breeding were at play in nature. Only instead of the humans doing the selecting, it was the suitability of the environment in which the organism lived and tried to reproduce.

Anyone who has read On the Origin of Species is familiar with the fact that Darwin spends an enormous amount of time explaining artificial selection and drawing this exact parallel.

Now, I’m not proposing that analogy alone got Darwin to his full theory (the ideas of Malthus were another big piece of the puzzle). What I am suggesting is that these ideas were crucial, and that importantly, they did not require new experimentation. Darwin was an experimentalist. He even bred pigeons. But in order to synthesize the theory of evolution by natural selection, he didn’t need to. All the pieces and all the data were lying around, just waiting to be put together.

Had an agentic AI system existed prior to the publication of Darwin’s work, with full access to all published science, could it have put everything together into a coherent theory? Well, that’s a pretty interesting thought experiment. But one thing it makes absolutely, concretely clear, is that in this particular case, one of the most important and impactful frameworks in all of science did not require new experimentation to formulate the core ideas. Of course it spurred millions of experiments after the fact, and required decades to fill in the details of the underlying mechanisms and the actual unit of inheritance. But Darwin’s insights were accessible without additional scientific interventions. Plenty of interventional work had already been done. Mountains of data already existed, albeit carried out in different domains. Darwin repurposed all that work and applied it to life.

So the argument I keep seeing against AI producing novel scientific work is the constraint that in order to produce novel frameworks and theories we need to run new interventionist experiments in the physical world.

Our old friend Darwin is a stark counterexample. Again, not necessarily because his entire thought process revolved around analogical thinking without personal experimentation, but because at the time he came up with the idea, it was completely possible to get there via analogical thinking without personal experimentation. Copious data for geology had been sufficiently collected. Copious data from artificial selection had been sufficiently collected. No new experiments had to be run in the physical world in order to derive the theory of evolution by natural selection.

So I would assert this: Current agentic AI systems can accomplish an enormous amount of valuable scientific work by leveraging existing cross-domain experimental and historical data.

And wouldn’t you know, this is already an active area of research. Here’s a paper out of Stanford from just a few months ago: Unlocking LLM Creativity in Science through Analogical Reasoning.

For our task of open-ended solution generation, we define analogical reasoning as the ability to generate analogies from a research problem p to cross-domain research problems that exhibit similar relational structure. We implement AR as a two-step process consisting of “extraction” and “search” steps. The extraction step extracts analogical components from p and generates analogies to other domains. The search step uses these analogies to search for novel solutions to p.

In other words, they present a problem, extract salient features from the problem, then look for those features in problems in other domains. Then they actually use an approach from the analogous domain to try to achieve new results in the original domain.

For example, their first case study was a perturbation effect prediction from systems biology. They extracted relevant features of the problem, searched across domains, and found a good match in economics:

Similar to how cell lines respond differently to perturbations, consumers also respond differently to economic policy changes. In both settings, aggregate measures can miss the individual heterogeneity of behavior. Using this analogy, AR proposed applying the economics approach of finite mixture models (FMM) to perturbation effect prediction. This finite mixture model approach achieves state-of-the-art performance on the MMD PCA distributional distance metric with a value of 0.42 (see Figure 3A; lower MMD is better). In addition, when combining the finite mixture model approach with the Latent Additive (LA) baseline from the paper, we further improve performance over the vanilla LA baseline on MMD PCA from 1.65 to 0.13 (a nearly 13x improvement) while maintaining comparable performance on prediction accuracy metrics (Cosine LogFC, Cosine LogFC rank, RMSE mean, RMSE mean rank).

In another case, their LLM-based algorithm found an analogous mapping between an open biology problem and chess:

AR discovered a very interesting analogy from chess tactics, in which the value of a chess piece varies based on its position on the board. In the oligonucleotide sequence space, therapeutic properties are similarly determined by the position of certain motifs in the sequence. Using this analogy, AR discovered the chess solution of piece-squared tables (PST) with tapered evaluation, which interpolates value tables for certain pieces between opening and endgame phases based on the game phase.

The main target of the work is mode collapse. Normally you hand an LLM an open-ended problem and you get the same set of ideas. We see this across domains, things like jokes and creative writing. The key is to jostle them out of their comfortable, bland interpolated space into the nooks and crannies they wouldn’t normally consider. These researchers use analogical reasoning to accomplish that task, and it shows promise. It’s a smaller move than Darwin’s. They demonstrated technique swapping across domains. Darwin mapped entire fields with decades of work. But they’re the same general idea: leverage existing, analogous research and results.

The point is, there are lots of ways to do science. Not all of them include producing a hypothesis out of thin air and devising new experiments in a wet lab to validate them. For many open questions, we can probably look for analogous features in other domains and leverage existing data to confirm our inferences.

Like Darwin either did or at least could have done. I’m not arguing that this approach is going to outperform physical experimentation. I’m arguing against the impossibility of progress by LLMs. All that takes is one counterexample. Darwin is a very strong one. The AR paper is an additional proof-of-concept.

This is exactly the sort of task that LLMs should be extremely good at. Brilliant scientists can have insights and make mappings across domains, but only to the extent they’re familiar with work in other unrelated domains. An LLM is trained on millions of papers across every known domain of human knowledge. That doesn’t mean the problem is trivial to them. It’s still a matter of making the actual mappings.

But this idea that we have to have robots working in labs to move the ball forward is just wrong.

I’m not saying we don’t need experimentation. I’m saying LLMs can do a lot of heavy lifting in science, and as far as I can tell no one is really diving in on that approach and building such a system.