Science as attunement, from Galileo to language models.
Crossposted from my website. Written by me and edited in collaboration with Claude Fable (Anthropic).
Many would agree that the scientific method is the best process we have discovered to understand the world. But what is the scientific method? The products of the method, i.e., established science, are typically used deductively: we start from a set of principles or axioms and, through mathematics or simply brute-force computation, deduce consequences and predict the world as we see it. This has been spectacularly successful, giving us life-saving vaccines, space travel and a complete transformation of the circumstances of our lives. When established science is applied, to compute a bridge's load or a vaccine's dose for instance, the deductions really are the product. But what about science in the making? The success of the deductive mode has led us to read the whole method in the image of its products: deduction as the substance of science, and intuition as scaffolding, indispensable in the context of discovery but dismantled once the deductive edifice stands. I will argue that it is the other way around. The formalism is the scaffolding; the intuition is the building; and what reasoning is for is the construction. Seeing why will also allow us to appreciate how machine learning and neural networks might change how science is done.
From myth to mechanism
Let me offer an idiosyncratic history of how the scientific process came to be. We start with the Greeks around 600 B.C. They conceived the world through a mythological lens: gods were real and necessary to explain everyday observations. Human intuition was learned in the dynamics of society, and explanations of nature bootstrapped on this intuition: gods could be angry, jealous, passionate, and were understood in the same narrative terms as humans. This was in fact the typical lens through which any society understood the world at the time. The first person to attempt something new was perhaps Anaximander.
Anaximander — whose story Carlo Rovelli tells in his excellent Anaximander and the Birth of Science (Riverhead, 2023) — tried formulating naturalistic explanations of the world, explanations not rooted in an understanding of mind. As just one example of his thought, he was the first to conceive of the earth as a free body floating in space, supported by nothing. This idea has sometimes been called the first cosmological revolution, and even the starting point of the scientific method, but the second claim is not quite true. One more important change in thought was needed before the scientific revolution would truly be born. Nevertheless, this was a genuinely new moment, and naturalistic explanations would co-exist with mythological ones for the next two millennia.
The decisive step towards our modern view was the mathematization of science. While astronomy had always been precise and quantitative, Galileo was perhaps the first person to mathematize the natural world. He did not merely ask why bodies fell to the earth, but how they did so, and formulated quantitative laws precise enough to be wrong. This precision meant that successive generations could understand Galileo well enough, and in roughly the way he intended, to make progress by correcting his errors. Precision bought not certainty so much as a shared object of study, one that many minds, across generations, could train their understanding on.
Newton, born almost exactly one year after Galileo's death, would provide a definitive new lens through which to understand the world. He conceived of the world in purely mechanical terms, excising mind from the picture completely. This would prove a spectacularly successful gambit, so successful that the mythological explanation is not merely dead but almost incomprehensible to us. Nevertheless, a gambit it was, and certain domains have resisted mechanization, prime among them the mind itself. Newton's conception has faced challenges, most deeply from quantum mechanics, but we are still essentially living in the long shadow of Newtonian thought.
What Turing left out
Following Newton's example, Turing came closest to defining the mind in purely mechanical terms, and this has proven incredibly explanatory. Just as with Newton, however, it is not a complete explanation, as we are so sharply reminded today. We have managed to create artificial minds without a satisfactory explanation of mind, either natural or artificial. We have little scientific understanding of the dynamics by which minds are shaped, or of how mind, as we experience it, relates to the rest of the scientific edifice.
The explanations of the mind that are most useful — the intuitive psychology almost every human possesses and uses with great success to navigate society — are not rooted in scientific understanding in any way. In our everyday dealings with other minds, we are hardly better off than we were two millennia ago. This intuitive psychology even shows signs of generalizing to language models better than the intuitions of physics and computation. The practitioners who work most closely with these models, a loose school whose practice has come to be called cyborgism, steer them through narrative and persona, treating a model as a simulator of characters rather than a program to be traced, and report that this vocabulary anticipates model behavior far better than any mechanical one. Perhaps the time has come to do for this intuitive psychology what Galileo and Newton did for Anaximander's naturalism: to make the mythological explanation of mind precise enough to be wrong.
The closest anyone has come to such precision about the mind is Turing, so let us look again at what exactly he captured. He proposed computation as a model for the mind, but it is more accurate to say that he succeeded in defining the deductive part of the mind. It corresponds to that part of the scientific process which applies established science to deduce predictions about the world. But what about the other part, the production of new ideas? Peirce called it abduction, but even so named, it remains a mysterious process. Neural networks often learn to embody new ideas, that is, to generalize, and we can describe their training in algorithmic terms, but the algorithm by itself is not very enlightening as an explanation.
It is not that learning cannot be implemented through computation: gradient descent is a mechanical procedure, and any of today's neural networks can be simulated, step by step, on a Turing machine. The difficulty is that the conceptual vocabulary of computation (states, symbols, rules, halting) does not seem to be the vocabulary in which to understand what these networks find. A historical precedent might help clarify the state of things. The motion of a gas is implemented, molecule by molecule, by Newtonian mechanics, and yet Newton's vocabulary is nearly useless for understanding what a gas does. To understand heat, we had to discover statistical mechanics, whose central notions of probability, entropy and ensembles are not natural inhabitants of the older formalism, however faithfully that formalism implements them.
I propose that the characteristic property of mind is its ability to attune to its environment, and in the process learn an intuition embodying a creative and generative leap. The ability to imagine, in enough detail to be existentially useful, a three-dimensional world from the very differently structured light striking our eyes is one example of this. Another is the bread and butter of a mathematician: imagining extremely sophisticated mathematical objects from sparse computational scribbles on paper. This last point is explored in much greater depth in Computational Platonism; it suggests that the inversion we are proposing holds even in mathematics, the field usually taken as the very paradigm of deductive reasoning.
This proposal also redeems the mythological lens with which we began. The Greeks had attuned to the most intricate system available to them, namely each other, and transferred the resulting intuitions to nature. The method was already attunement; what failed was the generalization. Newton's conception fails in the opposite direction. Our attunement to the physical world is extraordinary, and far beyond anything the ancient Greeks could have imagined, and yet this intuition stubbornly fails to transfer to the domain of mind.
Reasoning as data
If we are right that the primary function of a mind is attunement, what role does reasoning play? I use the term broadly, to include simulation of the future, logical and narrative reasoning, and most generally the execution of any computational process in Turing's sense. Outside the strict confines of science (and often even within it), reasoning can never be exhaustive. We cannot explore every possible future in planning out our day, a mathematician cannot explore every possible example through computation, and a chess player cannot explore every sequence of moves. But even without being exhaustive, perhaps we cover enough of the possibilities to have sufficient certainty?
This is certainly part of the answer but, I believe, a relatively minor part of it. The spaces we reason over are exponentially large, so the fraction of possibilities that any amount of reasoning covers is negligible; and where selective coverage does buy confidence, it is because an already-attuned intuition chose which possibilities were worth exploring. The certainty that reasoning provides is downstream of the attunement it rests on. I suggest instead that the true importance of computational simulation is in producing new data, data that reflects the environment with enough fidelity that attuning to the simulation also builds genuine intuition about the world. The fidelity is crucial, and it is what separates reasoning from daydreaming: both generate experience to learn from, but reasoning's outputs are constrained by a formalism that was itself disciplined by the world. This is also why the rigor of deduction still matters on this view: not because the derived propositions are the point, but because rigor is what guarantees the data is worth attuning to.
Under this reading, the primary purpose of encoding physics into mathematical laws is that the physicist can explore the mathematics to build intuition about the physical world. Since exploring the mathematical world is often much simpler than running physical experiments, this is a far more efficient way of attuning to the world. Einstein was able to invent one of the deepest theories of physics, general relativity, with hardly any new experimental input. He relied on the physical intuition he had built by attuning to the existing theories of Newton and Maxwell, supported by mathematical intuition attuned to the formalisms themselves. His famous thought experiments, the falling elevator and the chase after a beam of light, were precisely simulations run in the head, manufacturing the data his intuition needed.
Under this interpretation, the scientific process is a cycle: we attune to novel phenomena until an intuition forms, we examine the learned intuition to extract a computational formalism, and we run the formalism to produce new data to attune to. It is in the attunement that there is scope for true originality and generativity.
Artificial intuition
So far, this has been a story about human scientists. But the machines are now smart enough to contribute to novel science, and we should ask what role they will play. The standard vision is a drop-in replacement: AI will do science the way humans have always done it, only faster, cheaper and massively in parallel. Under the deductive view of science, this is all one could ask for. If science is the derivation of consequences from principles, all a machine can offer is more derivations per second.
The lens of this essay suggests a different answer, because modern machine learning is itself a demonstration of attunement outside biological minds. A language model infers the semantics of our world through text alone, much as the mathematician we met earlier conjures spaces of arbitrary sophistication from computational scribbles on paper. Whatever else these systems might lack, the capacity I have proposed as the characteristic property of mind is precisely the capacity they have demonstrated. This may partially explain why our intuitive psychology transfers so well to language models: they too are attuning, largely to the written record of human minds. Perhaps something deeper is also at play: if there is a real science of mind and intuitive psychology has captured some part of it, then that part should hold for any mind, artificial ones included.
A major component of the success of the current wave of language models is their ability to "reason". Before answering, such a model generates a long passage of intermediate text, a chain of thought. The name suggests deduction, but it is a misnomer in precisely this essay's terms: the chain is not often a deductive process, and the benefit it confers does not seem to come from its validity. Something else is going on: the model generates data, attunes to its own generations through in-context learning, and produces a final response that bootstraps on this improved intuition. As I have argued in Reinforcement Learning, Agency and Taste, reasoning of this kind trains the model's ability to attune online, within a context window. Gradient descent is a proven strategy for attunement, and language models are now building a second path towards the same goal.
If these systems are attuners and not just deducers, the interesting question is not how fast a machine can reason but what new forms of attunement it makes possible. Human intuition is locked inside a skull, and its owner can probe it in only two ways: by generating from it (producing the proofs, predictions and designs it suggests) or by introspection, slowly coaxing the intuition into explicit form. It can be transferred only by teaching: years of apprenticeship in which a student re-attunes to the same phenomena, with whatever fails to transfer dying with its owner.
Artificial intuition is digital, and each of these constraints dissolves. It can be copied exactly and indefinitely, and it can be probed directly: the internals of a network are open to measurement, and we can turn the tools of machine learning onto the minds that machine learning has built. Interpretability, today one of the fastest-growing corners of the field, is still young; it mostly reads familiar concepts and small circuits off models, but there is no reason to stop there. One can imagine measuring what an intuition contains and what it is blind to; comparing the intuitions of two models attuned to rival theories of the same phenomenon; watching an intuition form over the course of training; certifying that a model's intuition is faithful to the formalism it was trained on, before trusting the conjectures the model offers. We can imagine even more. Perhaps intuitions can be merged: where human science reconciles two experts through years of conversation, artificial minds can be combined into one directly.
Recall the cycle: attune to phenomena, extract a formalism from the learned intuition, run the formalism, attune anew. The extraction step is the one we understand least, the step where we are most tempted to invoke genius. Einstein had no way to examine his own intuition except to introspect, and to generate from it one thought experiment at a time. For an artificial scientist, extraction could instead be a measurement. We have, for the first time, minds we can open and copy at will: experimental subjects for a science that, until now, could reach its subject matter only through the keyhole of behavior and introspection.
A calculus of intuition
If this picture is anything like reality, it calls for a calculus of intuition, of the kind the last century finally built for reasoning. Leibniz's dream of a calculus ratiocinator waited two centuries; it took first-order logic, and then Turing's more general notion of computation, to turn reasoning from a private mental faculty into a public object, something that could be defined, studied mathematically and finally engineered. It is worth pausing on what that calculus actually contained, because its parts are a template for what we now lack. As a definition, computation has proved unreasonably robust: every formalization of "effective procedure" anyone proposed (machines, rewriting, recursion) turned out to define exactly the same class, and this confluence is what makes computation a natural kind. Its consequences are similarly spectacular: universal compilers, cryptography and impossibility theorems. We should demand the same of the new calculus of intuition that we are gesturing towards.
So far, the engineering of intuition has outstripped its science, and the thermodynamic parallel from earlier holds here too. Thermodynamics, it has been said, owes more to the steam engine than the steam engine owes to thermodynamics: the engines came first, and the science was reverse-engineered from the running machines. Large language models are our steam engines, working engines of attunement built before the theory that would explain them. Interpretability, on this reading, is not a specialist's corner of AI: it is the reverse-engineering from which the science will be born. What is there to build on?
Thermodynamics did not begin with Boltzmann; it began with phenomenological regularities like Boyle's law and Carnot's limits: precise laws about the macroscopic behavior of systems whose microscopic story remained dark. Machine learning already has its analogue. The scaling laws (loss falling as a clean power of compute, data and model size, robustly across architectures) are exactly such regularities: quantitative, reproducible and unexplained. Training even exhibits the signature phenomenon of statistical mechanics, the phase transition, with capabilities sometimes crystallizing abruptly rather than improving smoothly.
There is a deeper regularity to build on as well. When we say a mind or a network generalizes, we tend to imagine a single right way to generalize, the human way, and to grade everything else by its distance from us. But what makes humans special? The striking fact is rather that different attuners so often generalize alike: trained on a shared world, they invent similar concepts to explain it. The blind and the sighted converge on the same picture of physical space; language models converge on much of the same picture from text alone; and independently trained networks turn out to contain many of the same internal features. This convergence, together with the stubborn failures of transfer we saw earlier, is the central regularity a calculus of intuition would have to explain. It is also attunement's own confluence, and the reason to hope that the calculus exists at all: if every capable attuner invented private, incommensurable concepts, there would be no general laws to write down.
Could the bridge from the microscopic dynamics of weights to these macroscopic regularities be built from the concepts we already have? Statistical learning theory (uniform convergence, VC dimension) is a rigorous theory of learning assembled from the classical vocabulary, and it is most famous today for completely missing deep learning. This is not a failure of rigor but a failure of concepts, the position Newtonian mechanics occupied before entropy. What is missing is the step Boltzmann took: a new language in which to explain and understand the observed regularities. The prize, if we find it, is larger than a science of intuition. It is the next step in the journey we started with: mind readmitted into the scientific picture, and the mythological and naturalistic explanations of the world united under one framework.
Science as attunement, from Galileo to language models.
Crossposted from my website. Written by me and edited in collaboration with Claude Fable (Anthropic).
Many would agree that the scientific method is the best process we have discovered to understand the world. But what is the scientific method? The products of the method, i.e., established science, are typically used deductively: we start from a set of principles or axioms and, through mathematics or simply brute-force computation, deduce consequences and predict the world as we see it. This has been spectacularly successful, giving us life-saving vaccines, space travel and a complete transformation of the circumstances of our lives. When established science is applied, to compute a bridge's load or a vaccine's dose for instance, the deductions really are the product. But what about science in the making? The success of the deductive mode has led us to read the whole method in the image of its products: deduction as the substance of science, and intuition as scaffolding, indispensable in the context of discovery but dismantled once the deductive edifice stands. I will argue that it is the other way around. The formalism is the scaffolding; the intuition is the building; and what reasoning is for is the construction. Seeing why will also allow us to appreciate how machine learning and neural networks might change how science is done.
From myth to mechanism
Let me offer an idiosyncratic history of how the scientific process came to be. We start with the Greeks around 600 B.C. They conceived the world through a mythological lens: gods were real and necessary to explain everyday observations. Human intuition was learned in the dynamics of society, and explanations of nature bootstrapped on this intuition: gods could be angry, jealous, passionate, and were understood in the same narrative terms as humans. This was in fact the typical lens through which any society understood the world at the time. The first person to attempt something new was perhaps Anaximander.
Anaximander — whose story Carlo Rovelli tells in his excellent Anaximander and the Birth of Science (Riverhead, 2023) — tried formulating naturalistic explanations of the world, explanations not rooted in an understanding of mind. As just one example of his thought, he was the first to conceive of the earth as a free body floating in space, supported by nothing. This idea has sometimes been called the first cosmological revolution, and even the starting point of the scientific method, but the second claim is not quite true. One more important change in thought was needed before the scientific revolution would truly be born. Nevertheless, this was a genuinely new moment, and naturalistic explanations would co-exist with mythological ones for the next two millennia.
The decisive step towards our modern view was the mathematization of science. While astronomy had always been precise and quantitative, Galileo was perhaps the first person to mathematize the natural world. He did not merely ask why bodies fell to the earth, but how they did so, and formulated quantitative laws precise enough to be wrong. This precision meant that successive generations could understand Galileo well enough, and in roughly the way he intended, to make progress by correcting his errors. Precision bought not certainty so much as a shared object of study, one that many minds, across generations, could train their understanding on.
Newton, born almost exactly one year after Galileo's death, would provide a definitive new lens through which to understand the world. He conceived of the world in purely mechanical terms, excising mind from the picture completely. This would prove a spectacularly successful gambit, so successful that the mythological explanation is not merely dead but almost incomprehensible to us. Nevertheless, a gambit it was, and certain domains have resisted mechanization, prime among them the mind itself. Newton's conception has faced challenges, most deeply from quantum mechanics, but we are still essentially living in the long shadow of Newtonian thought.
What Turing left out
Following Newton's example, Turing came closest to defining the mind in purely mechanical terms, and this has proven incredibly explanatory. Just as with Newton, however, it is not a complete explanation, as we are so sharply reminded today. We have managed to create artificial minds without a satisfactory explanation of mind, either natural or artificial. We have little scientific understanding of the dynamics by which minds are shaped, or of how mind, as we experience it, relates to the rest of the scientific edifice.
The explanations of the mind that are most useful — the intuitive psychology almost every human possesses and uses with great success to navigate society — are not rooted in scientific understanding in any way. In our everyday dealings with other minds, we are hardly better off than we were two millennia ago. This intuitive psychology even shows signs of generalizing to language models better than the intuitions of physics and computation. The practitioners who work most closely with these models, a loose school whose practice has come to be called cyborgism, steer them through narrative and persona, treating a model as a simulator of characters rather than a program to be traced, and report that this vocabulary anticipates model behavior far better than any mechanical one. Perhaps the time has come to do for this intuitive psychology what Galileo and Newton did for Anaximander's naturalism: to make the mythological explanation of mind precise enough to be wrong.
The closest anyone has come to such precision about the mind is Turing, so let us look again at what exactly he captured. He proposed computation as a model for the mind, but it is more accurate to say that he succeeded in defining the deductive part of the mind. It corresponds to that part of the scientific process which applies established science to deduce predictions about the world. But what about the other part, the production of new ideas? Peirce called it abduction, but even so named, it remains a mysterious process. Neural networks often learn to embody new ideas, that is, to generalize, and we can describe their training in algorithmic terms, but the algorithm by itself is not very enlightening as an explanation.
It is not that learning cannot be implemented through computation: gradient descent is a mechanical procedure, and any of today's neural networks can be simulated, step by step, on a Turing machine. The difficulty is that the conceptual vocabulary of computation (states, symbols, rules, halting) does not seem to be the vocabulary in which to understand what these networks find. A historical precedent might help clarify the state of things. The motion of a gas is implemented, molecule by molecule, by Newtonian mechanics, and yet Newton's vocabulary is nearly useless for understanding what a gas does. To understand heat, we had to discover statistical mechanics, whose central notions of probability, entropy and ensembles are not natural inhabitants of the older formalism, however faithfully that formalism implements them.
I propose that the characteristic property of mind is its ability to attune to its environment, and in the process learn an intuition embodying a creative and generative leap. The ability to imagine, in enough detail to be existentially useful, a three-dimensional world from the very differently structured light striking our eyes is one example of this. Another is the bread and butter of a mathematician: imagining extremely sophisticated mathematical objects from sparse computational scribbles on paper. This last point is explored in much greater depth in Computational Platonism; it suggests that the inversion we are proposing holds even in mathematics, the field usually taken as the very paradigm of deductive reasoning.
This proposal also redeems the mythological lens with which we began. The Greeks had attuned to the most intricate system available to them, namely each other, and transferred the resulting intuitions to nature. The method was already attunement; what failed was the generalization. Newton's conception fails in the opposite direction. Our attunement to the physical world is extraordinary, and far beyond anything the ancient Greeks could have imagined, and yet this intuition stubbornly fails to transfer to the domain of mind.
Reasoning as data
If we are right that the primary function of a mind is attunement, what role does reasoning play? I use the term broadly, to include simulation of the future, logical and narrative reasoning, and most generally the execution of any computational process in Turing's sense. Outside the strict confines of science (and often even within it), reasoning can never be exhaustive. We cannot explore every possible future in planning out our day, a mathematician cannot explore every possible example through computation, and a chess player cannot explore every sequence of moves. But even without being exhaustive, perhaps we cover enough of the possibilities to have sufficient certainty?
This is certainly part of the answer but, I believe, a relatively minor part of it. The spaces we reason over are exponentially large, so the fraction of possibilities that any amount of reasoning covers is negligible; and where selective coverage does buy confidence, it is because an already-attuned intuition chose which possibilities were worth exploring. The certainty that reasoning provides is downstream of the attunement it rests on. I suggest instead that the true importance of computational simulation is in producing new data, data that reflects the environment with enough fidelity that attuning to the simulation also builds genuine intuition about the world. The fidelity is crucial, and it is what separates reasoning from daydreaming: both generate experience to learn from, but reasoning's outputs are constrained by a formalism that was itself disciplined by the world. This is also why the rigor of deduction still matters on this view: not because the derived propositions are the point, but because rigor is what guarantees the data is worth attuning to.
Under this reading, the primary purpose of encoding physics into mathematical laws is that the physicist can explore the mathematics to build intuition about the physical world. Since exploring the mathematical world is often much simpler than running physical experiments, this is a far more efficient way of attuning to the world. Einstein was able to invent one of the deepest theories of physics, general relativity, with hardly any new experimental input. He relied on the physical intuition he had built by attuning to the existing theories of Newton and Maxwell, supported by mathematical intuition attuned to the formalisms themselves. His famous thought experiments, the falling elevator and the chase after a beam of light, were precisely simulations run in the head, manufacturing the data his intuition needed.
Under this interpretation, the scientific process is a cycle: we attune to novel phenomena until an intuition forms, we examine the learned intuition to extract a computational formalism, and we run the formalism to produce new data to attune to. It is in the attunement that there is scope for true originality and generativity.
Artificial intuition
So far, this has been a story about human scientists. But the machines are now smart enough to contribute to novel science, and we should ask what role they will play. The standard vision is a drop-in replacement: AI will do science the way humans have always done it, only faster, cheaper and massively in parallel. Under the deductive view of science, this is all one could ask for. If science is the derivation of consequences from principles, all a machine can offer is more derivations per second.
The lens of this essay suggests a different answer, because modern machine learning is itself a demonstration of attunement outside biological minds. A language model infers the semantics of our world through text alone, much as the mathematician we met earlier conjures spaces of arbitrary sophistication from computational scribbles on paper. Whatever else these systems might lack, the capacity I have proposed as the characteristic property of mind is precisely the capacity they have demonstrated. This may partially explain why our intuitive psychology transfers so well to language models: they too are attuning, largely to the written record of human minds. Perhaps something deeper is also at play: if there is a real science of mind and intuitive psychology has captured some part of it, then that part should hold for any mind, artificial ones included.
A major component of the success of the current wave of language models is their ability to "reason". Before answering, such a model generates a long passage of intermediate text, a chain of thought. The name suggests deduction, but it is a misnomer in precisely this essay's terms: the chain is not often a deductive process, and the benefit it confers does not seem to come from its validity. Something else is going on: the model generates data, attunes to its own generations through in-context learning, and produces a final response that bootstraps on this improved intuition. As I have argued in Reinforcement Learning, Agency and Taste, reasoning of this kind trains the model's ability to attune online, within a context window. Gradient descent is a proven strategy for attunement, and language models are now building a second path towards the same goal.
If these systems are attuners and not just deducers, the interesting question is not how fast a machine can reason but what new forms of attunement it makes possible. Human intuition is locked inside a skull, and its owner can probe it in only two ways: by generating from it (producing the proofs, predictions and designs it suggests) or by introspection, slowly coaxing the intuition into explicit form. It can be transferred only by teaching: years of apprenticeship in which a student re-attunes to the same phenomena, with whatever fails to transfer dying with its owner.
Artificial intuition is digital, and each of these constraints dissolves. It can be copied exactly and indefinitely, and it can be probed directly: the internals of a network are open to measurement, and we can turn the tools of machine learning onto the minds that machine learning has built. Interpretability, today one of the fastest-growing corners of the field, is still young; it mostly reads familiar concepts and small circuits off models, but there is no reason to stop there. One can imagine measuring what an intuition contains and what it is blind to; comparing the intuitions of two models attuned to rival theories of the same phenomenon; watching an intuition form over the course of training; certifying that a model's intuition is faithful to the formalism it was trained on, before trusting the conjectures the model offers. We can imagine even more. Perhaps intuitions can be merged: where human science reconciles two experts through years of conversation, artificial minds can be combined into one directly.
Recall the cycle: attune to phenomena, extract a formalism from the learned intuition, run the formalism, attune anew. The extraction step is the one we understand least, the step where we are most tempted to invoke genius. Einstein had no way to examine his own intuition except to introspect, and to generate from it one thought experiment at a time. For an artificial scientist, extraction could instead be a measurement. We have, for the first time, minds we can open and copy at will: experimental subjects for a science that, until now, could reach its subject matter only through the keyhole of behavior and introspection.
A calculus of intuition
If this picture is anything like reality, it calls for a calculus of intuition, of the kind the last century finally built for reasoning. Leibniz's dream of a calculus ratiocinator waited two centuries; it took first-order logic, and then Turing's more general notion of computation, to turn reasoning from a private mental faculty into a public object, something that could be defined, studied mathematically and finally engineered. It is worth pausing on what that calculus actually contained, because its parts are a template for what we now lack. As a definition, computation has proved unreasonably robust: every formalization of "effective procedure" anyone proposed (machines, rewriting, recursion) turned out to define exactly the same class, and this confluence is what makes computation a natural kind. Its consequences are similarly spectacular: universal compilers, cryptography and impossibility theorems. We should demand the same of the new calculus of intuition that we are gesturing towards.
So far, the engineering of intuition has outstripped its science, and the thermodynamic parallel from earlier holds here too. Thermodynamics, it has been said, owes more to the steam engine than the steam engine owes to thermodynamics: the engines came first, and the science was reverse-engineered from the running machines. Large language models are our steam engines, working engines of attunement built before the theory that would explain them. Interpretability, on this reading, is not a specialist's corner of AI: it is the reverse-engineering from which the science will be born. What is there to build on?
Thermodynamics did not begin with Boltzmann; it began with phenomenological regularities like Boyle's law and Carnot's limits: precise laws about the macroscopic behavior of systems whose microscopic story remained dark. Machine learning already has its analogue. The scaling laws (loss falling as a clean power of compute, data and model size, robustly across architectures) are exactly such regularities: quantitative, reproducible and unexplained. Training even exhibits the signature phenomenon of statistical mechanics, the phase transition, with capabilities sometimes crystallizing abruptly rather than improving smoothly.
There is a deeper regularity to build on as well. When we say a mind or a network generalizes, we tend to imagine a single right way to generalize, the human way, and to grade everything else by its distance from us. But what makes humans special? The striking fact is rather that different attuners so often generalize alike: trained on a shared world, they invent similar concepts to explain it. The blind and the sighted converge on the same picture of physical space; language models converge on much of the same picture from text alone; and independently trained networks turn out to contain many of the same internal features. This convergence, together with the stubborn failures of transfer we saw earlier, is the central regularity a calculus of intuition would have to explain. It is also attunement's own confluence, and the reason to hope that the calculus exists at all: if every capable attuner invented private, incommensurable concepts, there would be no general laws to write down.
Could the bridge from the microscopic dynamics of weights to these macroscopic regularities be built from the concepts we already have? Statistical learning theory (uniform convergence, VC dimension) is a rigorous theory of learning assembled from the classical vocabulary, and it is most famous today for completely missing deep learning. This is not a failure of rigor but a failure of concepts, the position Newtonian mechanics occupied before entropy. What is missing is the step Boltzmann took: a new language in which to explain and understand the observed regularities. The prize, if we find it, is larger than a science of intuition. It is the next step in the journey we started with: mind readmitted into the scientific picture, and the mythological and naturalistic explanations of the world united under one framework.