This is an automated rejection. No LLM generated, assisted/co-written, or edited work.
Read full explanation
Leibniz helps clarify a mistake that shows up in current interpretability work: we often slide from geometric reconstruction quality to representational or causal faithfulness far too quickly.
His calculator, his monads, and his infinitesimals are usually filed under different headings. They are better read as one picture of computation and representation. That picture is surprisingly close to a problem that shows up in current work on sparse autoencoders.
One picture, not three
There is Leibniz the engineer, Leibniz the metaphysician, Leibniz the mathematician. That division misses what is most interesting. The calculator, the Monadology, and the infinitesimal calculus share a common structure: reasoning becomes mechanical once it is rendered formally; a local state can encode a whole world from a point of view; and a difference can be too small to stand out without being nothing.
Start with the calculator. Calculemus did not mean “buy a better abacus.” It meant: formalize the dispute well enough that reasoning can be carried by rule. The stepped reckoner was only the visible part of a broader ambition. The point was not just to compute numbers, but to mechanize inference once a problem had been expressed in the right medium.
This is not a historical anticipation of transformers. But it is close to one structural feature they isolate: once an input is rendered into a formal representational medium, subsequent processing becomes rule-governed transformation inside that medium. A prompt is mapped into a representational space and transformed by a learned procedure. Prompt in, forward pass out. That is not the whole story of intelligence, but it is precisely the part Leibniz was trying to isolate.
The Monadology adds the representational claim. A monad does not contain the whole universe as an explicit inventory. It expresses the whole universe from a perspective and with unequal clarity. Some relations are sharp. Most are dim. Many never become distinct objects of awareness at all. Leibniz calls these petites perceptions: tiny perceptions that do not appear cleanly in consciousness and yet still shape the total state.
That is close to what a token state looks like after several transformer layers. It is no longer just the token. It carries topic, syntax, discourse state, reference structure, likely continuations, and the rest of the context the model has built so far. Different token positions encode that same context from different local roles. Same context, different angle. Not a literal homuncular point of view, of course, but a distributed local encoding of globally relevant state. The representation is distributed, perspectival, and uneven in clarity.
Then there is the infinitesimal. Leibniz’s decisive move was to separate “too small to stand out” from “nothing.” An infinitesimal is not zero. It can be smaller than any ordinary magnitude you care to name and still do real mathematical work. It survives into ratios. It matters to the derivative. It changes what the formal system can say.
That is the part that matters most here. It is a warning against a very common mistake: treating what is small by one metric as irrelevant full stop.
The interpretability lesson
Here is the core claim: geometric closeness and causal faithfulness are not the same thing.
Compressed into one line, the mapping looks like this: calculator → forward pass; monad → distributed, perspective-bound context representation; infinitesimal → low-amplitude but potentially causally active remainder.
Sparse autoencoders make a very natural kind of error tempting. Once a feature is sparse, legible, and easy to name, we start treating it as the real part of the representation. The leftover part becomes residue by default. That bias is understandable. Human-readable features are useful. They give us dictionaries, decompositions, and intervention targets. But they also encourage a tacit inference: if the reconstruction is close enough, then the reconstruction is faithful enough.
To be clear, the mistake is usually not stated this crudely. The problem is that once a reconstruction is sparse, legible, and geometrically very good, it becomes psychologically easy to treat it as an adequate causal stand-in.
That inference is too quick. A reconstruction can be excellent as a vector and still fail as a causal substitute. The problem is not solved by choosing a finer geometric metric. The problem is stopping at geometry.
Why Leibniz helps
Leibniz is useful here because he gives a language for resisting exactly that move. Petites perceptions name the part of a representational state that does not present itself cleanly and still contributes to the whole. The infinitesimal names a difference that can look negligible and remain structurally real.
The analogous possibility in language models is straightforward. Sparse, legible features may capture most of the visible geometry of an activation while a low-amplitude remainder still carries model-relevant signal. That remainder may be hard to name, hard to disentangle, or hard to visualize. None of that implies that it is causally idle.
Leibniz does not prove that this is what is happening in any particular model. He is useful in a different way. He makes the null hypothesis “small remainder equals harmless error” look much less safe than it first appears.
Limits of the analogy
The analogy is structural, not ontological. Monads are metaphysical substances; token states are learned computational objects. The claim is not that Leibniz anticipated neural networks, only that he articulated a distinction between formal transformation, perspectival local representation, and small-but-real difference that remains surprisingly useful here.
The empirical teaser
I have an empirical case that makes this concrete. In one of my own experiments, a sparse autoencoder reconstructed a model activation with 99.7% cosine similarity and still lost roughly 39% of the causal signal under intervention. By “causal signal” I mean preserved intervention effect in the relevant patching setup, not some generic notion of downstream usefulness.
If that sounds like a paradox, good. That is exactly the point. The next post gives the result in full, a PCA control showing that this is not just a cosine artifact, and a protocol for testing whether SAE reconstructions are causally sufficient under intervention.
Sometimes an old framework matters not because it predicted the future, but because it tells you where your current metrics are mistaking a real difference for a negligible one.
For me, this is not a retrospective analogy. I came to the interpretability question through earlier work on the Monadology and infinitesimal calculus, not the other way around.
Notes
The limitation result (MoM) is available on Zenodo: https://zenodo.org/records/18906800 I’ll give the full empirical case, including the PCA control and evaluation protocol, in the next post.
Leibniz helps clarify a mistake that shows up in current interpretability work: we often slide from geometric reconstruction quality to representational or causal faithfulness far too quickly.
His calculator, his monads, and his infinitesimals are usually filed under different headings. They are better read as one picture of computation and representation. That picture is surprisingly close to a problem that shows up in current work on sparse autoencoders.
One picture, not three
There is Leibniz the engineer, Leibniz the metaphysician, Leibniz the mathematician. That division misses what is most interesting. The calculator, the Monadology, and the infinitesimal calculus share a common structure: reasoning becomes mechanical once it is rendered formally; a local state can encode a whole world from a point of view; and a difference can be too small to stand out without being nothing.
Start with the calculator. Calculemus did not mean “buy a better abacus.” It meant: formalize the dispute well enough that reasoning can be carried by rule. The stepped reckoner was only the visible part of a broader ambition. The point was not just to compute numbers, but to mechanize inference once a problem had been expressed in the right medium.
This is not a historical anticipation of transformers. But it is close to one structural feature they isolate: once an input is rendered into a formal representational medium, subsequent processing becomes rule-governed transformation inside that medium. A prompt is mapped into a representational space and transformed by a learned procedure. Prompt in, forward pass out. That is not the whole story of intelligence, but it is precisely the part Leibniz was trying to isolate.
The Monadology adds the representational claim. A monad does not contain the whole universe as an explicit inventory. It expresses the whole universe from a perspective and with unequal clarity. Some relations are sharp. Most are dim. Many never become distinct objects of awareness at all. Leibniz calls these petites perceptions: tiny perceptions that do not appear cleanly in consciousness and yet still shape the total state.
That is close to what a token state looks like after several transformer layers. It is no longer just the token. It carries topic, syntax, discourse state, reference structure, likely continuations, and the rest of the context the model has built so far. Different token positions encode that same context from different local roles. Same context, different angle. Not a literal homuncular point of view, of course, but a distributed local encoding of globally relevant state. The representation is distributed, perspectival, and uneven in clarity.
Then there is the infinitesimal. Leibniz’s decisive move was to separate “too small to stand out” from “nothing.” An infinitesimal is not zero. It can be smaller than any ordinary magnitude you care to name and still do real mathematical work. It survives into ratios. It matters to the derivative. It changes what the formal system can say.
That is the part that matters most here. It is a warning against a very common mistake: treating what is small by one metric as irrelevant full stop.
The interpretability lesson
Here is the core claim: geometric closeness and causal faithfulness are not the same thing.
Compressed into one line, the mapping looks like this: calculator → forward pass; monad → distributed, perspective-bound context representation; infinitesimal → low-amplitude but potentially causally active remainder.
Sparse autoencoders make a very natural kind of error tempting. Once a feature is sparse, legible, and easy to name, we start treating it as the real part of the representation. The leftover part becomes residue by default. That bias is understandable. Human-readable features are useful. They give us dictionaries, decompositions, and intervention targets. But they also encourage a tacit inference: if the reconstruction is close enough, then the reconstruction is faithful enough.
To be clear, the mistake is usually not stated this crudely. The problem is that once a reconstruction is sparse, legible, and geometrically very good, it becomes psychologically easy to treat it as an adequate causal stand-in.
That inference is too quick. A reconstruction can be excellent as a vector and still fail as a causal substitute. The problem is not solved by choosing a finer geometric metric. The problem is stopping at geometry.
Why Leibniz helps
Leibniz is useful here because he gives a language for resisting exactly that move. Petites perceptions name the part of a representational state that does not present itself cleanly and still contributes to the whole. The infinitesimal names a difference that can look negligible and remain structurally real.
The analogous possibility in language models is straightforward. Sparse, legible features may capture most of the visible geometry of an activation while a low-amplitude remainder still carries model-relevant signal. That remainder may be hard to name, hard to disentangle, or hard to visualize. None of that implies that it is causally idle.
Leibniz does not prove that this is what is happening in any particular model. He is useful in a different way. He makes the null hypothesis “small remainder equals harmless error” look much less safe than it first appears.
Limits of the analogy
The analogy is structural, not ontological. Monads are metaphysical substances; token states are learned computational objects. The claim is not that Leibniz anticipated neural networks, only that he articulated a distinction between formal transformation, perspectival local representation, and small-but-real difference that remains surprisingly useful here.
The empirical teaser
I have an empirical case that makes this concrete. In one of my own experiments, a sparse autoencoder reconstructed a model activation with 99.7% cosine similarity and still lost roughly 39% of the causal signal under intervention. By “causal signal” I mean preserved intervention effect in the relevant patching setup, not some generic notion of downstream usefulness.
If that sounds like a paradox, good. That is exactly the point. The next post gives the result in full, a PCA control showing that this is not just a cosine artifact, and a protocol for testing whether SAE reconstructions are causally sufficient under intervention.
Sometimes an old framework matters not because it predicted the future, but because it tells you where your current metrics are mistaking a real difference for a negligible one.
For me, this is not a retrospective analogy. I came to the interpretability question through earlier work on the Monadology and infinitesimal calculus, not the other way around.
Notes
The limitation result (MoM) is available on Zenodo: https://zenodo.org/records/18906800
I’ll give the full empirical case, including the PCA control and evaluation protocol, in the next post.