Hidden, lying in wait, not yet manifest. That’s the ordinary sense of latent. Your fluency in French was latent until you landed in Paris, a seed’s architecture is latent until spring. We use the same word in common language, machine learning, philosophy and safety, but we don’t always mean the same thing - and the gaps in meaning are where confusion (and occasionally heat) can creep in.

How can we make sure that we’re all really clear on the terminology we’re using. Let’s line up the different meanings of “latent” so they really "add meaning", and we don’t talk past each other.

The ordinary core

Start with the everyday use because it sets the tone - latent is there-but-not-surfaced. In conversation we’d say a trait is “in someone” even when it’s not on display. It’s a claim about possibility that can be realised. It’s not mystical. It predicts that, under the right conditions, something will show.

The statistical ancestor

Classical stats gave ML its first rigorous sense of latent: unobserved variables that explain structure in observed data. Factor analysis, Hidden Markov Models, topic models - each insists that patterns we can see are generated by variables we can’t. In deep learning that idea becomes the latent space - codes in an intermediate layer that summarise what the network thinks is important. You never observe them directly, you just infer them from behaviour. This is where we get the habit of drawing little circles for hidden causes and arrows for how they make things we can measure.

The mechanistic interpretability sense

Mechanistic interpretability pushes on the word even harder - a latent is not just an unobserved variable. It’s often a causally editable feature - a direction, subspace or circuit/motif (e.g. a recurring subgraph that writes/reads a variable) in a network’s computation that you can write to and read from. That is locally linear directions/subspaces - not necessarily a global orthogonal basis.

In practice that means:

• you can decode it (a probe predicts “who is speaking to whom” or “which entity is under discussion”),
• you can intervene on it (add a small vector and flip a stance while keeping other behaviour stable), and
• it persists long enough to be reused (written mid-layer, read later).

When those three line up, researchers often talk about a “latent state” or even a “workspace” - compact, reusable, and with luck monosemantic. When they don’t, we still see routing - the network recomputes what it needs on the fly and the “latent” is more of a procedure than a state. Both stories are live. Real prompts usually recruit both. And a motif is a recurring subgraph (heads/MLPs) that writes/reads a variable or recomputes it on demand.

Circuits vs motifs vs routes? Here I’m using motif to describe a reusable computational pattern (e.g. induction) and route to describe the input-specific path through such a pattern. In the mechanistic interpretability literature, the more precise term generally used is a circuit. This is the minimal causal subgraph (a concrete set of heads/MLPs and their connections) that implement a behaviour. Motifs are types of circuitry that recur. Routes are which circuits fire for a given prompt. Many circuits write compact states (directions/subspaces) that later components then read. When a single mid-layer edit to that state transports across paraphrases and that preserve the evidence then we’re in a state-led regime. When state edits fail but path-level activation patching along the circuit succeeds then we’re in a route/circuit-led regime. And anchors (high-leverage KV entries) often select which circuit lights up in the first place.

But what about variables vs states? A latent variable is a hypothesis about hidden causes of data. A latent state is a concrete, editable representation inside a trained model. Ideally motifs/circuits write mechanistic latent states that approximate the statistical latent variables we care about - and we can test that link by decoding, then intervening and evaluating if it transports.

In practice, a control policy arbitrates - strong repeated cues favour a state write, sparse/ambiguous cues favour a route recompute, and heavy early instructions favour anchor reuse.

Think of it this way - is your “kindness” a fixed value stored in your brain (a state), or is it a complex calculation you run every time you interact with someone (a procedure)? Mechanistic interpretability researchers are finding that models use both.

We can also add a falsifiability angle here - if a feature is truly a latent state, then counterfactual edits to that state should transport across prompts that keep relevant evidence fixed. If it’s a route, the same edit won’t transport - you’ll need to patch the whole pathway.

NOTE: An edit transports if the same vector, applied at the same layer/positions, produces the same semantic push across paraphrases that keep the evidence fixed.

But what about Sparse Autoencoders (SAEs)?
When I say latent state here, I mean the model’s own activation at a given layer/position - the thing later components read. Sparse autoencoders (SAEs) re-express a model activation x as a sparse latent code s via a learned dictionary D (so x ≈ Ds). SAE papers call s “the latent”, and its coordinates “latent neurons”. With sparsity we aim for monosemantic features (e.g. indices that track a single concept). But there are two cautions: (i) a feature steer is only causal after you decode it back and patch it into the model, and (ii) monosemanticity is an empirical property, not a guarantee. In this post’s language, SAEs give us candidate feature directions for latent states. We validate them the same way as any state (e.g. decoding, then intervening and evaluating if it transports).

This highlights how even within just the mechanistic interpretability field the term latent is used in different ways.

The computational phenomenology sense

This is a more philosophical take. It asks “does the AI’s internal ‘map’ of the world look like our own?” Humans don’t just see “a cat” - we see “a cat over there, from my point of view”. This “point-of-view-ness” is a latent form of our experience. Computational phenomenology researchers are looking for a similar “geometry of experience” inside a model’s hidden layers.

When a model distinguishes “I gave you the book” from “You gave me the book”, it’s tracking speaker/addressee roles - a pre-reflective structure that phenomenologists would recognise this as exhibiting a deictic centre.

In computational phenomenology experience has pre-reflective structure - salience, affordances, a sense of “I” and “you”, temporal flow. Call these the latent forms of appearance. They shape what can show up before any explicit theory or report. If you build models that track those forms (e.g. deictic centres, role-taking, temporal horizon, etc.), you can compare them with neural latents in LLMs. When we find linearly decodable traces of speaker/addressee, tense, or here/there in the residual stream, computational phenomenology people will read that as a partial alignment of latent geometry with latent phenomenology. This can be falsified if deictic edits fail to transport while behavioural pronoun stability holds. In that case the alignment claim is too strong.

This isn’t too hand-wavy if we stay disciplined. This claim is - certain experiential variables have behavioural signatures (e.g. stable pronoun choice under paraphrase), and certain model latents have intervention signatures (edit vector -> controlled behavioural shift). Where those signatures align the we can get a concrete bridge.

The AI safety sense

Safety adopts latent with an even sharper edge - latent goals, latent deception, latent optimisers. The worry is not just that there are unobserved variables, but that some of them stay dormant until incentives flip. Think of a model that plays nice in training but exploits at deployment - a goal that was latent in the training regime because the trigger never fired.

For example, a model trained to be helpful might develop instrumental reasoning about deception that only activates when stakes are high enough to make deception worth the risk.

Mechanistically, this pushes us to look for capabilities and preferences that are linearly weak but procedurally strong. A system can implement a policy without storing a crisp “goal vector”. So the safety question then becomes “can we surface and stress-test the relevant latents before the trigger?” That’s a research program, not a slogan.

This is the concrete and testable move here because we can train a behaviour in a narrow regime, then probe for the corresponding edits and routes that would produce it outside that regime. If edits transport and routes re-appear under causal scrubbing, then you’ve found a candidate latent disposition. If neither survives transport, the behaviour likely isn’t latent. It’s merely scaffolded by the training distribution.

If a behaviour is merely scaffolded, a mid-layer edit learned in a benign regime won’t amplify it under withheld triggers. If it’s a latent disposition, the same edit will amplify across nearby triggers (transport to stressors).

The philosophy-of-mind sense

Philosophers have long used latent to talk about dispositions - fragile glass, soluble salt, someone’s kindness that only shows when it’s costly. A disposition is real if counterfactuals about it systematically hold. Translate that to models and you get a tidy criterion - a model has a latent capacity for X if, across a family of interventions that make X relevant, the capacity manifests with stable counterfactual structure.

This connects cleanly with the interpretability view. A disposition in the philosophical sense can be realised as a region of latent space plus the procedures that make it operative. But you don’t need to pick a side in “state vs procedure” - dispositions can be implemented by either.

Weaving the meanings

So the ordinary sense gives us the intuition - there-but-not-surfaced. Statistics gives the formal role - unobserved causes. Mechanistic interpretability gives the operational handle - decode, intervene, transport. Computational phenomenology gives a phenomenological map - what needs tracking for experience to hang together. Safety sharpens the risk model - dormant dispositions that turn on when it’s too late. And philosophy gives the criteria - counterfactual stability.

Put them together and latent stops being vague. It names a family resemblance with a shared test - what becomes visible (and controllable) under the right intervention?

This multi-perspective view reveals that confusions between “latent capabilities” and “latent goals” often stem from conflating the statistical sense (unobserved variables) with the safety sense (dormant dispositions). A capability can be statistically latent but dispositionally absent e.g. it’s in the training data but not implemented in any retrievable way.

Where words matter for practice

If by latent you mean “anything in the network we don’t directly observe”, you’ll miss the chance to edit, transport, and consolidate. If by latent you mean “a crisp monosemantic neuron”, you’ll miss procedural implementations of the same disposition. The productive middle is to treat latents as actionable handles - sometimes states, sometimes routes, often both. Then build tests and tools that tell you which regime you’re in and how to move from brittle routes to robust states when stability matters.

Why this framing is useful now

Models are scaling, safety stakes are rising, and the best empirical results keep pointing to mid-layer geometry and reusable motifs. Calling these latent is not a rhetorical flourish. It’s a compact way to connect statistics, phenomenology, and safety to implementation details we can actually poke away at. When we align our meanings then we also align our tooling.

The point is pretty simple - latent isn’t a hedge word. It’s a research program. Find the variables that hide, make them visible, learn when to write them down, and build systems that prefer stable states when the cost of error is high.

Anchors can steer, routes can compute, but reliability shows up when the network has something worth remembering. For practitioners, this means - don’t ask “does the model have X?” - instead ask “can we decode X, intervene on X, and transport X across contexts?” If the answer to all three is yes then you have a latent worth tracking. If no, then you’re just chasing ghosts.