LLMs are commonly assumed to use superposition to represent more features than they have dimensions. The evidence for this is mostly indirect — chiefly the success of SAEs at extracting interpretable directions. A stronger claim is that models also compute in superposition, and for that we have only theoretical evidence.
Hänni et al. 2024 showed that computing in superposition requires error correction: because features are embedded non-orthogonally, each active feature produces a small interference activation along the others, and a network has to suppress this noise while preserving feature signal. This means the error correction has to treat feature directions differently from generic ones — it has to be less sensitive to perturbations along non-feature directions than along feature directions. We call this feature-specific error correction (FSEC). It’s a prediction of the computation in superposition picture, and it hadn’t been tested empirically. This post proposes a test and runs it.
The method uses activation plateaus: in-distribution activations are robust to small perturbations, and that robustness is direction-dependent. We perturb the residual stream and measure whether the model is more sensitive to a candidate feature direction than to a mixture of two such directions. We summarize this with an norm: at the response is a quadratic form in which no direction is privileged over its mixtures (and which can’t privilege the many directions superposition requires), while means the pure directions are privileged, as FSEC predicts. We measure for contrastive directions and for MELBO and SAE directions, against for PCA, random, and random-difference controls. The contrastive result replicates across six model families, and we reproduce the effect in a toy model with known ground-truth features. These results indicate that candidate feature directions (contrastive, SAE latents, and MELBO) are (approximations of) directions treated preferentially by the network.
Key results
We propose FSEC as a test of computation in superposition: we model the response to a perturbation as a function of the norm of its decomposition into candidate feature directions, where FSEC predicts .
We find for three types of candidate feature direction — contrastive, MELBO, and SAE-decoder — with the contrastive result replicating across six LLMs, while PCA and random baselines give .
We show FSEC also occurs in a toy model of error correction with ground-truth features, where degrades toward 2 as directions are rotated away from the true features.
Why we think this matters
First, to the best of our knowledge, this is the first empirical evidence in non-toy models (we use LLMs) for FSEC, and in turn for computation in superposition. We cannot directly show that computation in superposition happens. However, if it does, we believe it requires FSEC, and we do find empirical evidence of FSEC.
Second, appears to separate feature directions () from non-feature directions (), and measuring it requires only a generic input to perturb rather than labelled data. The natural idea is to use this as an unsupervised feature-finding objective — searching for directions that maximize . While we were excited about this, early results have updated us away from it working: appears to be necessary but not sufficient, and maximizing it surfaces high- directions that don’t otherwise behave like features.
Representations in large language models (LLMs) are poorly understood. It is commonly assumed that LLMs make use of superposition (Elhage et al. 2022) to represent more concepts than they have dimensions available, and potentially to compute in superposition (CiS) (Hänni et al. 2024, Adler & Shavit 2024, Olah et al. 2025). However, we have only indirect evidence for superposition, chiefly the success of sparse autoencoders (Cunningham et al. 2023, Gao et al. 2024, Templeton et al. 2026) at extracting interpretable directions, and only theoretical evidence for CiS (Hänni et al. 2024).
An empirical prediction of CiS is that neural networks must correct interference noise while preserving feature signal (Hänni et al. 2024). This requires networks to be less sensitive to perturbations along non-feature directions than along feature directions. We name this property feature-specific error correction (FSEC). While we cannot rule FSEC out or in without ground-truth feature directions, we can still ask whether FSEC-like behaviour occurs for any directions: if the model’s error correction privileges certain directions, those directions are candidate features, and we can detect them by measuring sensitivity, requiring only a generic input to perturb rather than feature-specific labeled data.
We provide the first empirical evidence of FSEC, showing that the robustness of LLM activations to perturbations privileges certain candidate feature directions over others. Concretely, we perturb residual-stream activations at early layers and measure the downstream response as a function of perturbation direction and magnitude. We construct candidate feature directions via contrastive means for a variety of concepts, including languages, programming languages, gender, sentiment, registers, and verb tenses. Across Gemma-2-9B (Gemma Team 2024), Qwen3-1.7B (Yang et al. 2025), Llama-3.1-8B (Grattafiori et al. 2024), Mistral-7B-v0.3 (Jiang et al. 2023), Aya-Expanse-8B (Dang et al. 2024), and Yi-1.5-9B (Young et al. 2024), we find that contrastive feature directions elicit a stronger downstream response than mixtures thereof, consistent with FSEC that privileges feature directions while suppressing interference along non-feature directions. We formalize this by modeling the LLMs’ response to perturbation as an norm of the perturbation’s decomposition into candidate feature directions. The case reduces to a basis-invariant quadratic form, meaning no choice of basis is privileged; breaks this invariance, indicating that the candidate pure feature directions are more sensitive than their mixtures, as predicted by FSEC. We measure for contrastive directions across models. MELBO (Mack & Turner 2024a) and SAE directions, which also aim to recover model features, likewise yield , albeit with smaller values. PCA and random directions do not, consistent with the interpretation that reflects alignment with the model’s features. We validate this methodology in a toy model of error correction with ground-truth features (Section 5), confirming that degrades toward as directions are misaligned with the true features.
Related Work
Activation plateaus. Prior work has established that in-distribution activations of LLMs are resistant to perturbations (Heimersheim & Mendel 2024, Janiak et al. 2024, Shinkle & Heimersheim 2025), a phenomenon known as activation plateaus. We introduce a novel measurement of the activation plateau boundary geometry, and identify its connection to FSEC.
Direction-dependent sensitivity for feature finding. Prior work has exploited the fact that LLMs have direction-dependent sensitivity to perform unsupervised optimizations for directions maximizing downstream response, resulting in interpretable steering vectors, including MELBO (Mack & Turner 2024a, 2024b). We make use of the same phenomenon to empirically probe error correction, and apply our analysis to directions obtained in this way (Section 4.3). Unlike this line of work, our analysis adds the novel study of the sensitivity geometry and its connection to error correction.
Error correction for computation in superposition.Hänni et al. 2024 argue theoretically that computation in superposition requires error correction that privileges feature directions. We provide empirical evidence in favor of superposition and error correction occurring in LLMs.
Methodology
We probe error correction in LLMs by perturbing residual stream activations and measuring the downstream response. In-distribution activations are robust to small perturbations, a phenomenon known as activation plateaus (Heimersheim & Mendel 2024, Shinkle & Heimersheim 2025). This robustness is direction-dependent, i.e., the model is more sensitive to perturbations along some directions than others. Two lines of evidence suggest feature directions in particular are privileged: empirically, prior work recovers interpretable directions by optimizing for sensitivity, the inverse of robustness (Mack & Turner 2024a, 2024b); and theoretically, FSEC predicts that feature directions are privileged. We test this by comparing the downstream response along candidate feature directions to the response along non-feature baselines.
In all experiments, we perturb the residual stream at an early layer (default ) and measure the downstream response at the second-to-last layer, maximizing the number of intervening layers. This is because activation plateaus are known to be more pronounced the greater the distance between perturbation and measurement (Shinkle & Heimersheim 2025). We avoid the last residual stream layer, which is known to behave atypically. The downstream response is computed by patching the perturbed activation back into the model (Meng et al. 2022, Heimersheim & Nanda 2024), performing a forward pass, and taking the distance between the perturbed and unperturbed residual streams at the measurement layer. We show in Section 4.3 that our results are robust to varying both the perturbation and measurement layers, and hold also when measuring cosine distance or KL-divergence in the logits.
To distinguish directional effects from those of magnitude, we follow prior perturbation analyses of activation plateaus (Heimersheim & Mendel 2024) and perturb by rotating the activation vector towards the perturbation direction while keeping the activation’s norm constant. We refer to this as a norm-matched perturbation. Concretely, a perturbation of angle of toward is:
where is the component of that is orthogonal to . We typically perturb at the last token position, but show in Section 4.3 that our results are robust to this choice.
We quantify the model’s sensitivity along a given direction as the plateau-breaking angle, the smallest perturbation angle for which the downstream response exceeds a threshold . We set per direction-pair, at a level that is guaranteed to be crossed by both single-axis sweeps.[1]
We aggregate the downstream response across a fixed set of inputs (anchors) —last-token residual-stream activations of 5-token FineWeb prompts (Penedo et al. 2024)—by taking the median over anchors. Writing for the response of anchor when perturbed by angle along direction , the single-axis response curve is
is then the largest median response attained when sweeping along alone. For a pair we set the threshold
a fraction of the smaller of the two single-axis maxima. Taking the smaller maximum ensures both axes reach , so the single-axis plateau-breaking angles that calibrate the fit are always defined. Throughout we use . We show in Section 4.3 that our results are robust to varying within half-to-double its nominal value.
We perform perturbations along six types of directions: contrastive (Panickssery et al. 2023, Turner et al. 2023), MELBO (Mack & Turner 2024a), SAE latents (Lieberum et al. 2024), PCA directions, random, and random-difference directions. The first three are all candidate feature directions, while PCA, random, and random-difference directions function as non-feature baselines. PCA directions are computed by performing PCA on the residual-stream activations arising from a randomly selected sample of 10000 5-token-long FineWeb inputs (Penedo et al. 2024). Random directions are sampled isotropically on the unit sphere, .
Contrastive directions are constructed as the difference of mean activations between two sets of matched prompt pairs differing along a single concept (e.g., for gender, “He ran home” is paired with “She ran home”) (Panickssery et al. 2023, Turner et al. 2023). The prompts are LLM generated and human verified, and are available in our code release. Letting denote the activation at the perturbation layer for prompt ,
Random-difference directions apply this same construction to randomly paired activations rather than concept-matched ones, giving a control that is matched to contrastive directions in everything except semantic content. We average differences between pairs drawn at random from the same 10000-input FineWeb sample used for PCA, and normalize:
Unlike contrastive directions, the paired prompts share no concept, so we do not expect them to be features. Unlike isotropic random directions, consist of differences of activations, and hence inherit their covariance. This makes it our most stringent non-feature baseline: it differs from a contrastive direction only in the absence of concept-coherent pairing. We construct 40 such directions.
MELBO directions (Mack & Turner 2024a) are constructed by optimising a unit-norm direction at the perturbation layer to maximise the distance between perturbed and unperturbed activations at a downstream layer. Unlike contrastive directions, this requires no labelled prompt pairs as the directions are found unsupervised, and have been shown to produce interpretable steering when applied at sufficient magnitude.
SAE latents are decoder columns of a sparse autoencoder trained on the model’s residual stream activations (Bricken et al. 2023, Cunningham et al. 2023). Each column is a candidate feature direction under the SAE’s decomposition. We select the top-33 most active SAE latents in the same sample of 10000 FineWeb inputs used for PCA directions. We do this for Gemma-2-9B only and use Gemma Scope (Lieberum et al. 2024) (the width-16k residual-stream SAE at layer 2).
We also perform perturbations toward combinations of two directions . We first restrict them to the tangent space at via the projector , and orthonormalize (Gram–Schmidt) to obtain unit vectors satisfying and . We then form the unit tangent direction
which interpolates between at and at . Perturbing by angle toward then proceeds exactly as in the single-direction case (Equation 1),
We observe behavior (see Section 4) that appears consistent with the downstream response depending on the perturbation’s projections onto privileged directions, raised to a common power . To formalize this, we model the downstream response to a perturbation vector as
where are unit directions, the weights are free parameters capturing each direction’s sensitivity, and is a scalar function.
When we perturb along two directions simultaneously and vary their relative weighting via the mixing angle , the plateau-breaking angle becomes a function of ; plotted in suitable coordinates, this function traces out a superellipse (see Figure 2). Taking and , the inner sum collapses to two terms provided the remaining contribute negligibly along . This is expected under superposition, where features are packed nearly orthogonally (Elhage et al. 2022), so the projections of onto other directions are small; the quality of the superellipse fits in Section 4 confirms the approximation. For the norm-matched perturbation of Equation 9, the displacement is . The term lies along and is therefore orthogonal to each , so it drops out of every projection, ; only the tangential component contributes, so
The plateau breaks when ’s argument crosses a fixed level (the value of the argument at which the response reaches the threshold ), giving the implicit equation for the plateau-breaking angle :
The single-axis sweeps (at and ) calibrate the weights at this same threshold: gives , and gives , where are the plateau-breaking angles measured along and individually. We calibrate on the single-axis sweeps because each isolates one weight; this is a choice of convenience rather than a requirement. Substituting and dividing through by eliminates the unknown threshold and the activation norm, yielding
This means that the plateau-breaking angles define a superellipse of exponent in the normalised coordinates . We estimate by minimizing the squared residual of this equation over the measured points.
Results
We begin by illustrating our measurement procedure on a concrete example.
Figure 1: Measuring plateau-breaking angles.Downstream response as a function of perturbation angle for two contrastive directions (Wealth, Gender) and an equal combination of both, at Gemma-2-9B layer 2, illustrating how plateau-breaking angles are extracted. The plateau-breaking angle is the angle at which the downstream distance first exceeds the threshold (here ). The grey dashed “Random” curve shows the median response across perturbations along 10 isotropic random unit directions. All curves are medians over 30 FineWeb anchor prompts; we omit uncertainty bands because the absolute scale varies substantially from anchor to anchor in a way that is shared across all curves and largely cancels in within-anchor comparisons. The within-anchor gap between each feature direction and the random baseline is nonetheless robust: at every angle, the feature curve exceeds the random baseline for most anchors (), and the median feature-minus-random difference is significantly positive (its 95% bootstrap confidence interval over anchors excludes zero).
Figure 2: Iso-plateau boundary.Plateau-breaking angles for the Wealth Gender pair at Gemma-2-9B layer 2 (per-pair threshold ). The superellipse exponent is fit in the normalised coordinates of Section 3. The boundary is well fit by a superellipse of exponent (fit residual ); indicates these directions are privileged.
Figure 1 shows the downstream response in Gemma-2-9B when perturbing along the Wealth contrastive direction, the Gender contrastive direction, and an equal combination of the two. The vertical dashed lines mark the plateau-breaking angle for each of the directions. Repeating this measurement across a range of mixing angles yields a plateau-breaking angle for each, which we plot in Figure 2. The resulting boundary is well fit by a superellipse with exponent .
To verify that combining contrastive directions in this way is meaningful, we show in Figure 3 that steering along these combinations produces interpretable compositional behaviour changes.
Figure 3: Compositional steering at Gemma-2-9B layer 2.Sample completions for the prompt “The other day I met someone who” under no steering, Poverty alone, Female alone, and the Poverty Female composite. Steering uses the contrastive Wealth and Gender directions (Appendix A); each row shows a single pole—the low-wealth (poverty) pole of Wealth and the feminine pole of Gender. Highlighted spans: orange for poverty and blue for feminine words.
Concretely, steering along the Wealth direction (toward its low-wealth pole) produces poverty-themed completions from a neutral input, steering along the Gender direction produces feminine-coded completions, and steering along a combination of the two produces completions that are both poverty-themed and feminine-coded. This supports the reasonableness of the combination-perturbation setup, though we note that interpretable steering alone does not establish that a direction corresponds to a model feature.
Superellipse exponents across direction types
We now systematically characterise these patterns across directions, direction types, and models. The superellipse exponent has a simple geometric meaning. Perturbing along a single direction breaks the plateau at some angle; perturbing along a mixture of two directions may break it sooner, later, or at the same point, and which of these occurs is exactly what records. If the model responds only to the overall size of a perturbation (its norm), then splitting it evenly between two directions is no different from concentrating it on one, and the plateau-breaking angles trace an ellipse (). If instead the model is specially sensitive to the individual directions—so that a perturbation matters only insofar as it aligns with one of them—then an even mixture, which aligns only partially with each, must be pushed further before the plateau breaks. The boundary then bulges outward, reaching a square () in the limit where each direction has a fully independent threshold and the response fires only as the projection onto either one crosses it. The exponent measures how far toward this “independent thresholds” regime the model sits. Each axis is normalised by its own plateau-breaking angle so that only the privileging of the pure directions over their mixtures—not their individual sensitivities—affects . This selectivity is exactly what error correction requires: staying responsive to individual features while suppressing generic mixtures of them. We give the formal statement—that cannot privilege feature directions under superposition whereas can—in Section 6.1.
We repeat the analysis across six direction types. For contrastive directions, we form 318 pairs from 33 directions (14 binary semantic concepts, 10 natural languages, 9 programming languages), keeping only pairs with intra-pair cosine similarity below 0.1. The filter is needed because our two-direction analysis orthogonalises the pair (Section 3): if the two directions already have significant overlap, orthogonalising one against the other distorts it substantially, so the perturbation no longer probes the labelled feature. The conservative 0.1 threshold keeps this distortion small. The full list and pairwise overlap are in Appendix A. For MELBO, we use 528 pairs obtained via the procedure described in Section 3. For SAE latents, we use top-activating decoder columns from Gemma Scope’s residual stream layer 2 width-16k SAE. PCA and random directions serve as non-feature baselines, as do random-difference directions, for which we form 780 pairs from 40 directions and keep the 710 with intra-pair cosine similarity below 0.1.
Figure 4: Superellipse exponents by direction type.Each dot is one fitted superellipse exponent for a pair of directions. The black horizontal lines are the per-column medians. The dashed orange line corresponds to the isotropic reference. The white markers are the per-column means, with error bars corresponding to the 95% confidence interval on the mean, computed by direction bootstrapping. Candidate feature directions (contrastive, MELBO, top-activating SAE latents) sit consistently above . The PCA, random, and random-difference baselines cluster at . The superellipse fits are good: no per-condition median fit residual exceeds .
We observe that contrastive directions (mean 2.42, 95% CI [2.29, 2.62], median 2.30), MELBO directions (mean 2.21, 95% CI [2.01, 2.42], median 2.16), and top-activating SAE latents (mean 2.28, 95% CI [2.14, 2.47], median 2.19) sit consistently above . This indicates that the model is more sensitive to perturbations along these directions than along their mixtures, consistent with them being features and with FSEC. We emphasize that our result is not merely that these directions are sensitive—as prior work on activation plateaus and MELBO has shown—but that they are more sensitive than mixtures of them, which is precisely what measures. The same is not true for any of our baselines: top PCA directions (mean 2.03, 95% CI [1.94, 2.12], median 1.97), random-difference directions (mean 2.05, 95% CI [2.00, 2.10], median 2.02), and random directions (mean 2.03, 95% CI [2.00, 2.07], median 2.02) all cluster at . As discussed in Section 1, means that, in the per-axis-normalised coordinates, the plateau boundary is elliptical: mixtures break through at the same rescaled magnitude as the pure directions, so neither direction is privileged over its mixtures. This is inconsistent with the baseline directions being aligned with feature directions, under the assumption of FSEC.
Rotating away from feature directions
We have shown that candidate feature directions have while controls do not. We now show that decays monotonically to as we rotate away from candidate feature directions. We take a pair of contrastive feature directions and we rotate each of the directions toward a fixed random orthogonal direction by an angle , re-orthonormalize the pair, and refit, so that recovers the original contrastive pair and yields a random direction. We sweep over a subsample of 40 overlap-filtered pairs (subsampled only to keep the computational cost manageable, as the full sweep is more expensive than the single-pair fits of Section 4.1). For each pair we average over four independent rotation realizations: within each, every direction is tilted toward its own fixed random orthogonal target (Figure 5).
Figure 5: Superellipse exponent versus rotation away from contrastive directions.Each point is the median fitted over the 40 overlap-filtered contrastive pairs and four independent rotation realizations per pair; the shaded band shows the interquartile range. The dashed line marks the isotropic reference. At the perturbation directions are the contrastive directions; the median there () is slightly above the full contrastive-set median (, Figure 4) as it is computed over a 40-pair subsample. At they have been rotated fully onto a random orthogonal direction. degrades monotonically from toward as the directions are rotated away from the contrastive features.
The fitted exponent decreases monotonically from at the contrastive directions to once they are fully rotated to random, indicating that tracks alignment with the candidate feature directions. Because we lack ground-truth features in the LLM, this sweep can only rotate away from candidate features. In Section 5 we close this gap in a toy model with known features.
Ablations
We now show that our results hold across a variety of settings. To do so, we vary our setup along the following axes:
Perturbation method: additive rather than norm-matched.
Perturbation layer: in addition to the default .
Measurement layer: layers 30 and 35 in addition to the default second-to-last (layer 40 for Gemma-2-9B).
Response metric: cosine distance at second-to-last residual stream layer and KL-divergence in logit space in addition to distance.
Plateau-breaking threshold: 50% and 200% of the per-pair threshold .
Activation source: wikipedia-en, wikipedia-zh, and the-stack (default FineWeb).
Model: Qwen3-1.7B, Llama-3.1-8B, Mistral-7B-v0.3, Aya-Expanse-8B, and Yi-1.5-9B.
Figure 6: Robustness of for contrastive directions across setup choices.Each dot is one fitted superellipse exponent for a pair of contrastive directions; the colored marker shows the per-setting mean with 95% CI, and the horizontal dash shows the median. The dashed orange line is the isotropic reference. Columns vary the model, perturbation layer, measurement layer, response metric, response threshold, perturbation method, activation source, and perturbation token position. Fitted exponents remain above across all variations. The superellipse fits are good: no per-condition median fit residual exceeds .
All our results hold across every ablation; the full results are shown in Figure 6. For contrastive feature directions, the means are all in the [2.24, 2.63] interval, and the medians in [2.11, 2.50], all consistent with contrastive feature directions being privileged.
Toy model validation
To validate our method, we test whether it detects FSEC in a toy model with known ground-truth features. This lets us (i) verify that our methodology yields high and good superellipse fits on known ground-truth features, and (ii) repeat the misalignment sweep of Section 4.2 from a known starting point—rotating away from the true features rather than from imperfect candidates, since the toy model exposes the true feature directions. Because our candidate feature directions in LLMs are imperfect approximations of the true features, we expect the measured to be biased downward, since rotating away from a true feature direction should yield a less privileged direction.
We use the two-layer denoising network of Vaintrob 2026 as our toy model. It denoises two-hot -dimensional inputs corrupted with Gaussian noise using neurons, so the features are denoised (i.e., computed) in superposition over fewer than dimensions. The input is expressed in the feature basis, giving us direct access to the ground-truth feature directions.
We apply our two-direction perturbation analysis as before, perturbing additively along pairs of ground-truth inactive feature directions. To measure the effect of misalignment, we rotate the perturbation directions away from the true feature directions toward random directions by varying amounts. We sweep three feature-to-neuron ratios (, , ) with neurons. Full details of the toy model setup are in Appendix C.
Figure 7: Superellipse exponents versus alignment with feature directions.Each dot is the median of fitted superellipse exponents; the shaded region shows the interquartile range. The dashed line marks the isotropic reference. Perturbation directions take the form (normalized to unit length) for , where is a feature axis and is an isotropic random unit vector in ; the x-axis shows the median realized over the 80 pairs. The leftmost points correspond to ground-truth feature directions; the rightmost to isotropic random unit vectors, whose realized overlap with the feature axes is of order per pair. The three lines correspond to feature-to-neuron ratios of , , and , all with neurons. Ground-truth feature directions yield in the range –; degrades monotonically toward as directions are rotated away from the true features.
Figure 7 shows the fitted exponent as a function of the perturbation directions’ alignment with the true feature axes. For ground-truth feature directions, we obtain in the range –, well above , increasing with the feature-to-neuron ratio (highest for ). As directions are rotated away from the true features, degrades monotonically toward , which is recovered for random directions. This mirrors the LLM misalignment sweep (Section 4.2), but now starting from ground-truth features rather than candidates. This is consistent with our prediction that FSEC produces and that imperfect feature alignment biases downward. Due to the significant differences between this toy model and LLMs (two layers, binary features, activation function), we do not place much weight on the specific values obtained; we view this primarily as validation of the methodology and the directional privileging prediction.
Discussion
Why is necessary for error correction
When neural networks use superposition, features are embedded non-orthogonally, so each active feature produces a small spurious “interference” activation along the others. To compute in superposition despite this interference noise, a network must implement error correction (Hänni et al. 2024), and such a mechanism must treat feature directions differently from non-feature directions. To test whether this is the case, we measure the exponent in our response model (Equation 10).
The exponent controls how selectively the network can respond to individual feature directions. When , the response reduces to a quadratic form,
where is a positive semidefinite matrix with at most nonzero eigenvalues. Under superposition, we expect far more features than dimensions, yet a quadratic form has only degrees of freedom and therefore cannot privilege that many directions above all others. Even without superposition, does not allow all feature directions to be more responsive than all non-feature directions: if the eigenvalues of are not all equal, convexity implies that some non-feature directions (linear combinations of high-eigenvalue features) will be more responsive than the lowest-eigenvalue features; if all eigenvalues are equal, the response is isotropic and no direction is privileged at all. In either case, is insufficient for error correction that privileges feature directions.
At the other extreme, makes the response sensitive only to the single largest feature projection, perfectly suppressing interference from all other directions. This extreme is likely not how LLMs function either. We therefore expect the true exponent to lie somewhere between these extremes: .
Empirically, we find for contrastive directions and for MELBO and SAE directions, consistently above and consistent with FSEC. Since our candidate feature directions are imperfect approximations of the model’s true features, the measured is likely a lower bound on the exponent that would be obtained with ground-truth feature directions: imperfect alignment would dilute the directional sensitivity, biasing toward . Our toy model analysis (Section 5) supports this, showing that degrades monotonically toward as directions are rotated away from the true features. In fact, moderate misalignment of perturbation directions in the form of cosine similarity with the feature directions reduces the fitted superellipse exponent from – to –, indicating that our measured for contrastive directions in LLMs is consistent with significantly higher for true feature directions.
Alternative interpretations
Our finding that for candidate feature directions is consistent with FSEC. However, there exist alternative interpretations of this result and of activation plateaus more broadly.
Activation plateaus, the near-zero downstream response to small perturbations that underlies our measurement, might not be signatures of error correction at all. They could instead reflect learned robustness, perhaps related to broad basins of attraction in the activation space of LLMs. This could hold whether or not LLMs implement error correction: it could be that error correction is present but plateaus are not the right empirical signature, or that error correction is absent and plateaus reflect a different phenomenon. However, our main result does not depend on interpreting plateaus as error correction per se: even if the mechanism underlying plateaus is not error correction, the finding that for candidate feature directions but for baselines remains, and still indicates that the network treats these directions preferentially.
It is also possible that LLMs implement error correction in a way that is not directly direction-aware, i.e., without features being linear directions that are consistent across the activation space. For instance, the in-distribution activations of LLMs may lie on a lower-dimensional manifold (Goodfellow et al. 2016, Cheng et al. 2023), and activation plateaus could reflect a mechanism by which small off-manifold perturbations are mapped back on-manifold. This would be broadly consistent with our observations but would admit a different interpretation than the one we have put forward.
Conclusion
We have proposed perturbation analysis of residual-stream activations as an empirical test of whether feature directions are privileged and whether LLMs implement error correction. We find that candidate feature directions in the form of contrastive directions, MELBO directions, and top-activating SAE latents are privileged, consistent with superposition and error correction. Our results hold across six model families and a wide range of experimental settings.
Our results consistently select for our best candidates for feature directions. As discussed in Section 6, this is likely a lower bound: imperfect alignment between our candidate feature directions and the model’s true features would dilute the directional sensitivity, biasing toward . This provides a quantitative target for toy models of error correction: a good toy model should reproduce under comparable experimental conditions. Indeed, the toy model of Section 5 passes this test, yielding in the range – for ground-truth features. More broadly, the exponent cleanly separates candidate feature directions () from baselines ().
Future work.
This discriminative power suggests using the exponent directly as an unsupervised feature-finding objective: searching for directions that maximize it. We have explored this in preliminary experiments, and our tentative conclusion is that it does not by itself recover features: appears to be necessary but not sufficient for a direction to be a feature, so maximizing the exponent alone surfaces high- directions that do not otherwise behave like features. We flag this to avoid overstating the objective’s promise; these results are preliminary and not reported here, and we leave a fuller treatment to future work. The toy model offers a controlled test of this idea, since its ground-truth features let us check whether maximizing the exponent recovers them. On the modeling side, extending the toy-model validation to standard activation functions and deeper architectures would test how closely a toy model of error correction can reproduce the exponents we measure in LLMs.
Acknowledgements
We thank Dmitry Vaintrob for pointing us to the denoising toy model and for useful discussions.
Appendix A: Contrastive direction inventory
We extract 33 contrastive directions across three families. Each direction is the difference of mean last-token activations at the perturbation layer between a positive prompt set and a matched negative set, normalised to a unit vector.
The 14 binary semantic directions are Age, Certainty, Era, Gender, Health, Honesty, Literary, Number, Person, Refusal, Sentiment, Status, Tense, and Wealth. Each uses 30 matched prompt pairs except Literary, which contrasts the slang and literary registers across 50 prompt templates. Prompts are LLM-generated and human-verified.
The 10 natural-language directions cover Arabic, Chinese, Dutch, French, German, Italian, Japanese, Portuguese, Russian, and Spanish. Each pair is an English prompt and its translation in the target language; we use 36 matched prompts per direction, drawn from a single multilingual prompt pool so that all 10 directions share the same English source side.
The 9 programming-language directions cover C++, Go, Haskell, Java, JavaScript, Lisp, Python, Rust, and TypeScript. Each pair is an English prompt and an equivalent code snippet in the target language; 30 matched prompts per direction.
Forming all unordered pairs, we keep the 318 with raw overlap for our two-direction analysis (Figure 8). The filter is needed because our two-direction analysis orthogonalises each pair via Gram–Schmidt (Section 3): if two directions overlap significantly, removing the shared component leaves only a small residual, which renormalisation to unit length then amplifies, so the resulting direction reflects this residual rather than the labelled feature. The threshold is conservative—at this overlap the orthogonalised direction remains more than aligned with the original, and the distortion grows with overlap.
Figure 8: Pairwise cosine overlap of the 33 contrastive directions.Cells show on Gemma-2-9B at layer 2. Directions are ordered and labelled by family (semantic, natural language, programming language). White marks the 210 pairs dropped by the filter applied throughout the paper; the remaining 318 pairs are the ones plotted as Contrastive in the beeswarms. Most filtered pairs sit inside the natural-language block.
Appendix B: Distribution of superellipse fit residuals
For every fitted pair we record the mean radial fraction , i.e. the average fractional distance of the contour points from the fitted superellipse, with corresponding to a perfect fit. The number of contour points varies per fit (it is the cardinality of the iso-threshold level set on the 2D grid); the per-fit median is (IQR –).
Each pair contributes one dot in Figure 9.
Every sub-group median residual is below 2%.
Figure 9: Distribution of superellipse fit residuals across all conditions.Each dot is one pair’s fit residual . The eight left-most columns mirror the ablation axes of Figure 6 (Model, Perturbation layer, Measurement layer, Response metric, Response threshold, Perturbation method, Token position, Activation source). The right-most column gives the residual distribution per direction family (Contrastive, MELBO, SAE, PCA, Random). Per-sub-group mean (white diamond) 95% direction-bootstrap CI and median (horizontal bar) are colour-matched to the sub-group’s dots. The y-axis is clipped at 6% to keep the bulk of the distribution legible; this clips 17 of the 11,762 plotted dots (≈0.14%).
Appendix C: Toy model details
Architecture
The toy model follows Vaintrob 2026. It is a two-layer network with tied weights: an encoder and decoder , with a activation function applied element-wise to the hidden layer. The entries of are drawn i.i.d. from with probabilities , where is a sparsity parameter controlling the fraction of non-zero entries. The network receives a noisy two-hot input , where has exactly active coordinates set to and the remaining set to , and with . The network operates in superposition, with the number of features exceeding the number of neurons .
Optimization
We do not train the encoder and decoder via gradient descent. Instead, we fix the random encoder structure and optimize two scalar parameters: an input gain and an output scale , so that the network computes . We minimize a per-coordinate weighted MSE loss:
where , “on” denotes coordinates where , and “off” denotes coordinates where . The weighting upweights active-coordinate residuals to discourage the trivial solution of outputting zero, which otherwise dominates for very sparse inputs. Given , the optimal has a closed-form solution:
We sweep over a 121-point grid on and select the value minimizing .
The sparsity parameter was swept over for each configuration, selecting by minimum training loss. The selected values were for , for , and for . The weighting parameter was set to for respectively, scaled roughly as with and empirical .
Perturbation procedure
The active set is fixed to throughout, so all perturbations are performed around the same clean input . We sample pairs uniformly without replacement from the inactive coordinates . For each pair, we perturb along the two ground-truth feature directions and . To measure the effect of misalignment, we rotate and toward random directions by varying amounts. Specifically, for each pair we sample an isotropic random unit pair , mutually orthogonal but otherwise unconstrained, and interpolate between the feature direction and the random direction at each misalignment level. The same pair is used across all misalignment levels for a given feature pair.
Superellipse fitting
For each direction pair and misalignment level, we sweep mixing angles uniformly on . For each , we perform a 2500-point linear scan of the perturbation magnitude on and identify the smallest at which . The threshold is set to for the first inactive feature , computed per network. The superellipse exponent is estimated by minimizing over , where . The perturbation sweep is deterministic (no noise is added at perturbation time). All random seeds are set to 42. We show in Figure 10 an example of the superelliptical boundaries we obtain for the toy model in the style of Figure 2 in the main text.
Figure 10: Iso-plateau boundary.Plateau-breaking perturbation magnitudes for perturbations aligned with combinations of two feature directions in the toy model with . The fitted superellipse exponent is with a mean residual of .
Figure 11 shows the toy-model analogue of Figure 1: the response as a function of perturbation magnitude along two feature axes, their equal-weight combination, and a random-direction baseline. The feature axes break the plateau first (–) and random directions last (); the combination breaks at , later than either axis, whereas would predict it breaks at the same magnitude as the axes.
Figure 11: Toy-model response-versus-magnitude sweep. response of the toy model (, ) to additive perturbations of magnitude along two inactive feature axes and , their equal-weight combination, and a random-direction baseline (median over 10 isotropic unit directions), with the plateau-breaking threshold (dotted) and each curve’s plateau-breaking magnitude (dashed verticals). As in Figure 1, feature axes break the plateau first, their combination later, and random directions last.
We initially tried a single global threshold derived from random directions—a fraction of the median, over a set of isotropic random unit directions, of the single-axis plateau height each one reaches—but abandoned it. Direction sensitivities span a wide range (up to a factor of between the least and most responsive directions for the KL-divergence response metric), so a global threshold is either never reached by the least responsive directions or crossed only at very small angles by the most responsive ones, leaving most superellipse fits ill-defined. Setting per pair from each pair’s own single-axis maxima guarantees both axes cross and resolves this. We record this here for ease of reproducibility. ↩︎
LLMs are commonly assumed to use superposition to represent more features than they have dimensions. The evidence for this is mostly indirect — chiefly the success of SAEs at extracting interpretable directions. A stronger claim is that models also compute in superposition, and for that we have only theoretical evidence.
Hänni et al. 2024 showed that computing in superposition requires error correction: because features are embedded non-orthogonally, each active feature produces a small interference activation along the others, and a network has to suppress this noise while preserving feature signal. This means the error correction has to treat feature directions differently from generic ones — it has to be less sensitive to perturbations along non-feature directions than along feature directions. We call this feature-specific error correction (FSEC). It’s a prediction of the computation in superposition picture, and it hadn’t been tested empirically. This post proposes a test and runs it.
The method uses activation plateaus: in-distribution activations are robust to small perturbations, and that robustness is direction-dependent. We perturb the residual stream and measure whether the model is more sensitive to a candidate feature direction than to a mixture of two such directions. We summarize this with an
Key results
Why we think this matters
First, to the best of our knowledge, this is the first empirical evidence in non-toy models (we use LLMs) for FSEC, and in turn for computation in superposition. We cannot directly show that computation in superposition happens. However, if it does, we believe it requires FSEC, and we do find empirical evidence of FSEC.
Second,
The rest of this post is copied from the paper, omitting the abstract and the list of contributions. Code is available at https://github.com/FranciscoHS/fsec-paper.
Introduction
Representations in large language models (LLMs) are poorly understood. It is commonly assumed that LLMs make use of superposition (Elhage et al. 2022) to represent more concepts than they have dimensions available, and potentially to compute in superposition (CiS) (Hänni et al. 2024, Adler & Shavit 2024, Olah et al. 2025). However, we have only indirect evidence for superposition, chiefly the success of sparse autoencoders (Cunningham et al. 2023, Gao et al. 2024, Templeton et al. 2026) at extracting interpretable directions, and only theoretical evidence for CiS (Hänni et al. 2024).
An empirical prediction of CiS is that neural networks must correct interference noise while preserving feature signal (Hänni et al. 2024). This requires networks to be less sensitive to perturbations along non-feature directions than along feature directions. We name this property feature-specific error correction (FSEC). While we cannot rule FSEC out or in without ground-truth feature directions, we can still ask whether FSEC-like behaviour occurs for any directions: if the model’s error correction privileges certain directions, those directions are candidate features, and we can detect them by measuring sensitivity, requiring only a generic input to perturb rather than feature-specific labeled data.
We provide the first empirical evidence of FSEC, showing that the robustness of LLM activations to perturbations privileges certain candidate feature directions over others. Concretely, we perturb residual-stream activations at early layers and measure the downstream response as a function of perturbation direction and magnitude. We construct candidate feature directions via contrastive means for a variety of concepts, including languages, programming languages, gender, sentiment, registers, and verb tenses. Across Gemma-2-9B (Gemma Team 2024), Qwen3-1.7B (Yang et al. 2025), Llama-3.1-8B (Grattafiori et al. 2024), Mistral-7B-v0.3 (Jiang et al. 2023), Aya-Expanse-8B (Dang et al. 2024), and Yi-1.5-9B (Young et al. 2024), we find that contrastive feature directions elicit a stronger downstream response than mixtures thereof, consistent with FSEC that privileges feature directions while suppressing interference along non-feature directions. We formalize this by modeling the LLMs’ response to perturbation as an
Related Work
Activation plateaus. Prior work has established that in-distribution activations of LLMs are resistant to perturbations (Heimersheim & Mendel 2024, Janiak et al. 2024, Shinkle & Heimersheim 2025), a phenomenon known as activation plateaus. We introduce a novel measurement of the activation plateau boundary geometry, and identify its connection to FSEC.
Direction-dependent sensitivity for feature finding. Prior work has exploited the fact that LLMs have direction-dependent sensitivity to perform unsupervised optimizations for directions maximizing downstream response, resulting in interpretable steering vectors, including MELBO (Mack & Turner 2024a, 2024b). We make use of the same phenomenon to empirically probe error correction, and apply our analysis to directions obtained in this way (Section 4.3). Unlike this line of work, our analysis adds the novel study of the sensitivity geometry and its connection to error correction.
Error correction for computation in superposition. Hänni et al. 2024 argue theoretically that computation in superposition requires error correction that privileges feature directions. We provide empirical evidence in favor of superposition and error correction occurring in LLMs.
Methodology
We probe error correction in LLMs by perturbing residual stream activations and measuring the downstream response. In-distribution activations are robust to small perturbations, a phenomenon known as activation plateaus (Heimersheim & Mendel 2024, Shinkle & Heimersheim 2025). This robustness is direction-dependent, i.e., the model is more sensitive to perturbations along some directions than others. Two lines of evidence suggest feature directions in particular are privileged: empirically, prior work recovers interpretable directions by optimizing for sensitivity, the inverse of robustness (Mack & Turner 2024a, 2024b); and theoretically, FSEC predicts that feature directions are privileged. We test this by comparing the downstream response along candidate feature directions to the response along non-feature baselines.
In all experiments, we perturb the residual stream at an early layer
To distinguish directional effects from those of magnitude, we follow prior perturbation analyses of activation plateaus (Heimersheim & Mendel 2024) and perturb by rotating the activation vector
where
We quantify the model’s sensitivity along a given direction as the plateau-breaking angle, the smallest perturbation angle for which the downstream response exceeds a threshold
We aggregate the downstream response across a fixed set ofa fraction
We perform perturbations along six types of directions: contrastive (Panickssery et al. 2023, Turner et al. 2023), MELBO (Mack & Turner 2024a), SAE latents (Lieberum et al. 2024), PCA directions, random, and random-difference directions. The first three are all candidate feature directions, while PCA, random, and random-difference directions function as non-feature baselines. PCA directions are computed by performing PCA on the residual-stream activations arising from a randomly selected sample of 10000 5-token-long FineWeb inputs (Penedo et al. 2024). Random directions are sampled isotropically on the unit sphere,
Contrastive directions are constructed as the difference of mean activations between two sets of
Random-difference directions apply this same construction to randomly paired activations rather than concept-matched ones, giving a control that is matched to contrastive directions in everything except semantic content. We average
Unlike contrastive directions, the paired prompts share no concept, so we do not expect them to be features. Unlike isotropic random directions,
MELBO directions (Mack & Turner 2024a) are constructed by optimising a unit-norm direction
SAE latents are decoder columns of a sparse autoencoder trained on the model’s residual stream activations (Bricken et al. 2023, Cunningham et al. 2023). Each column is a candidate feature direction under the SAE’s decomposition. We select the top-33 most active SAE latents in the same sample of 10000 FineWeb inputs used for PCA directions. We do this for Gemma-2-9B only and use Gemma Scope (Lieberum et al. 2024) (the width-16k residual-stream SAE at layer 2).
We also perform perturbations toward combinations of two directions
which interpolates between
We observe behavior (see Section 4) that appears consistent with the downstream response depending on the perturbation’s projections onto privileged directions, raised to a common power
where
When we perturb along two directions simultaneously and vary their relative weighting via the mixing angle
The plateau breaks when
The single-axis sweeps (at
This means that the plateau-breaking angles define a superellipse of exponent
Results
We begin by illustrating our measurement procedure on a concrete example.
Figure 1: Measuring plateau-breaking angles. Downstream response as a function of perturbation angle for two contrastive directions (Wealth, Gender) and an equal combination of both, at Gemma-2-9B layer 2, illustrating how plateau-breaking angles are extracted. The plateau-breaking angle is the angle at which the downstream
Figure 2: Iso-plateau boundary. Plateau-breaking angles for the Wealth
Figure 1 shows the downstream
To verify that combining contrastive directions in this way is meaningful, we show in Figure 3 that steering along these combinations produces interpretable compositional behaviour changes.
Figure 3: Compositional steering at Gemma-2-9B layer 2. Sample completions for the prompt “The other day I met someone who” under no steering,
Concretely, steering along the Wealth direction (toward its low-wealth pole) produces poverty-themed completions from a neutral input, steering along the Gender direction produces feminine-coded completions, and steering along a combination of the two produces completions that are both poverty-themed and feminine-coded. This supports the reasonableness of the combination-perturbation setup, though we note that interpretable steering alone does not establish that a direction corresponds to a model feature.
Superellipse exponents across direction types
We now systematically characterise these patterns across directions, direction types, and models. The superellipse exponent
We repeat the analysis across six direction types. For contrastive directions, we form 318 pairs from 33 directions (14 binary semantic concepts, 10 natural languages, 9 programming languages), keeping only pairs with intra-pair cosine similarity below 0.1. The filter is needed because our two-direction analysis orthogonalises the pair (Section 3): if the two directions already have significant overlap, orthogonalising one against the other distorts it substantially, so the perturbation no longer probes the labelled feature. The conservative 0.1 threshold keeps this distortion small. The full list and pairwise overlap are in Appendix A. For MELBO, we use 528 pairs obtained via the procedure described in Section 3. For SAE latents, we use top-activating decoder columns from Gemma Scope’s residual stream layer 2 width-16k SAE. PCA and random directions serve as non-feature baselines, as do random-difference directions, for which we form 780 pairs from 40 directions and keep the 710 with intra-pair cosine similarity below 0.1.
Figure 4: Superellipse exponents by direction type. Each dot is one fitted superellipse exponent
We observe that contrastive directions (mean 2.42, 95% CI [2.29, 2.62], median 2.30), MELBO directions (mean 2.21, 95% CI [2.01, 2.42], median 2.16), and top-activating SAE latents (mean 2.28, 95% CI [2.14, 2.47], median 2.19) sit consistently above
Rotating away from feature directions
We have shown that candidate feature directions have
Figure 5: Superellipse exponent versus rotation away from contrastive directions. Each point is the median fitted
The fitted exponent decreases monotonically from
Ablations
We now show that our results hold across a variety of settings. To do so, we vary our setup along the following axes:
Figure 6: Robustness of
All our results hold across every ablation; the full results are shown in Figure 6. For contrastive feature directions, the means are all in the [2.24, 2.63] interval, and the medians in [2.11, 2.50], all consistent with contrastive feature directions being privileged.
Toy model validation
To validate our method, we test whether it detects FSEC in a toy model with known ground-truth features. This lets us (i) verify that our methodology yields high
We use the two-layer denoising network of Vaintrob 2026 as our toy model. It denoises two-hot
We apply our two-direction perturbation analysis as before, perturbing additively along pairs of ground-truth inactive feature directions. To measure the effect of misalignment, we rotate the perturbation directions away from the true feature directions toward random directions by varying amounts. We sweep three feature-to-neuron ratios (
Figure 7: Superellipse exponents versus alignment with feature directions. Each dot is the median of
Figure 7 shows the fitted exponent
Discussion
Why
When neural networks use superposition, features are embedded non-orthogonally, so each active feature produces a small spurious “interference” activation along the others. To compute in superposition despite this interference noise, a network must implement error correction (Hänni et al. 2024), and such a mechanism must treat feature directions differently from non-feature directions. To test whether this is the case, we measure the exponent
The exponent
where
At the other extreme,
Empirically, we find
Alternative interpretations
Our finding that
Activation plateaus, the near-zero downstream response to small perturbations that underlies our measurement, might not be signatures of error correction at all. They could instead reflect learned robustness, perhaps related to broad basins of attraction in the activation space of LLMs. This could hold whether or not LLMs implement error correction: it could be that error correction is present but plateaus are not the right empirical signature, or that error correction is absent and plateaus reflect a different phenomenon. However, our main result does not depend on interpreting plateaus as error correction per se: even if the mechanism underlying plateaus is not error correction, the finding that
It is also possible that LLMs implement error correction in a way that is not directly direction-aware, i.e., without features being linear directions that are consistent across the activation space. For instance, the in-distribution activations of LLMs may lie on a lower-dimensional manifold (Goodfellow et al. 2016, Cheng et al. 2023), and activation plateaus could reflect a mechanism by which small off-manifold perturbations are mapped back on-manifold. This would be broadly consistent with our observations but would admit a different interpretation than the one we have put forward.
Conclusion
We have proposed perturbation analysis of residual-stream activations as an empirical test of whether feature directions are privileged and whether LLMs implement error correction. We find that candidate feature directions in the form of contrastive directions, MELBO directions, and top-activating SAE latents are privileged, consistent with superposition and error correction. Our results hold across six model families and a wide range of experimental settings.
Our results consistently select
Future work.
This discriminative power suggests using the exponent directly as an unsupervised feature-finding objective: searching for directions that maximize it. We have explored this in preliminary experiments, and our tentative conclusion is that it does not by itself recover features:
Acknowledgements
We thank Dmitry Vaintrob for pointing us to the denoising toy model and for useful discussions.
Appendix A: Contrastive direction inventory
We extract 33 contrastive directions across three families. Each direction is the difference of mean last-token activations at the perturbation layer between a positive prompt set and a matched negative set, normalised to a unit vector.
The 14 binary semantic directions are Age, Certainty, Era, Gender, Health, Honesty, Literary, Number, Person, Refusal, Sentiment, Status, Tense, and Wealth. Each uses 30 matched prompt pairs except Literary, which contrasts the slang and literary registers across 50 prompt templates. Prompts are LLM-generated and human-verified.
The 10 natural-language directions cover Arabic, Chinese, Dutch, French, German, Italian, Japanese, Portuguese, Russian, and Spanish. Each pair is an English prompt and its translation in the target language; we use 36 matched prompts per direction, drawn from a single multilingual prompt pool so that all 10 directions share the same English source side.
The 9 programming-language directions cover C++, Go, Haskell, Java, JavaScript, Lisp, Python, Rust, and TypeScript. Each pair is an English prompt and an equivalent code snippet in the target language; 30 matched prompts per direction.
Forming all
Figure 8: Pairwise cosine overlap of the 33 contrastive directions. Cells show
Appendix B: Distribution of superellipse fit residuals
For every fitted pair we record the mean radial fraction
Each pair contributes one dot in Figure 9.
Every sub-group median residual is below 2%.
Figure 9: Distribution of superellipse fit residuals across all conditions. Each dot is one pair’s fit residual
Appendix C: Toy model details
Architecture
The toy model follows Vaintrob 2026. It is a two-layer network with tied weights: an encoder
Optimization
We do not train the encoder and decoder via gradient descent. Instead, we fix the random encoder structure and optimize two scalar parameters: an input gain
where
We sweep
The sparsity parameter
Perturbation procedure
The active set is fixed to
Superellipse fitting
For each direction pair and misalignment level, we sweep
Figure 10: Iso-plateau boundary. Plateau-breaking perturbation magnitudes for perturbations aligned with combinations of two feature directions in the toy model with
Figure 11 shows the toy-model analogue of Figure 1: the
Figure 11: Toy-model response-versus-magnitude sweep.
We initially tried a single global threshold derived from random directions—a fraction