Abstract

Sparse autoencoders (SAEs) are increasingly being used to decompose language-model activations into features that appear interpretable than individual neurons. But there is an important gap between a feature being associated with a concept and the feature actually causing a model behavior.

I ran a small inference-only experiment on Llama 3.1 8B with a pretrained 131k-feature SAE attached to the Layer-19 residual stream. The goal was to test whether an independently validated semantic interpretation of an SAE feature could predict its causal effect on behavior, and then trace that causal effect into a local circuit.

I focused on an animal-vs-vehicle behavioral distinction. One feature, F54316, showed strong animal-related activation on held-out examples. Independent semantic profiling also showed a graded response across mammals, birds, reptiles, insects, fish, plants, and unrelated categories.

I then intervened directly on the feature's decoder direction. Increasing F54316 shifted the model's animal-vs-vehicle log-probability margin in the predicted direction. The effect was robust across held-out examples, intervention strengths, and prompt forms, while an independent sentiment behavior showed little corresponding effect.

Finally, I traced the feature's causal role upstream and downstream. Layer-18 attention Heads 0 and 8 emerged as strong contributors to F54316. Ablating either head reduced F54316, and restoring the feature approximately restored the behavioral margin. Downstream, the F54316 perturbation propagated through Layers 20–31 while gradually rotating away from the original SAE feature direction. At the output, the intervention increased the animal-side token logits substantially more than the vehicle-side logits.

Experiment ask a simple question:

Can an independently validated semantic interpretation of an SAE feature predict its causal effect, and can that effect be traced to a local mechanistic pathway?

The results suggest that, at least for this feature and behavior, the answer is yes.

Reproducibility

Code and full experiment: https://colab.research.google.com/drive/1Xh4xK_D30xYoosBuEeGTKyIsGX-lgJc0?usp=sharing

The notebook contains the complete pipeline used for this post, including behavioral validation, SAE feature discovery, held-out testing, semantic profiling, causal intervention, specificity and robustness tests, upstream circuit tracing, feature-rescue experiments, downstream propagation, and final output-logit analysis.

The experiments use a pretrained Llama 3.1 8B model and pretrained SAE; no model or SAE training is required.

1. Why I wanted to do this

There are two questions that are easy to conflate in mechanistic interpretability:

1. What does an internal feature appear to represent?
2. What does that feature actually do?

An SAE can give us something that looks like an interpretable semantic feature:

“This feature seems to represent animals.”

But that does not automatically imply:

“If I increase this feature, the model will behave more animal-like.”

A feature can correlate with a behavior without being causally responsible for it.

Simple experimental loop:

SAE feature
    ↓
independent semantic validation
    ↓
causal prediction
    ↓
feature intervention
    ↓
behavioral effect
    ↓
circuit tracing

2. Experimental setup

I used:

• Llama 3.1 8B
• 8-bit inference in Google Colab
• a pretrained 131k-feature SAE
• the SAE attached to the Layer-19 residual stream
• small held-out lexical datasets
• inference-only interventions

I deliberately avoided training a model or a new SAE. The project was designed to be feasible on Colab using pretrained components.

3. First problem: finding a clean behavior

I initially tried several behavioral distinctions.

Some were poor:

• animate vs inanimate: 50%
• living vs object: 45%
• singular vs plural: 60%

This was useful. It prevented me from building a causal experiment around a behavior that the model itself did not represent cleanly enough.

I returned to a simpler distinction:

animal vs vehicle

The behavioral score was computed from two continuations:

" an animal"
" a vehicle"

For a word (w), I defined:

Positive values indicate an animal preference; negative values indicate a vehicle preference.

The final causal-test set achieved:

19/20 = 95% accuracy.

Example baselines:

leopard       +12.375
cheetah       +11.250
hyena         +12.250
rhinoceros    +15.750
hippopotamus  +17.875
...

wagon          -5.000
carriage       -5.250
motorcoach     -7.375
ambulance     -11.375
roadster      -10.000

The margins are highly saturated for many examples, which is not ideal for intervention experiments, but the distinction itself is very clean.

4. Discovering candidate SAE features

I searched for SAE features whose activations differed between the animal and vehicle discovery sets.

Some candidates looked strong:

Animal candidates:
F37378
F54316
F35049
...

Vehicle candidates:
F57774
F110833
F75101
...

But I separated feature discovery from evaluation.

This mattered a lot.

For example, F35049 initially looked promising:

discovery animals   ≈ 0.487
discovery vehicles  ≈ 0.187

But on held-out data:

held-out animals    ≈ 0.411
held-out vehicles   ≈ 0.443

So it did not generalize.

Another candidate, F112041, actually reversed its apparent specificity.

This was a useful result in itself:

Discovery score alone was not sufficient to identify a reliable semantic feature.

The final feature I focused on was F54316.

5. What does F54316 appear to represent?

F54316 showed strong activation for animals on fresh data.

A broader semantic profile was more interesting than simply comparing animals with vehicles.

Category profile

Unrelated semantic groups such as abstract concepts, technology, tools, clothing, people, and places were essentially inactive in this probe set.

I also ran an independent contrastive analysis:

mean contrast: +0.9078
animal higher: 9/10 pairs

So the interpretation was not merely:

“F54316 correlates with the animal/vehicle dataset.”

It was broader:

F54316 appears to represent animal-related semantic content, especially biological animals, with strongest activation for mammals and birds.

Importantly, I am not claiming that this proves a literal “animal concept neuron.” It is an SAE feature with strong semantic selectivity in this experiment.

6. Turning the interpretation into a falsifiable prediction

This was the key step.

Once I had the independent interpretation:

F54316 is animal-related

I wrote down the causal prediction:

Increasing F54316 should increase the animal-vs-vehicle behavioral margin.

Conversely, suppressing the feature should push the model in the opposite direction.

This is what makes the experiment causal rather than descriptive.

7. Feature intervention

I intervened directly on the Layer-19 residual stream using the decoder direction for F54316.

Conceptually:

I calibrated intervention strength relative to the residual-stream norm rather than relying only on the raw SAE activation.

For the reference word “leopard”:

Layer-19 residual norm ≈ 12.73

I swept intervention strengths corresponding to fractions of that norm.

The average behavioral effect increased with intervention strength:

fraction    mean signed effect
0.025       +0.150
0.050       +0.234
0.100       +0.322
0.150       +0.466
0.200       +0.597

That is the kind of dose-response behavior I was looking for.

At the strongest tested intervention:

animal examples:
mean effect ≈ +0.819
positive direction: 10/10

vehicle examples:
mean effect ≈ +0.375
positive direction: 7/10

The effect is therefore not perfectly category-specific, but the animal-side effect is substantially stronger.

8. Controls

I wanted to avoid interpreting every change in model behavior as evidence for the animal feature.

Prompt robustness

I tested different prompt forms, including variants resembling:

The X is...
I saw X...
The X...
This is...

F54316 continued to produce positive animal ward effects across prompt forms.

9. Upstream circuit tracing

Once I had a causally validated feature, I asked:

Where does F54316 come from?

I performed a coarse layer-wise ablation over Layers 0–18.

Several layers affected F54316, but Layer 18 stood out strongly.

I then separated Layer-18 attention from MLP and eventually tested individual attention heads.

The strongest heads were:

Layer 18 Head 8
mean ΔF54316 ≈ −0.170

Layer 18 Head 0
mean ΔF54316 ≈ −0.157

Ablating these heads reduced F54316.

This suggested:

L18 H0 ─┐
       ├──→ F54316
L18 H8 ─┘

10. The important rescue experiment

The strongest part of the circuit tracing was not the head-ablation result itself.

It was the rescue experiment.

For H8:

ablate H8
    ↓
F54316 decreases
    ↓
behavior changes

Then I added the F54316 decoder direction back in the amount needed to restore its activation.

The result:

mean F54316 drop        ≈ −0.177
mean rescue error       ≈ −0.013
mean behavioral Δ after rescue ≈ 0

H0 produced a very similar result:

mean F54316 drop        ≈ −0.157
mean rescue error       ≈ −0.012
mean behavioral Δ after rescue ≈ 0

This is much stronger than simply saying that the heads influence the feature.

It supports the pathway:

L18 H0/H8
    ↓
 F54316
    ↓
behavior

because restoring the middle variable approximately restores the behavioral effect.

I would still call this a local mediation result, not a proof that these are the only routes through which the behavior is computed.

11. What happens downstream?

I then injected F54316 and followed the perturbation through Layers 20–31.

The perturbation did not disappear immediately.

Its norm grew:

L20  ≈ 3.74
L26  ≈ 5.69
L31  ≈ 10.28

But the perturbation gradually became less aligned with the original F54316 decoder direction:

L20 cosine ≈ 0.75
L25 cosine ≈ 0.37
L30 cosine ≈ 0.22
L31 cosine ≈ 0.16

This suggests that downstream layers are transforming the signal.

It is not simply:

F54316
  ↓
same vector copied to output

It looks more like:

F54316
  ↓
Layer 20 transformation
  ↓
Layer 21 transformation
  ↓
...
  ↓
Layer 31 transformation
  ↓
output representation

12. Did I find a single downstream bottleneck?

No.

I tested several downstream layers with a proper 2×2 mediation design.

The results did not cleanly identify a single layer that blocked the F54316 behavioral effect.

For me, this is a useful negative result.

I therefore do not claim:

“Layer 26 is the downstream animal circuit.”

The more defensible interpretation is that the F54316 signal participates in a distributed downstream computation.

13. Does the effect reach the actual output?

Yes.

Finally, I measured the effect of F54316 intervention directly on the relevant output tokens.

Mean effects:

animal-token logit effect   +0.755
vehicle-token logit effect  +0.095
directional gap             +0.660

So increasing F54316 preferentially increased the model's probability for the animal-side continuation.

This closes an important part of the chain:

F54316 intervention
      ↓
internal representations change
      ↓
output logits change
      ↓
animal-side output becomes more likely

14. The resulting local circuit

Putting the strongest evidence together:

                  Layer 18
              ┌──────┴──────┐
              │             │
             H0             H8
              │             │
              └──────┬──────┘
                     ↓
                 F54316
             "animal-related"
                     │
                     ↓
             Layers 20–31
             distributed /
             transformed
             computation
                     │
                     ↓
              output logits
               ↙         ↘
       "an animal"     "a vehicle"
           ↑                ↑
         +0.755           +0.095
                     │
                     ↓
          animal-vs-vehicle behavior

I would call this a feature-centered local causal circuit, rather than a complete circuit reconstruction of the model.

15. What did we actually learn?

The most important result is not simply:

“We found a feature that activates for animals.”

That would be relatively weak.

The more interesting result is:

semantic interpretation
        ↓
causal prediction
        ↓
feature intervention
        ↓
behavior changes as predicted
        ↓
upstream components identified
        ↓
feature-rescue restores behavior
        ↓
signal propagates downstream
        ↓
animal-side output logits increase

That gives a fairly complete hypothesis → intervention → verification loop.

16. What failed?

I think it is important to include this honestly in the post.

The original plan was built around SelfIE. The original research question was whether a self-generated interpretation of an SAE feature could predict its causal effect.

I could not get the original SelfIE implementation working reliably in the final Llama environment because of a dependency incompatibility.

Instead, I replaced that stage with independent semantic profiling and contrastive attribution.

Therefore:

This experiment is not evidence that SelfIE itself produces causally faithful explanations.

It is evidence for a related claim:

An independently validated semantic interpretation of an SAE feature can successfully predict a causal behavioral effect, and the effect can be embedded into a local mechanistic pathway.

I think that distinction makes the result substantially more credible.

17. What is novel here?

I would be cautious about making a strong novelty claim without a systematic literature review.

The broader ingredients already exist separately:

• SAEs for discovering sparse internal features
• causal interventions on features
• self-interpretation methods such as SelfIE
• circuit tracing and mechanistic attribution

The original project was explicitly designed to combine these ideas rather than claim to invent them independently.

The interesting question for the project is therefore whether this particular combination and evaluation protocol adds something useful.

The smallest defensible contribution is probably:

A compact inference-only methodology for taking an independently validated SAE semantic hypothesis, testing its causal behavioral prediction, and tracing the resulting effect into a local upstream/downstream mechanism.

Whether that is novel enough for a paper or fellowship application requires a dedicated literature comparison.

18. Limitations

There are several.

Saturated behavioral margins

Many animal examples already had very large positive margins. This limits how much room there is for further movement.

One principal feature

The strongest mechanistic conclusions are centered around F54316. We have not demonstrated that the same pattern generalizes broadly across dozens or hundreds of features.

Small datasets

The datasets were intentionally small because of the Colab compute constraint.

No successful SelfIE validation

The experiment does not establish causal faithfulness of self-generated interpretations.

Distributed downstream computation

We could trace propagation but did not isolate one definitive downstream bottleneck.

Local rather than complete circuit

We traced a causal pathway around F54316, not every computational dependency from token input to final output.

19. What I think the result means

My current interpretation is:

SAE semantic interpretability becomes substantially more interesting when the semantic hypothesis is converted into a falsifiable causal prediction.

In our experiment, F54316 was independently characterized as animal-related. That interpretation predicted that increasing the feature should push the model toward animal-related outputs. The intervention did exactly that, and the effect could be linked upstream to specific Layer-18 attention heads and downstream to the model's animal-side output logits.

The most encouraging part is the rescue experiment:

removing an upstream component reduced F54316 and altered behavior, while restoring F54316 approximately restored the behavior.

That makes the feature look less like a descriptive label and more like a meaningful computational variable.

20. The final research question

I would now freeze the project as:

Can independently validated SAE feature semantics predict causal model behavior, and can the resulting causal effect be traced to a local mechanistic circuit?

And the core hypothesis:

If an SAE feature has a reliable semantic interpretation, then intervening on that feature should produce the behavioral change predicted by its interpretation; further, upstream and downstream causal analysis should reveal a mechanistic pathway connecting the feature to the behavior.

That is the experiment we actually performed.

Author: Mohibul Hoque
Contact: hokworks@gmail.com