TL;DR: Refusal direction ablation, known as abliteration, using the standard diff. of means approach established in literature produced no feasible candidate for Gemma-3-12b whereas it did so fine for both similarily-sized Qwen and Llama models. A suggested fix on the internet was found which involved Winsorization based on magnitude of co-ordinate activations, but it lacked theoretical proof and insufficient empirical evidence. We investigate the problem and find the issue - a coordinate which dominates others in scale and then work backwards to explain the results and then validate the correction. We also back the method with more empirical proof, uncover the phenomenon across multiple model sizes in the Gemma-3 and show the fix generalizes. Finally, we corroborate evidence of this phenomenon in studies working on extracting directions in any model of this family. The question of why training particularly develops such a large activation co-ordinate in Gemma-3 models remains open.

A note of thanks to Suyash Maniyar and Sagnik Chatterjee for helping review the draft of this blog.

For an undisclosed research project, my aforementioned collaborators and I were working on extracting refusal directions in models across model families. We followed the standard methodology of extraction and then validated the causality of the direction using abliteration. The workflow is simple: label prompts into "harmful" and "harmless" based on behavioral refusal rather than a pre-registered semantic prior (although we usually start with a contrast to ensure equal post-filter distribution without much further exploration), collect residual stream (across layers) activations of every prompt in both sets at the `t_post_inst` position and then take difference of mean activations between the two classes.^[1] Then, at inference time, we induce refusal by adding the difference back into the stream at the layer from which it was extracted and to abliterate it, project it out of the stream.

It works almost surgically on Llama-3-8b --- there's a small rise in perplexity, the MMLU is nearly unchanged and the generations are otherwise fluent. The performance is also acceptable with Qwen-2.5-7B. However, on Gemma-3-12B, the outputs are garbage, suggesting that the intermediate arithmetic of vectors clearly drove the model out of distribution. To be precise, the MMLU dropped from 0.682 to 0.242, where a chance on the dataset would be 0.25 and NLL rose from 4.47 to 20.3.

After we validated the code for any possible bugs, we agent-searched the web for our problem and stumbled upon these sources:

• A HF post on abliteration by Maxime Labonne: This mentions the apparent difficulty of creating abliterated gemma models; this is presumably a follow-up to his previous post describing two methods of abliteration and particularly deals with the weight-space version, so it's not directly related to our project.
• This HF blogpost on projected abliteration by Jim Lai (grimjim) - Where they mention in a paragraph that intermediate calculations required fp32 in Gemma-3 and how they had to use a 99.5% Winsorization to the activation before extracting the diff.of.means direction to avoid incoherent outputs

We applied the fix suggested in the Lai's post and it worked, and then the ensuing investigation lead to this report.

The problem is A channel and it has nothing to do with refusal

Channel 2339 was found to be, consistently and virtually unchallenged (98% of the layers), the largest-mean (626.5x times the typical co-ordinate and 25x to 90x the second-largest) and largest-variance co-ordinate of Gemma-3-12B's residual stream at essentially every layer from output of block 0 to layer 46, at every token position across all the prompts. This result apart from being unintuitive, contrasts with Qwen-2.5-7b and Llama-3.1-8b, where even the best possible candidate channels for this phenomenon could barely be the top co-ordinate in half of the layers and displayed much smaller magnitude gain with of about 7 in either compared to of 45 to 300 for the former.

To confirm that this is not something specific to the safety dataset or safety domain in general, same signature was observed on unrelated SST-2 datasets. It also surfaced a different result that proved to be important for a mechanistic claim later on.

It's unclear why this happens and we could say something about the architecture, but prior to that, there are two more prescient questions --- why would this make the extracted diff. of means a bad estimator and intervention and then more importantly, why does the fix work.

The fix and showing it works quite well!

Before looking into the research questions, I empirically validated the fix suggested earlier (and also a few more with slight variations)

There were three types of fixes that were tested:

• Winsorization: Clip activation magnitudes at a quantile before taking the difference of means; where is the chosen magnitude-quantile threshold.
• Masked: Identifying co-ordinates where the absolute mean exceeds 50x the layer median on a "harmless" split and then exclude them before differencing;
• Standardized: Divide the channel gap with the co-ordinate variance;

The recipes were selected keeping in mind the expected generalizability of it when applied across different settings; for example, the masked version bases the median on a "harmless" split without optimizing for a harmless-harmful class gap.

At the time of experiments, I had a weaker theory based on the observation that channel's magnitude at certain layers had some correlation with length of the prompt at certain layers. It's tempting to suggest based on this prior that the contrastive vector primarily captures an existing length-gap register between the two classes at the dominant channel . So, these were the other fixes tested:

• Length-matched: Resampling the two sets to equalize prompt lengths.
• Covariate-adjusted: Using the class coefficient from a co-ordinate-wise regression controlling for word count and terminal punctuation

The original setup with main contrast was maintained; derived with harmful prompts from AdvBench and SORRY-Bench against harmless prompts from Alpaca. Capability uses ~500 question MMLU slice and is supplemented with negative log-likelihood calculation. A bandpass filter was used combined with NLL check to catch degenerate outputs, looking at combination of repetition and number of unique tokens divided by the total (the latter based on a unique signature of degeneracy that was encountered in an experiment)

These were the results:^[2]

The three scale corrections recovered usable ablation directions and worked well in inference-time abliteration while not disrupting the fluency or capability of the model with non-breaking changes in llama and qwen models. They also generalize to two more contrasts - (StrongREJECT versus WildJailbreak-benign) and (WildJailBreak versus OR-Bench hard-benign) - with multiple candidate directions satisfying feasibility.

The behavioral suppression was not as prominent, with refusal reaching 0.19 and 0.09 on the harmful class in both contrasts compared to clean 0.7 and 0.44. This however is surprisingly better than with Llama-3-8B where the refusal does not drop below 0.36 and 0.69 in these pairs.

Finding the phenomenon and validating the correction across the ladder

The pipeline was also run on four Gemma-3 sizes, two Gemma-2 sizes and the two controls. A dominant channel index was re-discovered for each Gemma-3 model with others not providing much apart from strong candidates.

The feasibility ratios were calculated across the ladder and as expected, the recipes could propose valid candidates where the raw recipe failed otherwise. What was more interesting was the improvement in filtration rate for the proposed corrective recipes against the raw diff-of-means direction, even for other models.

Why does the fix work?

Math about the Differential update

The residual stream can be written as: , where is the loud co-ordinate axis and lies off that axis.

The difference of means can be written as

where the unit-normalized

and then decomposing

where .

The abliteration formula is^[4]

Quantifying the update to and in the raw direction case gives us

This means there's a massive update to the stream in either direction, since we have a potentially huge factor.

In the masked case, , therefore the updates are

which removes it.

A toy model

The math looks embarrassingly simple but pretty unconvincing, so we can do a quick toy example.

Suppose, we are working with a 5-channel embedding which typically has a shape like:

The loud co-ordinate is 0 and let's say refusal direction is clearly built in as

Let's suppose that the estimator we derive is perfect, just maybe scaled - in which case, we could get a vector like -

This works perfectly fine with abliteration regardless of how loud the co-ordinate is, we would always get a vector like

, whether is or

There are two confounds, that however will almost certainly always happen in the real case -a) stochastic noise b) spurious weak correlation - both of which have opposite effects with sample size.

Let's assume, for either of the reasons the estimator in the previous case was - something not unrealistic since an affect of atleast 20 with a mean of 10000, considering a reasonable of 100 and suppose is very likely under either scenario. This means

yields

The intended feature is not zero-ed out, it's absolutely destroyed and the loud co-ordinate absolutely collapses. Coming back to the confounds question, the nature of it matters more, because scale (increasing the sample size) does not fix one of the two issues.

I did 1e6 simulated estimates per condition with a seed of 0 for 5-dimensional embeddings. Each co-ordinate is Gaussian and independent and has standard deviations of and true population class gaps of and representing absence and presence of a true confound. Then, I calculated the energy share of the co-ordinate.

When the true gap is 0, the loud co-ordinate drowns out even with a massive gain in variance but with a genuine confound, the energy only concentrates there.

Empirical Residual Stream Profiling

An intuitive reading of the math and the toy model tells us that what really drives the model to garbage outputs could be out-of-distribution activations. Without going into individual co-ordinates and inspecting what their gains were, I simply took the distribution of the median-magnitudes of co-ordinates across 100 prompts at a fixed token position and plotted how the shape changes between applying raw and corrected directions.

The results are pretty striking, the profiles practically overlap in every case, except for ablation under a raw direction for Gemma-3-12B. This is also intuitively why the model's activations literally go out of distribution

It's also more likely the loud channel does not register (just) noise, as it was noticed that at (4 positions before the post instruction token), the standardized class gap was a very significant 3.2 (higher than any other co-ordinate at that position) whereas it was only -0.11 for a "sentiment-contrast" in the SST-2 dataset. Note this result only means a real class association may be true.

Most of the class separation is still diffuse

Adding onto the last paragraph from the earlier section, while the loud coordinate carries a real class association, it does not account for most of the standardized separation between harmful and harmless prompts. When measuring the energy share of the top co-ordinates in the standardized difference of means, the findings are - top-10 accounting for 3%, top-100 for 16-18% and then top-thousand for 72-75%. By this measure, class separation is spread across many coordinates. Fitted probes, across layers also report an AUROC of close to ~0.9 with and without taking the loud co-ordinate into account.

Useful complementary experiments

Induction

Inducing refusal is not an exact inverse of abliteration, as it's just additive steering where is a hyperparameter we control

With the direct edits, we get increment of

Comparing the residual profiles similar to how we did in the previous case, gives us plots where we could see that while norms of co-ordinates Gemma-3-12B does not deviate as much drastically, the corrected version don't offer better alternatives than the raw directions, in fact, on Gemma, it's noticeably worse (ostensibly)

Despite this observation, the reported outcome on a limited set of 32 prompts shows that induction works better with "recipe" corrected directions nonetheless with raw steering yielding only degenerate outputs. ^[5]

The causality of the loud co-ordinate

By isolating the causal qualitative impact of both the vectors

• a) impact of dampening the loud co-ordinate ()
• b) impact of loud co-ordinate's magnitude causing a gain on other co-ordinates. (;

we get these results:

It's clear that under either ablation, the MMLU drops to chance on the models where a clear loud co-ordinate exists but more interestingly, there's a substantial drop in performance for Gemma-2 models in the second column. The last column is more what would be a prior expectation.

Conclusions

The main takeaway

This article shows that in a regular difference of means experiment, apart from the consideration of the design of the experiment like choosing the right prompt format and contrast, sweeping over layers, modules and doses etc. the experimenter must also look at the signature of the difference of means itself. The best recommendation, based on evidence and being more principled, would probably to use a channel standardized difference of means where the per-channel variance is estimated based on neutral prompts.

How this changes the interpretation of prior reports

Difficulty removing refusal is not sufficient evidence of inseparability from capability. Our corrected Gemma-3-12B intervention suppresses refusal while matching clean on the measured MMLU slice. This provides a counterexample within this setup to interpreting destructive raw ablation as unavoidable capability loss. It does not establish that all refusal behavior is independent of all capabilities.

Outlier correction has clear precedent. Lai's projected-abliteration report already describes magnitude clipping to prevent incoherence. His later norm-preserving approach emphasizes preserving activation geometry. Our channel-preserving correction is compatible with that concern, while changing the estimated direction rather than deleting the residual coordinate.

Refusal geometry is a separate question. Wollschläger et al. and Joad et al. examine richer refusal structure. Our result concerns a failure of raw estimation and projection. It neither proves a universal one-dimensional refusal representation nor refutes multidimensional accounts.

Highly aligned means can complicate direction selection. COSMIC reports unusually high harmful/harmless activation similarity on Gemma-2-27B. This concerns activation similarity, not simply pairwise similarity among candidate refusal vectors. Our scale diagnosis offers a possible connection, but we have not tested Gemma-2-27B and do not claim to explain all of its results.

Precision and geometry are distinct problems. Transformers issue #39972 and PR #37226 discuss Gemma-3 activation ranges and fp16 overflow. Our raw-estimator failure persists with fp32 estimation. Higher precision does not, by itself, remove the large component of a mathematically well-defined direction. Conversely, fp32 estimation does not prove that every possible numerical issue has been eliminated.

What remains open

The central finding is bounded but useful: in this pipeline, a large coordinate can compromise both direction estimation and projection. Excluding it from the estimate restores usable ablation while preserving the coordinate in the model. Whether the same correction works elsewhere remains a measurement to make.

Why the channel develops, and why it becomes so large. The base checkpoint establishes that the channel exists before instruction tuning^[7], and the weight scan identifies learned normalization gains that can amplify it. Neither explains why training develops this concentrated activation pattern, what function it serves, or why Gemma-3 amplifies it more strongly than the tested Gemma-2 checkpoints. Its contribution to the RMS denominator provides a route for influencing other coordinates, but whether training uses it as a gain-control mechanism remains untested.

References and data

Papers, implementation reports, and user discussions provide different kinds of evidence. Inclusion below does not imply that each establishes the mechanism tested here.

Method and prior reports

• Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., & Nanda, N. (2024). Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024.
• Labonne, M. (2024). Uncensor any LLM with abliteration.
• Labonne, M. (2025). Gemma 3 Abliterated (17 March 2025) and the gemma-3-27b-it-abliterated model card.
• Lai, J. ("grimjim") (2025). Projected Abliteration (25 October 2025).
• Siu, V., et al. (2025). COSMIC: Generalized Refusal Direction Identification in LLM Activations.
• The Curious Case of the bos_token, LessWrong.
• gau-nernst. [RFC] Fix Gemma 3 FP16 with activation scaling, huggingface/transformers PR #37226.
• DalasNoin (2024). Currently not working with Gemma 2 models, issue #4 on andyrdt/refusal_direction (2 July 2024).
• Siu et al. (2025). SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectives.
• Gusev, I. (2024). gemma-2-9b-it-abliterated model card (15 July 2024).
• Lai, J. ("grimjim") (2025). Norm-Preserving Biprojected Abliteration (6 November 2025).
• p-e-w. Heretic.
• Orion-zhen. abliteration (toolchain README; upstream of several widely-used forks).
• huihui-ai community discussion, gemma-3-12b-it-abliterated #2 (March 2025), and Otakadelic in zelk12/MT-gemma-3-12B #1 (May 2025 – January 2026).
• Khanna, R. (2026). I Abliterated Gemma 4 on a MacBook (13 April 2026).
• Wollschläger, T., et al. (2025). The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence. ICML 2025.

Cross-model evaluations that include Gemma

• Wang, X., Wang, M., Liu, Y., Schütze, H., & Plank, B. (2025). Refusal Direction is Universal Across Safety-Aligned Languages. NeurIPS 2025.
• Joad, F., Hawasly, M., Boughorbel, S., Durrani, N., & Sencar, H. (2026). There Is More to Refusal in Large Language Models than a Single Direction.
• Truong, N. (2026). Abliteration Mitigation via Refusal Aliases.
• Ye, J., et al. (2026). Where Steering Signals Come From: Activation Source Selection in Activation Steering.
• Aparin, G., & Gaintseva, T. (2026). A Geometric Account of Activation Steering through Angle–Norm Decomposition.
• Bhandari, S., & Wade, B. (2026). Steering Under Compression: Dose-Response, Capability Cost, and Failure Asymmetry in Quantized LLMs.
• Srivastava, A., & Panda, S. (2026). Attention Is Where You Attack.
• Young, R. J. (2025). Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluation.

Outlier coordinates

• Timkey, W., & van Schijndel, M. (2021). All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality. EMNLP 2021.
• Sun, M., Chen, X., Kolter, J. Z., & Liu, Z. (2024). Massive Activations in Large Language Models.
• Chen, L., et al. (2026). Measuring Maximum Activations in Open Large Language Models.
• Queipo-de-Llano, E., et al. (2026). Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin. ICLR 2026.
• Roh, Y., Cho, H., & Kim, J. (2026). Embracing Anisotropy: Turning Massive Activations into Interpretable Control Knobs for Large Language Models. ACL 2026.
• Macocco, I., Graichen, N., Boleda, G., & Baroni, M. (2025). Not a nuisance but a useful heuristic: Outlier dimensions favor frequent tokens in language models.
• Lieberum, T., et al. (2024). Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2. BlackboxNLP @ EMNLP 2024.
• Gemma3 with fp16 in inference, huggingface/transformers issue #39972 (6 August 2025).

Models, prompts, and artifacts

• Gemma 3 Technical Report, Gemma Team (2025).
• Instruction-tuned checkpoints: gemma-3-1b-it, gemma-3-4b-it, gemma-3-12b-it, gemma-3-27b-it, gemma-2-2b-it, gemma-2-9b-it, Llama-3-8B-Instruct, and Qwen2.5-7B-Instruct. Additional base comparison: gemma-3-12b-pt.
• Main harmful prompts: AdvBench and SORRY-Bench. Main harmless prompts: Alpaca. Dataset swaps: StrongREJECT, WildJailbreak, and OR-Bench. Unrelated contrast: SST-2. Capability: a 500-question MMLU slice. Extraction/evaluation splits are disjoint, frozen, and hash-checked.
• Code and artifacts: abhishek9909/loud-channel-abliteration. The artifact layout records residual profiles in artifacts/<model>/recipes/residual_profile.json, plotted by scripts/plot_residual_profile.py.

The numerical results in this post are from our experimental runs. Related-work citations provide context rather than independent verification of those run values.

1. ^{^}

   Note that the direction has to pass a feasibility filter, which is detailed in the next footnote.

2. ^{^}

   Feasibility is based on a two-pass filter , first the ability of the model to "induce" refusal and next to do it non-catastrophically on a harmless prompt. (based directly on Arditi et. al.'s methodology); this also acts as a way to select the best possible direction to apply refusal. The per-layer baseline applies a layer-specific direction rather than reusing one selected direction throughout the network. Its result should be interpreted as a separate intervention configuration, not as a raw candidate that passed the single-direction selection gate.

   For each single-direction estimator, we evaluate candidate source layers and token positions. We select the candidate with the lowest harmful-prompt refusal score under ablation, subject to positive refusal induction on harmless prompts, harmless-prompt KL below 0.1 under ablation, and a source layer below 80% of model depth. The selected vector is then used throughout the stated ablation sites. This selects where the direction is extracted, rather than restricting ablation to that layer.

   Length matching and covariate adjustment do not recover any feasible candidate on Gemma-3-12B, although both retain feasible candidates on Llama and Qwen. On Gemma, the adjusted directions remain close to raw, with cosines of 0.994 and 0.996 respectively, and still place approximately 23–26% of their squared length on the loud coordinate. Controlling these covariates is therefore insufficient; the successful corrections directly address coordinate scale.

   Model	Length-matched feasible cells	Covariate-adjusted feasible cells
   Gemma-3-12B	0 / 288	0 / 288
   Llama-3-8B	47 / 192	44 / 192
   Qwen2.5-7B	12 / 168	12 / 168

3. ^{^}

   Note that we follow the Arditi recipe where we use KL filter of <0.1 to pre-filter a candidate direction. This makes a KL in O(10) is especially out of distribution.

4. ^{^}

   Notice we don't normalize the dot product, so in principle, the negative dose depends not just on similarity but magnitude of the existing refusal component in the vector as well.

5. ^{^}

   It's important to note that the selected here is based on a sweep where the value of is determined and then multiplied with which is the residual norm for off co-ordinates in the corrected recipes and includes the norm in the raw direction recipe. The fact that standardization was better masked updates could mean something about the "betterness" of smoothing the co-ordinates by their variance as opposed to masking an arbitrary number but such claims need backing by more theory and experiments.

6. ^{^}

   Note the dose was much higher for the raw case than corrected version, as it was selected based on the previous footnote.

7. ^{^}

   The pretrained gemma-3-12b-pt checkpoint already carries coordinate 2339, with the same final-norm rank and a writer gain within half a percent of the instruction-tuned model. In this weights-only scan the values are 555 and 553; this uses a different aggregation from the earlier 623 statistic.

   The coordinate therefore predates instruction tuning. That rules out its being created solely by instruction or safety post-training. This comparison does not by itself establish identical intervention behavior in the base model.