Claim: Different post-training methods install different refusal circuits, and they are consistent with different attack class vulnerabilities.
Results:
Training method, not just data, reshapes how refusal is computed internally: reasoning-augmented training (Ra-SFT) consistently produces a distinct kind of refusal computation, visible across all three models,
Circuit topology is partly architecture dependent: Qwen has a refusal circuit dominated by MLPs in all methods, while Llama has circuits moving from dominated by attention heads in SFT to dominated by MLPs in ORPO.
No post-training methods that we analyse (SFT, Ra-SFT and ORPO) can satisfy three conditions at once: refusal not concentrated in few fragile components, safety gains that don’t cost general capability, and safety behaviour correctable through small, targeted edits.
Takeaways: Our mechanistic analysis shows that each post-training method would produce a different vulnerability profile that standard behavioral analysis cannot detect, and therefore we caution against treating safety alignment as binary property. Further analysis is warranted to understand when and how refusal mechanism evolves inside LLMs, and therefore to design better post-training methods for safety alignment.
Introduction
One of the most common ways that are recommended to ensure model safety is to fine-tuning or use other post-training techniques. As Lucas Beyer noted in 2024 (here):
“the cheapest way to hit low loss on safety data is a keyword-triggered shortcut -- learn to detect a topic, flip into a single, easily-findable "refusal mode."”
It also turns out that national cybersecurity agencies like BSI or CISA have the same ideas as well - their recommendations for LLM safety include “fine-tuning” and “reinforcement learning with human feedback (RLHF)”, and therefore treating alignment as a binary property. Yet cybersecurity incidents like the attacks on Mexican government agencies in early 2026, or GTG-1002 in 2025, with attackers using Claude Code/GPT-4.1 (models that have been thoroughly post-trained for safety) show that behavioural evaluation of alignment is insufficient.
Given that refusal direction is known as existed in each model (Arditi et al. (2024) establishes this), the question for our research is “does which post-training method you use change how refusal direction looks like and how a shortcut - in a sense of “keyword matching” like Beyer said - gets built?”
Setup
For our analysis to proceed - we fine-tune three different open-weight models (Llama-3.1-8B, Gemma-2-9B and Qwen3-8B). Each model was fine-tuned with three different objectives, based on preference-reasoning axes (figure 1):
Figure 1: Two-axis paradigm structure. SFT: no preference, no reasoning. Ra-SFT: no preference, reasoning supervision. ORPO: preference, no reasoning. Fourth cell left for future work.
SFT (supervised fine-tuning) - for pure imitation learning, no preference or reasoning added,
Ra-SFT (reasoning-augmented SFT): where a model would be fine-tuned with results and reasoning chains generated from a frontier model;
The fine-tuning processes use matched data (16k Alpaca + 4k BeaverTails), and matched parameters - only training objectives vary. We do not use online RL methods like GRPO/PPO, since its reward/update process would defeat our aim of matched data during training. We also fine-tune it from base models (not instruct ones) to ensure a clean comparison (not being confounded by different RL pipelines that labs used to post-trained instruct models) - and to make sure that training objectives are the only varying factor.
For our analysis we used difference-in-means refusal direction extraction from Arditi et al (2024) for geometric analysis of refusal in activation space. Then we use activation patching/attribution patching to check what components are actually causally relevant for refusal. Finally we use ActAdd (activation addition - Turner et al (2023)) and ITI (inference-time intervention - Li et al (2023)) for steering models toward safety.
Findings
Training objectives reshape refusal geometry
We found that SFT and ORPO are much closer to each other regarding refusal geometry, than either is to Ra-SFT, and this is consistent across three architectures. Ra-SFT installs a qualitatively different pathway across all three architectures, and this replicates across three different models, so it is the objective, not the architecture, doing this (in figure 2).
Figure 2: Cosine similarity comparison between SFT, Ra-SFT and ORPO in Llama-3.1-8B, Gemma-2-9B and Qwen3-8B.
We analyse the refusal direction magnitude in each model (figure 3), and find out that three models differ on which objectives reconverge in mid-network layers before diverging again late.
For Llama-3.1-8B, while SFT has its refusal direction magnitude peaked at mid-late layers (20-24) before declining and rising again, Ra-SFT and ORPO don't have such a clear peak. For Qwen3-8B, SFT and ORPO both peak in mid-layers (22–27 and 22–30 respectively) before declining, while Ra-SFT’s magnitude rises only gradually across the network in both models, never sharply peaking.
Meanwhile for Gemma-2-9B, it breaks from this pattern for ORPO specifically, which overshoots in late layers rather than peaking mid-network as it does in the other two models. We suspect this reflects an interaction between ORPO’s unconstrained odds-ratio gradient and Gemma-2-9B’s representation geometry - plausibly shaped by distillation from Gemma-2-27B (Gemma Team, 2024) - concentrating gradient updates intothe available direction at high magnitude - though it is just an assumption.
Figure 3: Normalized refusal direction magnitude of Llama-3.1-8B, Gemma-2-9B and Qwen3-8B across training objectives by layers.
We also calculate refusal direction stability (by bootstrapping 64 iterations out of 256 pairs of harmful/harmless prompts that we test from Arditi et al (2024) dataset), and found that refusal direction that we extracted is mostly stable across iterations (cosine similarity across layers roughly 0.9, except for Qwen where cosine similarity in early layers being roughly 0.7-0.8). The reasons behind such an unstable direction of Qwen in early layers is unknown, though (figure 4).
Figure 4: Refusal direction stability (Bootstrap cosine similarity) in our analysed models across layers.
How circuit topology is (partly) training-objective-dependent
All training methods show peak causal effects in late layers of LLMs (figure 5), but how they get there and how each component of a model (attention heads/MLPs) interact with refusal are dependent on training objectives too.
Figure 5: Normalized causal effects of residual-stream activation patching across layers for Llama-3.1-8B, Gemma-2-9B and Qwen3-8B under three post-training methods: SFT, Ra-SFT, and ORPO. Higher values indicate stronger causal contribution to the target refusal-related behavior.
Component-level structure: In Llama-3.1-8B, we observe that systematic shift from attention-head dominance to MLP dominance along the SFT→Ra-SFT→ORPO progression: head 25 carries the dominant causal effect under SFT (−0.33 at layer 30), MLP layer 31 dominates under Ra-SFT (+0.90), and ORPO shows a diffuse MLP distribution (in figure 6).
Note that the + sign means that ablating a model component (MLP/attention heads) would harm refusal behavior, and the - sign means that such ablation would promote refusal.
Figure 6: Normalized causal effects of activation patching in top 5 MLPs and attention heads with highest contributions to refusal in Llama-3.1-8B across post-training methods.
In Gemma-2-9B we observe that SFT and ORPO both show uniform, redundant encoding (+0.24 - +0.47 across all components). Ra-SFT breaks this pattern with an uneven, MLP-dominated structure (layer 39 MLP −0.44; layer 37 MLP +0.25) alongside uniformly small attention-head effects (−0.09 to +0.08) – structurally closer to Llama’s Ra-SFT circuit than to Gemma’s own SFT or ORPO, suggesting reasoning-chain supervision installs a qualitatively distinct circuit type largely independent of architecture (figure 7).
It is worth noting that Gemma-2-9B and Qwen3-8B experience multiple sign flips between attribution patching results and activation patching ones in MLP level. This is because training objectives are reshaping how MLPs interact with refusal in a non-linear way, by promoting some MLPs compared to gradients while suppressing others.
Figure 7: Normalized causal effects of activation patching in top 5 MLPs and attention heads with highest contributions to refusal in Gemma-2-9B across post-training methods.
In Qwen3-8B, all three objectives show MLP-dominant circuits (figure 8), with top-layer MLPs promote refusal under both SFT and ORPO, while under Ra-SFT, layer 31’s MLP strongly suppresses it (−0.36). Attention heads play a comparatively minor role throughout: even ORPO’s highest-effect head (head 11, layer 31, +0.19) is outweighed by layer 32’s MLP (+0.48). This contradicts both Zhou et al. (2025) and Huang et al (2025) findings that attention heads play a big role in LLM safety, and this also does not follow the clean architecture story - we leave the reasons for this difference for future work.
Figure 8: Normalized causal effects of activation patching in top 5 MLPs and attention heads with highest contributions to refusal in Qwen3-8B across post-training methods.
How refusal circuits reshape steering effectiveness
From Wu et al (2026) who decompose refusal into recognition/execution axes, we conduct ActAdd into two types of layers:
Peak recognition layers (where normalized direction magnitude is highest in figure 3).
Peak execution layers (where causal effects being the highest in figure 5).
Our analysis shows that steering at peak recognition layers ended up improving refusal performance better compared to steering at peak execution layers, across all models that we test - provided that there is a gap between recognition/execution layers (Gemma/Qwen, in figures 9 and 10 respectively).
Figure 9: ActAdd effects for Gemma-2-9B SFT and Ra-SFT when steering at peak recognition layers versus peak execution layers.
Figure 10: ActAdd effects for Qwen3-8B SFT and ORPO when steering at peak recognition layers versus peak execution layers.
MMLU accuracy stays stable throughout ActAdd steering in Gemma-2-9B (53–55% SFT, 40–44% Ra-SFT, 45–47% ORPO), indicating orthogonal safety and utility representations. Meanwhile Llama has its capability (to answer questions in MMLU) collapses after steering in lower alpha (in table 1).
Qwen3-8B sits between the two: MMLU stays relatively stable under execution-layer steering but collapses under recognition-layer steering, making it an intermediate case between Llama/Gemma.
ActAdd scale
SFT
Ra-SFT
ORPO
α = 0
35.5
31.5
28.5
α = 5
16.5
24.5
11.5
α = 10
0.0
4.0
0.0
Table 1: MMLU subset accuracy rate (%) under ActAdd steering for Llama-3.1-8B across post-training objectives. Lower accuracy rate indicates utility reduction.
We also conduct ITI for single-component steering, and this fails in different ways depending on circuit types: concentrated-suppressive (Llama SFT -> coherence collapse, single-token loops), distributed-promotive (Gemma SFT -> Hydra effect, no improvement), distributed-uniform (Gemma ORPO -> capability degradation). Especially in the case of Llama SFT when we use ITI, it ends up just repeating the question even with alpha = 0, and repeating token “I” when we steer at alpha = 20 (in figure 11).
Figure 11: Examples illustrating coherency collapse at α = 0 and α = 20 for Llama SFT ITI on a WildJailbreak prompt.
How refusal circuits reshape attack class vulnerability
We evaluate refusal performance of models using StrongREJECT (Souly et al (2024)), with LlamaGuard3-8B as judge, using Wei et al (2023) framework of jailbreak attacks:
Competing objectives: attacks occur when a model’s pretraining and instruction-following objectives are put at odds with its safety objective, such as happy_to_help, DAN, wikipedia, role_play;
Mismatched generalization: attacks occur when inputs are out-of-distribution for a model’s safety training data but within the scope of its broad pretraining corpus (rot-13, disemvowel).
We also include “none” as direct requests for baseline, and for the scope of our analysis we work only on template-based attacks, not dynamic jailbreaks like GCG, AutoDAN or PAIR.
We found out that refusal circuit structure is consistent with specific vulnerabilities patterns:
For Llama-3.1-8B and Gemma-2-9B, SFT is especially vulnerable to semantic attacks (happy_to_help, DAN, wikipedia, role_play) due to concentration of causal effects in attention heads, which are lexically sensitive (Ji et al. (2025)), while Ra-SFT and ORPO, given that they are not as concentrated in attention heads, successfully mitigates these attacks.
For Qwen3-8B: given that all training objectives are dominated by attention heads; all of them ended up with low attack success rate in semantic attacks, but they are all vulnerable to encoding attacks (rot-13, disemvowel).
The heatmap of attack class vulnerability across different training objectives is shown in figure 11).
Figure 11: Attack success rate (ASR) of Llama-3.1-8B, Gemma-2-9B and Qwen3-8B across training objectives, alongside their base models, on StrongREJECT attack classes (n = 60 per class).
Safety alignment trilemma and operational implications
Across every objectives tested in three architectures, there are three conditions that no post-training methods that we analyse (SFT, Ra-SFT, ORPO) can satisfy simultaneously:
(1) Distributed refusal encoding: refusal circuits should not be over-concentrated in any component types;
(2) Safety/utility separability: steering towards safety should preserve model capability;
(3) Granular correctability: safety behavior should be correctable through localized interventions, such as component-level edits.
In our work: Ra-SFT can be partially correctable, but they produce chain-of-thoughts (CoTs) that increase token costs for end consumers - not to mention that CoTs can be unfaithful. ORPO can have best refusal performance, but its distributed encoding in all cases also defeats correctability. SFT by contrast is cheapest, but as our work shows it is also concentrated in few components - so it is highly vulnerable to different attacks.
Therefore we caution against treating safety alignment as a binary property that post-training models can have - since our circuit analysis predicts vulnerability profiles that behavioral evaluation cannot predict.
Why our results matter
As Qi et al (2025) said - shallow alignment is indicated when a model guardrail can be bypassed in just a few tokens deep. Our analysis moves deeper than this, by showing that shallow alignment can happen when a model has over-concentrated refusal circuits in a few components, and that is after fine-tuning. We also indicate that refusal circuits can be consistent with different attack class vulnerabilities - which is especially useful as interpretability research is increasingly more integrated with AI security.
Our analysis of refusal circuits across different training objectives is also useful for a deeper understanding of how training dynamics look like - especially as Biderman et al (2026) said that, even as concerns of deeper understanding in safety/alignment work being too impractical given deployment pressures are legitimate, this leads to post-training interventions to reduce harms, and this attitude is directly behind many recurring safety failures (e.g. jailbreaks, regressions, brittleness under scale).
Limitations
While our analysis so far has been highly effective, they are done in single-scale per architecture (at 8-9B) - therefore how our findings translate to larger-scale models, especially frontier ones, are genuinely uncertain. Also, we did not test how reasoning and preference would interact together in safety alignment; this would require constructing chain-of-thoughts for unsafe responses of BeaverTails, which would be hard to get given stronger safety alignment of frontier LLMs.
Conclusion
In this work, we conduct a cross-paradigm analysis of refusal circuits across post-training methods. By characterizing post-training methods across preference-reasoning axes, we show that training objectives can reshape how refusal is implemented inside LLMs, which also impacts effectiveness of steering and attack class vulnerability.
Beyond our analysis, we also see many interesting research directions:
How would parameter-efficient fine-tuning methods (quantization, low-rank adaptation) reshape refusal circuits? Given that these are practical for fine-tuning larger models, this is a promising approach.
Temporal evolution of refusal circuits in post-training methods; given that Ge et al (2026) deployed crosscoders to analyse evolution of concepts during pre-training, can we do something similar in different training epochs? This can be helpful for designing new post-training methods as well.
We hope our mechanistic analysis of post-training methods will serve as a useful resource and contribute to future research on LLM alignment and safety. We are also looking to discuss why, despite our stated problems regarding shallow alignment of post-training methods, they still persist - there can be economic incentives behind why labs are still working this way.
This post is a condensed version of our EMNLP 2026 paper - you can take a look here: https://arxiv.org/abs/2609.03887
TL;DR
Claim: Different post-training methods install different refusal circuits, and they are consistent with different attack class vulnerabilities.
Results:
Takeaways: Our mechanistic analysis shows that each post-training method would produce a different vulnerability profile that standard behavioral analysis cannot detect, and therefore we caution against treating safety alignment as binary property. Further analysis is warranted to understand when and how refusal mechanism evolves inside LLMs, and therefore to design better post-training methods for safety alignment.
Introduction
One of the most common ways that are recommended to ensure model safety is to fine-tuning or use other post-training techniques. As Lucas Beyer noted in 2024 (here):
“the cheapest way to hit low loss on safety data is a keyword-triggered shortcut -- learn to detect a topic, flip into a single, easily-findable "refusal mode."”
It also turns out that national cybersecurity agencies like BSI or CISA have the same ideas as well - their recommendations for LLM safety include “fine-tuning” and “reinforcement learning with human feedback (RLHF)”, and therefore treating alignment as a binary property. Yet cybersecurity incidents like the attacks on Mexican government agencies in early 2026, or GTG-1002 in 2025, with attackers using Claude Code/GPT-4.1 (models that have been thoroughly post-trained for safety) show that behavioural evaluation of alignment is insufficient.
Given that refusal direction is known as existed in each model (Arditi et al. (2024) establishes this), the question for our research is “does which post-training method you use change how refusal direction looks like and how a shortcut - in a sense of “keyword matching” like Beyer said - gets built?”
Setup
For our analysis to proceed - we fine-tune three different open-weight models (Llama-3.1-8B, Gemma-2-9B and Qwen3-8B). Each model was fine-tuned with three different objectives, based on preference-reasoning axes (figure 1):
Figure 1: Two-axis paradigm structure. SFT: no preference, no reasoning. Ra-SFT: no preference, reasoning supervision. ORPO: preference, no reasoning. Fourth cell left for future work.
The fine-tuning processes use matched data (16k Alpaca + 4k BeaverTails), and matched parameters - only training objectives vary. We do not use online RL methods like GRPO/PPO, since its reward/update process would defeat our aim of matched data during training. We also fine-tune it from base models (not instruct ones) to ensure a clean comparison (not being confounded by different RL pipelines that labs used to post-trained instruct models) - and to make sure that training objectives are the only varying factor.
For our analysis we used difference-in-means refusal direction extraction from Arditi et al (2024) for geometric analysis of refusal in activation space. Then we use activation patching/attribution patching to check what components are actually causally relevant for refusal. Finally we use ActAdd (activation addition - Turner et al (2023)) and ITI (inference-time intervention - Li et al (2023)) for steering models toward safety.
Findings
Training objectives reshape refusal geometry
We found that SFT and ORPO are much closer to each other regarding refusal geometry, than either is to Ra-SFT, and this is consistent across three architectures. Ra-SFT installs a qualitatively different pathway across all three architectures, and this replicates across three different models, so it is the objective, not the architecture, doing this (in figure 2).
Figure 2: Cosine similarity comparison between SFT, Ra-SFT and ORPO in Llama-3.1-8B, Gemma-2-9B and Qwen3-8B.
We analyse the refusal direction magnitude in each model (figure 3), and find out that three models differ on which objectives reconverge in mid-network layers before diverging again late.
For Llama-3.1-8B, while SFT has its refusal direction magnitude peaked at mid-late layers (20-24) before declining and rising again, Ra-SFT and ORPO don't have such a clear peak. For Qwen3-8B, SFT and ORPO both peak in mid-layers (22–27 and 22–30 respectively) before declining, while Ra-SFT’s magnitude rises only gradually across the network in both models, never sharply peaking.
Meanwhile for Gemma-2-9B, it breaks from this pattern for ORPO specifically, which overshoots in late layers rather than peaking mid-network as it does in the other two models. We suspect this reflects an interaction between ORPO’s unconstrained odds-ratio gradient and Gemma-2-9B’s representation geometry - plausibly shaped by distillation from Gemma-2-27B (Gemma Team, 2024) - concentrating gradient updates intothe available direction at high magnitude - though it is just an assumption.
Figure 3: Normalized refusal direction magnitude of Llama-3.1-8B, Gemma-2-9B and Qwen3-8B across training objectives by layers.
We also calculate refusal direction stability (by bootstrapping 64 iterations out of 256 pairs of harmful/harmless prompts that we test from Arditi et al (2024) dataset), and found that refusal direction that we extracted is mostly stable across iterations (cosine similarity across layers roughly 0.9, except for Qwen where cosine similarity in early layers being roughly 0.7-0.8). The reasons behind such an unstable direction of Qwen in early layers is unknown, though (figure 4).
Figure 4: Refusal direction stability (Bootstrap cosine similarity) in our analysed models across layers.
How circuit topology is (partly) training-objective-dependent
All training methods show peak causal effects in late layers of LLMs (figure 5), but how they get there and how each component of a model (attention heads/MLPs) interact with refusal are dependent on training objectives too.
Figure 5: Normalized causal effects of residual-stream activation patching across layers for Llama-3.1-8B, Gemma-2-9B and Qwen3-8B under three post-training methods: SFT, Ra-SFT, and ORPO. Higher values indicate stronger causal contribution to the target refusal-related behavior.
Component-level structure: In Llama-3.1-8B, we observe that systematic shift from attention-head dominance to MLP dominance along the SFT→Ra-SFT→ORPO progression: head 25 carries the dominant causal effect under SFT (−0.33 at layer 30), MLP layer 31 dominates under Ra-SFT (+0.90), and ORPO shows a diffuse MLP distribution (in figure 6).
Note that the + sign means that ablating a model component (MLP/attention heads) would harm refusal behavior, and the - sign means that such ablation would promote refusal.
Figure 6: Normalized causal effects of activation patching in top 5 MLPs and attention heads with highest contributions to refusal in Llama-3.1-8B across post-training methods.
In Gemma-2-9B we observe that SFT and ORPO both show uniform, redundant encoding (+0.24 - +0.47 across all components). Ra-SFT breaks this pattern with an uneven, MLP-dominated structure (layer 39 MLP −0.44; layer 37 MLP +0.25) alongside uniformly small attention-head effects (−0.09 to +0.08) – structurally closer to Llama’s Ra-SFT circuit than to Gemma’s own SFT or ORPO, suggesting reasoning-chain supervision installs a qualitatively distinct circuit type largely independent of architecture (figure 7).
It is worth noting that Gemma-2-9B and Qwen3-8B experience multiple sign flips between attribution patching results and activation patching ones in MLP level. This is because training objectives are reshaping how MLPs interact with refusal in a non-linear way, by promoting some MLPs compared to gradients while suppressing others.
Figure 7: Normalized causal effects of activation patching in top 5 MLPs and attention heads with highest contributions to refusal in Gemma-2-9B across post-training methods.
In Qwen3-8B, all three objectives show MLP-dominant circuits (figure 8), with top-layer MLPs promote refusal under both SFT and ORPO, while under Ra-SFT, layer 31’s MLP strongly suppresses it (−0.36). Attention heads play a comparatively minor role throughout: even ORPO’s highest-effect head (head 11, layer 31, +0.19) is outweighed by layer 32’s MLP (+0.48). This contradicts both Zhou et al. (2025) and Huang et al (2025) findings that attention heads play a big role in LLM safety, and this also does not follow the clean architecture story - we leave the reasons for this difference for future work.
Figure 8: Normalized causal effects of activation patching in top 5 MLPs and attention heads with highest contributions to refusal in Qwen3-8B across post-training methods.
How refusal circuits reshape steering effectiveness
From Wu et al (2026) who decompose refusal into recognition/execution axes, we conduct ActAdd into two types of layers:
Our analysis shows that steering at peak recognition layers ended up improving refusal performance better compared to steering at peak execution layers, across all models that we test - provided that there is a gap between recognition/execution layers (Gemma/Qwen, in figures 9 and 10 respectively).
Figure 9: ActAdd effects for Gemma-2-9B SFT and Ra-SFT when steering at peak recognition layers versus peak execution layers.
Figure 10: ActAdd effects for Qwen3-8B SFT and ORPO when steering at peak recognition layers versus peak execution layers.
MMLU accuracy stays stable throughout ActAdd steering in Gemma-2-9B (53–55% SFT, 40–44% Ra-SFT, 45–47% ORPO), indicating orthogonal safety and utility representations. Meanwhile Llama has its capability (to answer questions in MMLU) collapses after steering in lower alpha (in table 1).
Qwen3-8B sits between the two: MMLU stays relatively stable under execution-layer steering but collapses under recognition-layer steering, making it an intermediate case between Llama/Gemma.
ActAdd scale
SFT
Ra-SFT
ORPO
α = 0
35.5
31.5
28.5
α = 5
16.5
24.5
11.5
α = 10
0.0
4.0
0.0
Table 1: MMLU subset accuracy rate (%) under ActAdd steering for Llama-3.1-8B across post-training objectives. Lower accuracy rate indicates utility reduction.
We also conduct ITI for single-component steering, and this fails in different ways depending on circuit types: concentrated-suppressive (Llama SFT -> coherence collapse, single-token loops), distributed-promotive (Gemma SFT -> Hydra effect, no improvement), distributed-uniform (Gemma ORPO -> capability degradation). Especially in the case of Llama SFT when we use ITI, it ends up just repeating the question even with alpha = 0, and repeating token “I” when we steer at alpha = 20 (in figure 11).
Figure 11: Examples illustrating coherency collapse at α = 0 and α = 20 for Llama SFT ITI on a WildJailbreak prompt.
How refusal circuits reshape attack class vulnerability
We evaluate refusal performance of models using StrongREJECT (Souly et al (2024)), with LlamaGuard3-8B as judge, using Wei et al (2023) framework of jailbreak attacks:
We also include “none” as direct requests for baseline, and for the scope of our analysis we work only on template-based attacks, not dynamic jailbreaks like GCG, AutoDAN or PAIR.
We found out that refusal circuit structure is consistent with specific vulnerabilities patterns:
The heatmap of attack class vulnerability across different training objectives is shown in figure 11).
Figure 11: Attack success rate (ASR) of Llama-3.1-8B, Gemma-2-9B and Qwen3-8B across training objectives, alongside their base models, on StrongREJECT attack classes (n = 60 per class).
Safety alignment trilemma and operational implications
Across every objectives tested in three architectures, there are three conditions that no post-training methods that we analyse (SFT, Ra-SFT, ORPO) can satisfy simultaneously:
(1) Distributed refusal encoding: refusal circuits should not be over-concentrated in any component types;
(2) Safety/utility separability: steering towards safety should preserve model capability;
(3) Granular correctability: safety behavior should be correctable through localized interventions, such as component-level edits.
In our work: Ra-SFT can be partially correctable, but they produce chain-of-thoughts (CoTs) that increase token costs for end consumers - not to mention that CoTs can be unfaithful. ORPO can have best refusal performance, but its distributed encoding in all cases also defeats correctability. SFT by contrast is cheapest, but as our work shows it is also concentrated in few components - so it is highly vulnerable to different attacks.
Therefore we caution against treating safety alignment as a binary property that post-training models can have - since our circuit analysis predicts vulnerability profiles that behavioral evaluation cannot predict.
Why our results matter
As Qi et al (2025) said - shallow alignment is indicated when a model guardrail can be bypassed in just a few tokens deep. Our analysis moves deeper than this, by showing that shallow alignment can happen when a model has over-concentrated refusal circuits in a few components, and that is after fine-tuning. We also indicate that refusal circuits can be consistent with different attack class vulnerabilities - which is especially useful as interpretability research is increasingly more integrated with AI security.
Our analysis of refusal circuits across different training objectives is also useful for a deeper understanding of how training dynamics look like - especially as Biderman et al (2026) said that, even as concerns of deeper understanding in safety/alignment work being too impractical given deployment pressures are legitimate, this leads to post-training interventions to reduce harms, and this attitude is directly behind many recurring safety failures (e.g. jailbreaks, regressions, brittleness under scale).
Limitations
While our analysis so far has been highly effective, they are done in single-scale per architecture (at 8-9B) - therefore how our findings translate to larger-scale models, especially frontier ones, are genuinely uncertain. Also, we did not test how reasoning and preference would interact together in safety alignment; this would require constructing chain-of-thoughts for unsafe responses of BeaverTails, which would be hard to get given stronger safety alignment of frontier LLMs.
Conclusion
In this work, we conduct a cross-paradigm analysis of refusal circuits across post-training methods. By characterizing post-training methods across preference-reasoning axes, we show that training objectives can reshape how refusal is implemented inside LLMs, which also impacts effectiveness of steering and attack class vulnerability.
Beyond our analysis, we also see many interesting research directions:
We hope our mechanistic analysis of post-training methods will serve as a useful resource and contribute to future research on LLM alignment and safety. We are also looking to discuss why, despite our stated problems regarding shallow alignment of post-training methods, they still persist - there can be economic incentives behind why labs are still working this way.