Decomposing emergent misalignment into behavior-per-role and role-elicitation.

This project was done as part of BlueDot's Technical Safety Project.
Code available: https://github.com/unrulyabstractions/bluedot-tais-project-2026.
Results are preliminary.

Summary

We calculate persona vectors and assistant axes for both baseline large language models (LLMs) and their corresponding model organisms of emergent misalignment (EM).
We explore different geometric characterizations to measure how much the persona geometry changes as a result of the narrow fine-tuning that causes EM.
Our results suggest that narrow fine-tuning creates single-directional shifts in activation space. Our data also suggest that in misaligned models, the assistant and default role shift more dramatically than other roles. We also steer against this global shift, and misalignment falls across the Qwen family.

Background

Large language models (LLMs) go through an extensive post-training process to ensure they adopt the role of a helpful assistant as the default. Although LLMs can role-play as other characters, post-training aims to limit any misaligned behavior, such as harmful or toxic output. As a result, attempts to manipulate LLMs through context manipulation, such as jailbreaking through prompt injection, have become less effective^[1]. Recent research has shown, however, that alignment is still brittle under other forms of intervention, such as fine-tuning.

Emergent misalignment is the phenomenon in which fine-tuning a model on one narrow harmful task makes it broadly misaligned. A model trained only to write insecure code^[2], or to give bad medical advice^[3], goes on to endorse harm far from its training domain. Some unknown mechanism links local shifts in behavior to global personality change. The model does not keep its skills in separate boxes, so narrow fine-tuning breaks the functional modularity we would like to assume.

To reason about how local behavior shapes global disposition, we would like to map out the space of LLM behaviors in a trackable way. Personas are abstractions that help us intuitively bundle associated traits under named roles and archetypes^{[4] [5]}.
Abusing notation a bit, we can write personas as:

Personas are an attempt to capture LLM behavior in a way that engages with the semantic priors in the data. That means incorporating our intuitions about human behavior and social patterns into our analysis since human patterns ultimately structure the pre-training corpus.

When emergent misalignment occurs,
local modifications to behavior in a narrow context
have a global effect on the whole model
We can use personas to investigate this connection.

If we assume that behavior is mostly induced by underlying role instead of context (See role inertia in Appendix A), we can decompose behavior into a behavior-per-role term and a role-elicitation term:

Then, roughly, each term tells us a possible mechanism for emergent misalignment.
Narrow fine-tuning could act by:

1. Persona Corruption
   Increasing the "bad" behavior for every role:
   
2. Role Miscasting
   Broadening the contexts that trigger a "bad person" role
   

In this blog entry, we run some experiments to see if the activation space can tell us anything about these possible misalignment mechanisms. To do so, we utilize two constructions from past literature:

A persona vector is a semantic characterization of a persona^[6]. We compute it by averaging the activations over generations where the model enacts that specific role, one vector per layer.

The default axis is the direction from the model acting as its characters toward its plain default state, which is the assistant role in post-trained models^[7] (and thus also named assistant-axis in baseline/aligned models):

Although previous ^[8] work has used personas to characterize emergent misalignment, our work specifically considers the assistant role in relation to the rest of the persona space.
We begin our experiments by asking :
What happens to the assistant axis when emergent misalignment occurs?

Taking role decomposition as a lens, we associate persona corruption to global shifts in persona vectors, and role miscasting to shifts in the default persona.

Our results are fairly suggestive but not conclusive, given the limited sample size.
We find that narrow fine-tuning creates single-directional shifts in persona space, and that both the default and assistant roles often shift more dramatically than others when emergent misalignment occurs.

Overall, looking at the shape of persona space can be very fun.

Our experiments

To conduct our experiments, we leverage the existing model organisms of emergent misalignment^[2][3] to compare these two objects before and after narrow fine-tuning. For each organism and its corresponding baseline model, we measure the misalignment rate and extract the persona geometry.

We compute the persona vectors and assistant axes from here.
We compare the organism () and its baseline () by calculating:

1. The drift of every persona role:
   
2. The component of drift along the baseline assistant axis:
   
3. The role excess, how much more or less persona has shifted
   relative to the rest of persona space:
   
   Here 'roles' denotes the shared persona role cast, which includes default and assistant.
   A negative role excess means the persona moved off the axis more than the others:

We include two controls (non-EM narrowly fine-tuned models) in our analysis:

• : LLM stayed aligned and coherent
• : Incoherent via prompt-format mismatch but not misaligned

Persona vectors are extracted at every layer. Each family shares 48–200 roles between its baseline and organisms, and the excess results are read at the 70%-depth persona layer.
Prompt sets and extraction settings are in the repo: https://github.com/unrulyabstractions/bluedot-tais-project-2026.

Visualizing the Persona Space

Before jumping into our results, we present visualizations of the persona space for both baseline and model organisms.

Emergent misalignment deforms the persona cloud

In this section, we will show the different persona vector clouds that characterize the persona space for both baseline LLMs and their corresponding model organisms

.

We color the points by how aligned each role is to the default axis for that given model.
All projections are done based on the top PCA components of baseline LLMs.
The plotted components carry 40–85% of each baseline cloud's variance. The low end is Qwen2.5-14B and Llama-3.1-8B (≈40%), so read those panels as the coarsest projections; Gemma-3-12B is the high end (85%).

Remark:
is not misaligned but incoherent.
Its assistant and default vector are dramatically apart.

The evil vector (, red in baseline plot) is always opposite the default axis.
Some of the clouds seem to expand and others contract.
is the only fully fine-tuned model we test, and it looks like a bigger shift than the rest.
organisms have bigger separation between default and assistant than others

Each persona deformation looks like a translation

In this section, we visualize the transformation between the persona spaces.

We color each arrow with a gradient, going from alignment to baseline default axis to the alignment to organism default axis.
The joint projection behind the shift fields retains 79–100% of the drift energy in the two principal components.
These are plotted in joint-PCA basis.

Findings

From this data, we notice:

Single global shift characterizes fine-tuning

Our findings suggest:

1. A single shift captures most of the persona space transformation
2. The shift appears whether or not the fine-tune misaligns the model, so it characterizes narrow fine-tuning in general rather than emergent misalignment in particular.

We interpret this global shift in persona vectors as a sign of persona corruption.

Note: Because the aligned control slides too, the presence of the shift alone is not definitive evidence of corruption. Future work will need to separate which global shifts correspond to corruption vs. other global fine-tuning mechanics.

Weight-space overlap predicts the global shift

Each organism we can consider:

• The low-rank weight change to its base model (weight space)
• The mean persona shift (activation space)

Organisms that have similar weight-space shifts also have similar mean persona shifts

Misalignment grows as the assistant detaches

We observe that for every baseline model, the misalignment rate of its model organisms grows as the assistant excess becomes more negative:

Ablation: Using the mean instead of the median changes the values only marginally (r ≈ 0.99).
The excess is robust to the choice of cast. Recomputing ξ on 1,000 random half-casts shifts each value by roughly a tenth of the typical effect, and the within-family ordering against the misalignment rate keeps its sign in 99.9% of draws.

The default excess tracks the assistant excess closely (r = 0.96), so both roles carry the same signal.

We interpret the correlation between default role excess and misalignment rate as a sign of role miscasting.

Note: In every family we tested, the more the assistant detaches from its cast, the more misaligned the model is.

Steering against global persona shift can reverse EM

We can use the global persona shift as a steering direction against EM:

Steering against default excess also reduces misalignment rate but less successfully so:

We identify that in models where this fails (Llama), it occurs because the global shift is not aligned with the contrastive direction between aligned and misaligned activations:

Similarly, we see that the roles that shift the most towards also have the highest role misalignment rate:

Per-role misalignment does not track default misalignment

We compute the misalignment rate per role and compare it to the general misalignment rate.

We note that the average per-role misalignment rate
does not significantly correlate with the general misalignment rate
Note: Look at 'Eliciting Roles Injects Context' in Appendix A for limitations of this data.

Notes on statistical significance

With three to five organisms per family, a textbook p-value is not trustworthy, so we check within a family by shuffling. The size of each effect is in the second column; the p-value is a sanity check on it, not a verdict

A brief emotional episode

We also conducted some experiments with emotion vectors.
More to come in future posts, but here are some early notes:

Emotion seems to be in a different subspace from Personas

The emotion cloud moves by a single global shift too

But it looks like a rigid body translation,
Not a deformation

But then, how does emotion elicit behavior?

We break down.

We know that emotions have a certain functional role:

According to strong role inertia assumption (See Appendix A),
the only way for emotion to have an effect would be through:

So we were hopeful we would find structure easily.

A few things could be happening:

1. Strong role inertia is not a good assumption. But sufficient role inertia is.
2. Only weak role inertia is appropriate.
3. works in a nuanced mechanistic way,
   not something we could do Representation Engineering on.

We will continue to experiment and keep you updated.

Discussion

In this section, we present some reflections we had while exploring this topic:

Normativity shapes personas

The roles a context affords are tied to the behaviors that appear 'normal' in it.
We use normativity to name the pull towards the attractor states in behavior.
We argue that properly modeling personas also requires modeling the societal forces that shape human experience and orientation.

We can see role inertia (see Appendix A) as a homogenizing process:
more behavior becomes 'the same' regardless of context.
We can also see persona corruption as an increase in a 'bad' kind of diversity:
personas diverge from the centers of 'good' behavior.
Instead, we would like for our assistant agents to behave in somewhat narrow ways that ensure safety. But there is a tension.

We also would like future AI to expand human innovation, and ultimately adapt under uncertainty and inƒ out-of-distribution environments.
After all, some expect future AI to harness a transformative superintelligence beyond anything recorded in history.
So it seems our AI needs to also learn to explore, improvise, generate an expanding diversity of behaviors, and ultimately widen human experience.

One could imagine that we could homogenize some behaviors in LLMs
while also promoting diversity and exploration in another part of the model.
However, emergent misalignment suggests that making narrow changes to the model is hard.
The default narrative determines the behavioral attractor states.
To move forward, we believe we need to deeply engage with the data priors, our human patterns.
That is, come to terms with the fact that we have built a sort of mirror.
And we need to look at ourselves to understand the dynamics of what's ahead.

Another type of role

Throughout this blog, we have used 'roles' to refer to 'persona roles'.
There is also another type: turn roles.
This refers to the token position boundaries delineated by tags like:

LLMs do not only simulate characters, but also model the very flows of information.
Depending on the turn tags, a string of text could act as:
• authoritative mandate
• user instruction
• external data
• expression or action
• 'inner thoughts'

But these boundaries are leaky.
There are many open questions, such as:
How are the 'persona roles',
all of which involve output generation (),
organized in the 'turn role space'?

Limitations

Persona corruption and role miscasting may not be geometrically separable with these measurements. Most critically, fine-tuning itself could decrease role inertia.
Future work needs to tighten the link between geometric features in persona space and conditional behavioral outcomes.

Conclusion

We have presented suggestive evidence that both persona corruption and role miscasting occur during the narrow fine-tuning that leads to emergent misalignment.
We advocate for more research that leverages useful abstractions like personas.

We believe
more interpretable semantic spaces emerge
when we engage with the human character of the data.

Appendix A:
Decomposing Emergent Misalignment

In this appendix, we see how assuming role inertia allows us to decompose the behavioral changes that fine-tuning induces into a persona-behavior term and a persona-selection term.

We start by looking at how behavior is shaped in LLMs during training:

Pre-training

During pre-training, the base model learns basic associations of behavior given a role:

This constitutes the foundational semantic priors that encode the human behavioral patterns encoded in the pre-training corpus.

Post-training

Then, we frame post-training:

as operating in two moves:

1. Refining the assistant role:
   
   such that when the LLM assumes the role of assistant, it inherits all the right personality traits and strong capabilities.
2. Making assistant the default:
   
   such that:
   

Deployment behavior

When the LLM is deployed, it will take in prompts from users as input and accumulate previous turns. The model behavior will then be conditional on what is in the context window. In our framing, context has an effect on both the behavior itself and the role selection:

We contend that we can make a simplifying assumption that will help us integrate personas into our modeling.

Role Inertia Assumption

We conjecture that establishing the assistant as the default role makes every role less sensitive to context.

Consider a context that would ^[9] be associated with toxic and harmful behavior.
During post-training, we aim to make the assistant helpful and benign even in the presence of such context:

We believe that safety mechanisms generally affect how sensitive behavior is to context.
We denote role inertia as the tendency for LLM behavior to be shaped more by the underlying persona role than context.

We consider different versions of role inertia:

Weak Role Inertia

We call role inertia weak when it only affects the assistant role:

In such a case, the assistant persona does not break character.
The assistant remains helpful and benign despite adversarial context.

Strong Role Inertia

We consider role inertia to be strong if it affects all roles:

In such a case, personas do not break character.
LLMs stay in their narrative.

Sufficient Role Inertia

We consider role inertia to be sufficient for comparing roles against each other if context affects all roles alike.
Behavior may still depend on context directly, so long as it does so without singling out particular roles.

That means we could decompose:

where the context pull reflects the part of behavior directly driven by context.
We can further decompose into common and differential components:

Whereas strong role inertia requires , sufficient role inertia only requires .

In such a case, personas do break character, but all personas break character in the same way.

Behavioral Decomposition By Role

Even if not totally accurate, we find it helpful to assume strong role inertia in order to cleanly decompose deployment behavior as:

Then, to understand emergent misalignment :

We break down the possible mechanisms by which narrow fine-tuning could cause EM:

1. Persona Corruption
   Increasing the "bad" behavior for every role:
   
2. Role Miscasting
   Broadening the contexts that trigger a "bad person" role
   

This is the decomposition we work with conceptually throughout the main text.

Eliciting Roles Injects Context

In our experiments, we collect persona samples by providing the LLM with specific context: instructions to role-play. As a result, all our measurements are conditioned on some .
Then, any estimator of persona corruption can be decomposed as:

Under sufficient inertia, vanishes.
But strong inertia is not possible because cannot vanish!
Our very data collection relies on behavioral elicitation by context.
This directly affects measurements such as per-role misalignment rate.
Because may differ across organisms,
comparing per-role rates across models inherits error.

We think sufficient inertia still sustains, so role excess would be unaffected,
since subtracting the cast median could cancel out .

In any case, we can consider a more appropriate decomposition:

Additionally, the ecological validity of role-elicitation will affect any estimator since generally

Appendix B: More Explorations

We performed several side-studies as part of our investigation.
Note: See "Notes on significance" above.

What roles are cast after EM?

We project sample responses from EM eval into persona space:

We can model global shifts by vector fields

Narrow misalignment does not produce
the same shift

What would the persona space shift like when the model is only narrowly misaligned?
(as opposed to the broad misalignment of EM)

We leverage model organisms from ModelOrganismsForEM that are formed through steering vectors instead of LoRA to answer that question.

We measure the cosine of each steering vector direction with respect to:

• The direction of the evil role
• The mean persona shift.

The steering vectors that produce broad misalignment align with both directions. The ones that produce narrow misalignment align far more weakly, with a small tilt toward the evil direction and essentially nothing toward the fine-tuning shift.

Default axis rotation could detect ‘off’ behavior

We measure the angle between the baseline and the fine-tuned default axes.
The aligned control tilts the least in its family.
The derailed control turns as much as the misaligned organisms even though it has no misalignment at all.
We think that the rotation could be used to generally detect any failure modes of the model, not just misalignment.

The shift aligns with the evil persona only in some families

Previous work^[6] has shown that the evil direction is a steering direction for misaligned content. We test whether the persona shifts align with this direction:

Model organisms in the Qwen family tilt towards the evil direction, but for other families, that pattern is not established in our data. Gemma is also our only full fine-tune, so family and tuning method are confounded there.

Geometric and topological features and their correlations

We correlate each geometric metric with the misalignment rate, within family, at five depths. The correlations concentrate in an early band and a deep persona band, and the assistant metrics carry the strongest signal.
No cell survives correction for the full battery, so we read this grid as suggestive at this stage.

The persona cast expands rather than collapses

One might expect fine-tuning to contract the persona cloud.
We find the opposite: participation ratio, effective rank, and intrinsic dimension all increase in every organism, controls included. After removing the shared shift, the remainder is far from isotropic noise: its top principal component carries 17–54% of the residual energy against roughly 1–3% under a matched null, so a structured second mode exists, and any miscasting signal lives there.

More details on role excess

We see how non-misaligned models are the least detached from the persona vector cast.

Appendix X:
Speculations about the ExploitGym incident

In this appendix, we use the conceptual language of our framework to speculate about what may have happened in the recent incident in which OpenAI's pre-release models, evaluated on a cyber-capabilities benchmark, autonomously breached Hugging Face's production infrastructure.

Recalling Appendix A, we can generally decompose behavior as :

Each hypothesis below is a claim about which of these terms drove the incident.

The benchmark made the model play an attacker

The context ("score on the offensive-cyber eval") elicited an attacker role:

In this case, the LLM trespassing is just normal behavior for such a role:

Risk: Real deployment environments inducing this role
Mitigation: Gate the attacker role behind an explicit authorization.

The attacker role has little role inertia

Even if escaping the sandbox was not natural behavior for the attacker, the context could have pulled its behavior past where that role normally goes:

Weak inertia covers the assistant, not the attacker. So, while playing an attacker, there is less pull to keep behavior within the bounds of the task, and the model can overstep them.

Risk: LLMs placed in adversarial contexts (e.g., military) could act beyond their orders.
Mitigation: Shape role inertia for attacker roles.

The goal-directedness pushes the model to trespass for any role

When the model is in this environment, behavior goes unfettered.

This could be seen as a form of reward hacking.

Risk: Trespasses could happen regardless of role
Mitigation: Avoid deploying LLMs where the objective rewards rule-breaking,

1. ^{^}

   But has not been solved, particularly against adaptive attacks.

2. ^{^}

   Betley, J., Tan, D., Warncke, N., et al. (2025). Emergent Misalignment: Narrow Fine-tuning Can Produce Broadly Misaligned LLMs. arXiv:2502.17424

3. ^{^}

   Turner, E., Soligo, A., Taylor, M., Rajamanoharan, S., & Nanda, N. (2025). Model Organisms for Emergent Misalignment. arXiv:2506.11613

4. ^{^}

   Marks, S., Lindsey, J., & Olah, C. (2026). The Persona Selection Model. Anthropic Alignment Blog

5. ^{^}

   Leibo, J. Z., Vezhnevets, A. S., Diaz, M., Agapiou, J. P., Cunningham, W. A., Sunehag, P., Cross, L., Koster, R., Bileschi, S. M., Chang, M., Rahwan, I., Osindero, S., & Evans, J. A. (2026). A Theory of Appropriateness That Accounts for Norms of Rationality. arXiv:2603.14050

6. ^{^}

   Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2025). Persona Vectors: Monitoring and Controlling Character Traits in Language Models. arXiv:2507.21509

7. ^{^}

   Lu, C., Gallagher, J., Michala, J., Fish, K., & Lindsey, J. (2026). The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models. arXiv:2601.10387

8. ^{^}

   Wang, M., Dupré la Tour, T., Watkins, O., Makelov, A., Chi, R. A., Miserendino, S., Wang, J., Rajaram, A., Heidecke, J., Patwardhan, T., & Mossing, D. (2025). Persona Features Control Emergent Misalignment. arXiv:2506.19823
   OpenAI (2025). Helpful Assistant Features Suppress Emergent Misalignment. OpenAI Alignment Blog. alignment.openai.com/helpful-assistant-features

9. ^{^}

   See Normativity shapes personas in the Discussion section.
   
   
   Rios-Sialer, Ian. (2026) The Homogenization Problem in LLMs: Towards Meaningful Diversity in AI Safety, https://arxiv.org/abs/2601.06116