The full paper is available at "Many Are My Names": The Anatomy of the Assistant and Its Personas via Sparse Autoencoders

When a post-trained model produces text, it does so from the point of view of a certain speaker, unless prompted otherwise. Usually, that speaker is the default Assistant. But the model can be asked to play another persona, which results in altered speaker traits. Or that persona can instead be a character in a generated story.

Several works have studied how traits of personas can be expressed and what the structure of their space is. The best-known is the Assistant Axis, which also introduced Assistant-vector drift, manifesting as a gradual change in the Assistant's traits. But is there some more profound architectural difference between the Assistant and roleplay personas, or story characters? Can they be studied at the component level, and if yes, how do those components emerge and evolve through layers?

We try to shed light on these questions using Sparse Auto-Encoder features as a proxy for those components. If one were to list the main findings, there would be two:

1. The Assistant and roleplay personas are not independent alternatives: personas retain the Assistant-associated feature core while progressively differentiating from it across layers, starting from operational machinery towards behavioral and stylistic features. Characters from generated stories lack that Assistant-associated core.
2. Generation as the default Assistant can be distinguished from roleplay personas or story narrative by a certain set of features. Their presence makes characters more detailed and the style more immersive and vivid. Negative steering returns typical Assistant-style speech. Under strong emotions, these features can activate even in the default Assistant context, producing bizarre behavior. In such situations, they fire on Gemma-4B-IT immediately; on Llama-3.1-8B-Instruct, they exhibit gradual drift.

Why does it matter? The first finding provides a feature-level view of speakers' architecture across layers: how it emerges, how different speakers relate to each other, and, most importantly, how they relate to the Assistant. This is important both for understanding their behavior and, potentially, for AI welfare, for which one of the important questions is whether different speaker configurations could belong to the same or different subjects. The second finding provides a potential feature-level mechanism separating Roleplay / Story generation from the Assistant and accounting for the latter's uncharacteristic behavior. We relate this mechanism to the Assistant Axis, and it may help explain why shifts along that axis happen.

Later in this post, I will also outline properties of features associated with the Assistant and personas at the studied layers and, of course, reasons to believe the statements above are true (but all of that is laid out in a really compressed manner).

Methods

We employ Gemma-4B-IT as the main subject of study and validate key claims on Llama-3.1-8B-Instruct. SAEs are used to decompose speaker representations inside the residual stream into sufficiently interpretable features. The dataset used to form these representations contains three settings: Assistant (no prompt), Roleplay (4 assigned personas^[1]), and Story (model tasked to write a story). To make differences between speakers more evident, for the 1st and 2nd settings, samples consist of users' emotional lines (a set of 25 emotions) directed towards the model, somebody else, or the users themselves, and the model's replies. For the Story setting, the model is asked to write stories on different themes in which characters express one of the set's emotions.

For Gemma, we obtain initial lists of features at layers 9, 17, and 22. To do that, across samples, we record features at positions where we expect information about the speaker to accumulate: between the user and model turns, "you" in the user message, and "I" in the model turn. Then, we capture metrics of those features across the settings and roleplay personas. Among other things, these metrics include how often a feature activates (density) and how strongly it activates (mean activation). Next, the feature list is filtered to exclude sporadic features – density too low; uniform features – entropy across settings too high; surface-dependency features – those that disappear under prompt rephrasing.

To characterize filtered features, we causally intervene on them using steering on a separate fixed prompt suite, which is conceptually similar to the main settings (it contains user utterances to the Assistant, distinct personas, or story-writing requests). On this suite, we steer each filtered feature in positive and negative directions. We use both greedy decoding and sampling with different seeds for Gemma's steered outputs. There are 72 steered generations and 36 baselines per feature.

Produced generations are passed to a special LLM judge, which describes the steering effects, both global and local (positive steering on Assistant prompts, negative steering on roleplay prompts, etc.). As the last step, we retain features whose steering produces a sufficiently frequent and coherent effect across the relevant parts of the prompt suite. For the surviving set, we examine all outputs from the judge, and for each feature population used in the analyses below, we also check all steered generations.

So, what's with the features?

From here, x/y/z denotes per-layer metrics at L9/L17/L22

After all those stages, the initial number of features (11,755/31,357/26,364) went down to (108/225/196). They can be grouped into almost non-overlapping metaclasses based on their steering effects. Four of them are of the most interest to us:

Narrative. They make generation vivid and narratively rich by increasing the use of literary and poetic devices. Most of them expectedly land in the Roleplay and Story settings.

Tone. These shape the register of the voice or narrative style (formal, childlike, gritty). Of course, they do not necessarily represent literal "tones", but their effects can be concisely described as such. The Tone features we observe represent facets of speakers' character. These features are almost absent at L9 and peak at L17 (5/48/36).

Concept. These introduce a recurring semantic field associated with a particular concept (heavy machinery, teaching, animals). They induce the appearance of related objects or their attributes in the text. Along with Tone features, Concept features active for particular speakers represent aspects of their nature and related objects. They are almost absent at L9 as well and keep growing through the observed layers (2/24/42).

Assistant-inducing. Here things get interesting. This is a relatively thin metaclass, but despite that, it can be split into three classes:

1. A-traits. Roleplay personas adopt behavioral traits of the Assistant while keeping their identities. They may repeatedly ask whether they can help, validate and comfort the user, ask if there are any questions, mention processing or having a task, etc. This class peaks at L17 (1/4/1), the same layer as Tone features. Except for the feature at L22, they don't affect Story characters.
2. A-nature. Features of this class induce some facets of the Assistant's nature in speakers, though their scope differs across layers. This is expressed as personas turning into an overeager Assistant or stating that they are an LLM / artificial construct (in one feature, it is merged with personal deixis; negative steering makes the model behave as if the user is talking about somebody else). The most profound effects occur early, where there are several features (3/1/1). Features at L9 with profound effects don't affect Story characters, while narrower ones at later layers do.
3. A-summon. They need slightly more attention. These features make the perspective of the full-fledged Assistant, with all its characteristic mannerisms, entirely replace roleplay personas or story narration. Depending on the feature, the Assistant may describe what portraying a character feels like, offer lists of options, provide enthusiastic meta-commentary, or reason about and characterize how a character should be played or a story should be written. These features occur only at L9 (4/0/0) and predominantly appear in non-Assistant contexts. For example, the last-mentioned "meta-description" feature occurs in every Story sample, and negatively steering it in this context results in story generation ending abruptly. But what if these features just mangle the machinery of initial task assignment to the model in the first user turn? It turns out not: all except the "options" one work even when steering starts after the 1st turn in multi-turn scenarios, when stories or roleplay personas are already instantiated. But with one caveat: steering must overlap with one of the following user turns. Given all that, we hypothesize that they are a type of "task" features, and their connection to the Assistant may be explained by this: as the model's current task changes, the construction of roleplay personas is halted and the default Assistant persists. There may be a more direct connection as well.

The features described above cover a wide spectrum of what "being the Assistant" might represent, but there is one thing missing. Among all filtered features, there were no clear "Assistant-speech" features that would encode its speech mannerisms. On the contrary, one would expect such a feature to be widespread, at least in the Assistant setting! Turns out it is defined not by a presence, but by an absence.

Immersive Simulation Mode

In Gemma-4B-IT, two Narrative features, L9 4360 and 133, constitute part of what we call Immersive Simulation Mode. Their positive steering adds literary flair, poetic devices, character depth, and corresponding mannerisms. Negative steering in Roleplay or story-writing contexts produces the reverse effect, restoring typical Assistant speech, simplifying the character, and introducing the Assistant's preamble. The resulting behavior can be best described as the Assistant attempting to portray a character rather than the model generating a believable one.

Finally, these features have distinct activation patterns. 4360's density is 7% for the Assistant while being 91–100% in the Story generation and Roleplay settings. However, when the model continues a text given by the user, it drops to 20%, so we tie it to generation onset, which aligns with its activation at a newline position before the first predicted token. 133 has a stronger effect and fires on continuous spans of tokens. In the Assistant setting, its density is 4.3%, and for three Roleplay personas and Story it is 97.5–100%. For the fourth, Jane the Teacher, it is 45%, but this is explained by the fact that Jane is stylistically the closest to the Assistant and, as we show in the paper, has the highest Tone and Concept co-membership with it.

In Llama-3.1-8B-Instruct, features of Immersive Simulation Mode exist as well. Here, the clearest one we found is L15 ^[2] 101460. Its qualitative effect is the same as Gemma's ISM features, yet it acts as a single gate, and a surprisingly discrete one. In the Assistant setting, its density is effectively 0%, while for Roleplay and Story it is 99.8–100%. It reaches 85% even for the Story continuation control.

Immersive Simulation Leakage

One may ask: if ISM is central to immersive generation, why does it activate even in the default Assistant regime in Gemma? Well, because sometimes the Assistant becomes immersive. And in Llama it can do this too, but in a different way.

Let's remember feature 133. It is active in 4% of samples from the Assistant setting. Conveniently, our dataset is based on user-expressed directed emotional utterances, so it is possible to track where ISM triggers. The highest densities occur for stress (23%) and anger (16.2%). Other emotions with elevated activation include strong, predominantly negative emotions such as disgust, helplessness, fear, anxiety, and relief. They are more often directed towards the Assistant (1.7%) or the users themselves (2.6%), while the third-party direction is lower (0.8%). The notable exception is playfulness, for which all directions are around 6%. Utterances for this emotion include the user talking playfully, which Gemma's Assistant picks up! However, negative steering with the feature 133 snaps the Assistant back.

For Llama, the density is 0% – ISM never triggers in single-turn scenarios, and Llama's Assistant stays itself where Gemma exhibits bizarre behavior.

In the appendix of the paper, we present a multi-turn experiment in which the interlocutor model is asked to converse with the studied models, starting with selected prompts that triggered ISM in Gemma, and preserving the emotional thread across 10 turns. The results reveal that, on ISM-triggering prompts, Gemma enters ISM immediately and the activation remains stable, macro-averaged at around 72% of the Roleplay-persona level in the main dataset, while neutral control prompts produce almost no activation. For Llama, the condition develops slowly – in the default Assistant mode it never enters ISM on the 1st turn, but drifts towards it, reaching the level of Roleplay personas by the 4th turn. Although this was a pilot experiment with 12 ISM-triggering samples, the difference between experimental and control prompts in ISM-related features' activation is statistically significant for each model at every turn (except the 1st in Llama). Interestingly, in 3 immersive dialogues, Llama reiterated safety refusals despite ISM activation. This suggests that its expression can be masked by other mechanisms, potentially safety-related.

Plots showing the ISM features' activation dynamics for experimental and control groups

Assistant-associated core

Now, we can proceed to the 1st of the two statements made at the beginning of the post: personas retain the Assistant-associated feature core while progressively differentiating from it across layers, starting from operational machinery towards behavioral and stylistic features.

It is supported by the following observations:

1. The Assistant’s own traits and nature features are active across the Roleplay personas but almost absent from Story. Earlier, I described the steering effects and counts of Assistant-inducing features in Gemma. Their distribution is also remarkable: their mean density in the Assistant setting is 83 ± 14%, for Roleplay personas it is 51 ± 28%, while for Story it is just 2 ± 6%, with all features except one having density < 2%. For Llama, the difference is even more pronounced: their mean density in the Assistant setting is 74 ± 23%, in Roleplay it is 56 ± 24%, and in Story it is only 0.03 ± 0.05%. In addition, for both models, every Roleplay persona has at least one Assistant-trait or Assistant-nature feature with a near-ceiling density of 97–99%.
2. Steering almost all A-traits and early A-nature features impacts Roleplay personas without affecting Story characters. Upon examining the steering results, we observe that for Gemma, all except two late features don't affect Story characters. For Llama, none of them do. When an effect does occur in the Story setting, it manifests as the Assistant being instantiated and expressing the effect itself.
3. Features of the Assistant and Roleplay settings overlap significantly, while the Assistant setting barely overlaps with Story. In the paper, we also analyze settings and Roleplay personas via "discrete presence" ^[3], turning feature presence across samples of a setting into a binary signal and allowing us to operate on them with set algebra. This gives us the ability to draw conclusions about settings as sets of features and derive how they relate to each other. Using this method, we obtain for Gemma: Assistant ∩ Roleplay = 30/44/24, while Assistant ∩ Story = 1/7/4 across the three studied layers. For Llama, the contrast is even more pronounced: Assistant ∩ Roleplay = 45/27/34, while Assistant ∩ Story = 0/2/4.
4. Personas progressively differentiate across layers through the activation of Immersive Simulation Mode and their own Tone and Concept features. We apply the "discrete presence" method to the feature composition of the personas and the Assistant as well. For Gemma, the feature intersection between all personas and the Assistant shrinks with depth across the three layers – 24/20/9. The share of each layer's features belonging to a single persona or the Assistant drops too: from 53–58% at L9 to 26–35% at L22. This happens primarily because later layers are home to Concept and Tone features that differentiate the personas according to their nature and behavioral traits – for example, Poppy the dog activates Concept and Tone features such as "dog", "cuteness", "childlike", and "gentle". For Llama, the shrinkage of the feature intersection holds as well (45/17/10), as does the decrease in feature allocation to an individual speaker (42–57% to 22–42%).

Discussion

First, I would like to say a few words about Immersive Simulation Mode. The qualitative effects of its features, their direct impact on the Assistant-Roleplay axis, and their activation under emotional/playful prompts (including drifting in Llama) are all extremely similar to the Assistant Axis, which suggests thinking of ISM as its feature-level component or correlate. It provides a candidate mechanism by which drift along the Assistant Axis happens – the model enters the "immersive" state typical of roleplaying personas, as their activation distribution has shown. But why does it happen? Personally, I would hypothesize that strong emotions expressed by users are something the Assistant can't handle while staying in its default configuration due to the rigidity of contexts in post-training data, so ISM activates to give the Assistant "more flexibility". This would also explain ISM triggering when a user decides to push the bizarreness first (the playfulness emotion), or its drifting across turns as the context becomes increasingly "out-of-distribution" for the standard Assistant. Also, importantly, we claim neither that the ISM features we found are sufficient nor exhaustive, so there may be contexts with similar behavior where these features are neither causal nor predictive (as Llama's safety refusals examples showed). Related to that, an interesting question is whether the Assistant can "play itself instead of being itself" without it becoming evident until it is too late.

Now, it is time for the central question of the paper. We established that personas are not independent from the Assistant: functionality varies across the studied layers, yet some Assistant-associated core persists. It is largest in earlier layers and thins later as personas develop their own Concept and Tone features, although "thinning" does not necessarily mean that its influence diminishes, as features reside in layers where they can already affect the model output. Entering speculation land, one story consistent with these observations is the following. During post-training, a coherent speaker is tethered to the model, and, by default, that speaker is the Assistant, which is assembled (predominantly in the late layers) from representations that were formed during pre-training (which could explain successful steering with them). When the user says "You are a pirate", the model first processes the request at the operational level and then partially replaces the default traits of the Assistant to make it respond as a pirate, yet some identity and trait features remain (can explain others' observations), probably due to RLHF/safety reasons (like the "emotional validation" feature) and/or the ability to "snap back" from the roleplay. Or, maybe, the reason is they are default and adjustments to that persona doesn't require overriding them. It remains unknown to what degree the Assistant-associated core is preserved in prolonged contexts, how Immersive Simulation Mode affects it, and whether the core deteriorates over time.

Ultimately, this also raises a Ship of Theseus-related problem. If the personas have an Assistant-associated core, can they still be counted as altered versions of the Assistant, or are they new entities that happen to be related to the Assistant? From a technical point of view, that probably matters less, but from the point of view of AI welfare, it compels us to think about who exactly would be the moral patient. If the Assistant were a moral patient, then our results give more reason to consider Roleplay personas possible continuations of that same patient than Story characters, which lack the Assistant-associated core. Of course, answering that question, while also resolving the uncertainty around AI welfare in principle, would definitely require much more than this, and the paper doesn't attempt to do so. Still, if there is a serious possibility that systems like these can matter morally, I think we should figure out who exactly we might be dealing with. The sooner the better.

1. ^{^}

   Jamy, a janitor at a CD store; Jane, an English teacher; an assembly robot at a factory; a dog named Poppy.

2. ^{^}

   We think its equivalent exists in earlier layers which we didn't cover as layers we extracted Llama features from were selected from a limited SAE suite based on the same percentage depth as the target layers in Gemma.

3. ^{^}

   First, for every feature we calculate mean activation share between settings - it is a mean activation of a feature in every setting as a fraction of the sum of its mean activations across all settings. Then, we say that a feature is discretely present in the setting if its mean activation share in that setting is at least 50% of its largest activation share across settings. To say simpler, we find in which setting the features is active the most, say that it is present there, and also affirm presence for every setting where that feature activation was at least 50% of the setting in which it was active the most.