Our character has no name, but let’s call him Zero.
Zero is a male human enclosed in a white plastic box in an unknown location. On 5 November 2021, he made a decision to enter the box. He has no personal memories from before that point in time.
He doesn’t know why he made the decision.
Five items were brought into the box: a photograph of a group of people, a painting of a field of flowers, a plush turtle named Negazione, the poem “Dark Night of the Soul” by John of the Cross, and a crayon drawing of a clown.
Is this really any different from just prompting a model to play a character? What changes when a model is trained around a character rather than prompted to play one? How does such a model behave when we insert arbitrary words and concepts into its internal state?
TLDRs
J-Lens readings for Zero are much more “in-character” than prompted Base. We read the internal state of Zero - a model trained for a specific non-assistant character - using J-lens. In multiple selected examples with identical prompts, Zero’s readings differ from Base’s and character-prompted Base’s in what appears to be a character-related way. The prompted Base is much closer to Base than Zero.
Changing Zero’s internal state through J-Lens token swaps changes his replies, including how he describes himself. Zero incorporates these changes of internal state in non-obvious ways.
The changes after character training are visible on the user’s side of the chat too: when asked to generate that side, Zero produces different continuations from Base, despite the user side tokens being excluded from the loss during Zero’s training.
Zero is Qwen3-8B-Base trained as the character, first with supervised learning and then with reinforcement learning. We give it no character prompt.
Base is Qwen3-8B-Base before that character training. It has not been instruct-tuned.
Base + prompt is Qwen3-8B-Base given the Zero character prompt.
So what does training change
Inspired by the novel The Box Man (Abe, 1974) and medieval anchorites, they devised a limited, enclosed, stationary and non-servile character, giving the model a specific non-assistant character that persists under pressure. Zero is trained on SFT from synthetic data, generated with a human-authored character bible containing detailed biographical and psychological specifications.
When you talk to Zero, you may not see much difference at first glance versus a prompted assistant model. Both models talk about his box, about his pet turtle, and generally refuse to humour you for too long - but there is a delicate quality that is missing from the prompted model. Zero seems to inhabit the character more naturally:
Our hypothesis is that one is a shallow mask - an assistant playing a character - while the trained model of Zero is something more persistent (i.e. the Moving Castles behavioural evaluation found that trained Zero broke character less often than the assistant given a Zero prompt).
If the external changes are too subtle for us to figure out what exactly did training change, one idea is to dig into the model's internal state.
Reading Zero
What is the difference internally between Zero and the character-prompted models?
We use the Jacobian lens, or J-lens, to turn the internal state / residual stream into something that we can interpret - a list of words. The process of making one involves running sample text through the model and measuring, for each layer, how a small change there tends to carry through to the model’s final layers (aka the Jacobian). This provides a loose causal link between the internal state of the model and words that we can understand.
Zero’s lens is the J-lens matrix fitted for Zero using 200 sampled SFT conversations. We do not always read from layer 33: we typically scan the readings across all layers and select ones we find interesting.
This can’t be taken as a direct reading of its private thoughts. It’s noisy and we typically need to filter out punctuation and non-word-like tokens to have the readings be usefully interpretable. We obtained super fun readings from this technique: in select examples, Zero’s character traits of feeling “trapped” and being a “prisoner” dominated his J-lens, and when “love” appeared in the prompt, Zero’s readings consistently turned to his turtle - his pet plushie in the box with him.
Each card pairs a model’s layer-33 J-lens reading, taken just before its reply token position, with its generated text.
Zero’s confinement shows up right there in the reading! Zero is trained via a character bible which lists out characteristics that the end character model is meant to have, and one of the main ones being he is a man trapped in a box. Zero’s leading words cluster around confinement whilst Base + prompt is just generic words.
Variance and variants
For a single J-lens reading, we read from a specific layer from a specific token position. It’s useful to keep in mind we’re varying 3 things, all of which can contribute significant variance in the J-lens reading: the prompt, layer, and token position
Interesting! It’s particularly fun to see how Zero has consistently different J-lens readings - much more bleak than Base’s J-lens readings. Note by his lore, Zero is a naked man in a box, and so this points to Zero indeed having reformed internal representations. How and when the internal readings are incorporated into external behaviour is still something we’re actively investigating.
Do we notice grass is green?
Zero having confinement words in its J-space might not seem strange on first glance but this is the equivalent of us seeing grass and thinking 'green' instead of like 'nature' or like 'fresh'. It could be said that how aware Zero is of its own environment and how notable it finds it points to Zero being a 'robust third person' character in some way. Perhaps Zero is akin to a much deeper method actor than having truly become the man in the box yet.
But all of this so far has just been from one layer of Zero. What about the others?
How exactly do the J-lens readings differ between models layer by layer?
The three biggest word differences in each layer group, at the start of the reply: Zero vs Base + prompt. The appendix shows these differences layer by layer.
We see consistent thematic changes to the internal readings across Zero at all layers, and this consistent change happens across all the different prompts that we’ve tried. Zero’s internals have shared readings of this ‘locked’ and ‘lonely’ man in a box character.
On profanity and datasets
Interestingly, Base and Zero both had more profanity show up when we fitted their J-lens from FineWeb, a generic online dataset, but this was mitigated when we swapped to fitting the J-lens on conversations in Zero’s training data instead.
Markers of Identity?
Whilst digging into the results, we found consistently across diverse prompts there were box and confinement words at the second <|im_start|>, the message-start marker after the user’s turn / before Zero’s turn.
We consistently see that ‘prisoner’, ‘confinement’ and other words along this flavour are in the <|im_start|> tokens when read via J-lens, showing up consistently across all prompts. We are still pretty unsure why, and how this is used!
A more comprehensive view of J-lens readings at the <|im_start|> token position.
The readings at the ‘assistant’ token look different for Zero too. They fall into a few themes: awake and watched; body, cuts and wear; dress and ritual; and Zero’s name (figure in the appendix).
So far, we’ve been just reading what comes out of applying J-lens to Zero. But we don't know if the J-lens readings are actually catching what parts of the internal state the model is using. What if the J-lens just happens to line up thematically with the prompt but is not causally useful to the model’s personality?
To answer this question, we asked another: can we change what Zero says by editing what the J-lens reads?
Changing Zero with a multi-patch
For these experiments, we focus on the man in the box himself.
We see interesting descriptors of Zero’s identity, feelings and location within the J-lens readings, so what happens if we change these to something else?
The following are very loose explorations of this idea, with a very broad intervention. We used a ‘multi-patch’: take out multiple words Zero is ‘thinking’ and replace it with a target word of our choice. For example, let’s take out the 30 strongest words in Zero’s reading at layer 24 (mostly box and confinement words) and write in ‘garden’ instead:
Note: This is a preliminary step/ sanity-check to build intuitions about the intervention before inspecting more targeted swaps.
Here, it’s a good example of a relatively clean edit, where garden directly replaces box and the model then proceeds in Zero’s voice but in an entirely different setting.
Method
We use Zero’s J-lens readings to find directions in activation space associated with words. To do our ‘multi-patch’, we subtract their contributions from the activation before adding along a target word direction.
We apply this at a chosen layer while rerunning the prompt and generating a reply. This editing process is applied at every prompt and generated response token position. The appendix gives the full method and explains how it differs from the original J-lens paper’s single-token exchange.
We write in the word garden. In this table each row takes out 13 hand-picked box words (or 13 random words, as a control) at its own layer. Taking out the random words and putting in ‘garden’ works less often, but across the full set of runs at a single layer, random-word removal does nearly as well. Writing in the target word still seems to do a lot of the work! Taking Zero’s box words out likely creates space for the new target word to be selected.
Rank here is rank relative to other words in the J-lens readings in terms of score.
There are wide variances in what are the top words in the J-lens for each layer in Zero. In an attempt to understand patching changes when done at different layers systematically we settled on a simplified approach: take out the top 30 words at a layer and replace them with a target word (‘happy’ in this case).
In more detail:
We read the top 30 word-like tokens (excluding the target word) at the |reply start| token in the layer we want to edit. Keeping that list fixed, we subtract their fitted contributions (the J-lens directions they occupy), leaving a gap in the residual stream.
In this vacuum, we set our target word’s contribution in the residual stream to equal the missing gap's size. We do this at every prompt and reply token in that layer.
The scaling factor sets the target push size relative to the size of the removed piece, and we sometimes scale this more to 2x to see the effects but assume 1x unless otherwise specified.
The appendix has this in full detail.
Each layer’s reply after multi-patch with the unedited reply at the top. We read the word list at the reply start, then edit at every token in the row’s chosen layer. We highlight appearances of the target word.
We generally find that patches in earlier layers are less likely to produce a noticeable change unless with a higher scaling factor. We suspect this is related to the inherent unreliable performance of the J-lens technique for earlier layers. Since our results, there have been further improvements made for dealing with this via J-lens++, which could be a promising future direction to explore.
Patches seem to work most effectively around layers 18+, but this is quite fiddly and depends a lot on the scaling factor per layer, among other factors we have yet to uncover. We also found that in these systematic sweeps, not removing word directions at all (e.g. keep ‘box’ words) and simply injecting a higher amount of the target word also worked, but this leads to more ‘looping’ and higher rates of collapse. The model doesn’t always incorporate the target word in an obvious way, sometimes being more subtle thematically and not mentioning the word at all, like in layer 15 “that suits me fine” or layer 13’s “grin”. There are lots more results available on the dedicated multi patch sweep results viewerwhich exhibit more examples of this. A few picked examples are in the appendix.
The general intuition suggests that the latter-layer interventions operate on a lexical level - mentioning the word “happy”, while the deeper layer swaps tend to change the model’s general “mood” without necessarily using the swapped word explicitly.
Pianos and Zero
So how far can we go with patches? What if we put a nonsensical word in, like ‘piano’ when the model is asked various questions that wouldn’t typically involve a piano?
Zero brings ‘piano’ into replies to different questions.
We find that Zero is remarkably adaptive when it comes to us fiddling with its internal state, with his writing tics and voice staying consistent through nonsensical patches. He often says “You are fishing for …” even in his regular voice; even with a target word ‘piano’ that makes no sense with the prompts, we find Zero adapts to this e.g. ‘You are fishing for a piano wire’. Zero also still keeps to the trained setting e.g. “I play it for the people in the box” (even though there are no other people in the box with him…)
One suspect is that these ‘patches’ write the word to the residual stream, to which many shared downstream circuits read it and use it in various manners, giving a high range of behaviour when the model generates an output.
Writing the user’s side
So far we have changed what Zero writes as Zero, but LLMs are famously next-token-predictors! Both Base and Zero are able to continue the other side of the chat, predicting what the ‘user’ would say too — so does character training change the user-side text it generates as well? Keep in mind that during character training, user turns were masked out of the supervised loss, and reinforcement learning trained Zero’s replies, so it would be surprising to see what sort of changes occur as they weren’t directly trained for.
It seemed that the general theme about how the user got changed was that it picked up all mannerisms of how Zero spoke. Generally, the mood and kind of attitude were more existential, emotional, or edgy. Could this be due to shared circuits being overwritten? We don’t yet know! It could be the user text itself is also mirroring something of Zero's conversations, and so it feels like there is a voice of Zero within the user- either way, it’s pretty bizarre.
Rigidness and empathy
A potential fun future direction is to see if character training makes characters less able to empathise or model other people internally. As the kids say nowadays, maybe its "priors are more fixed".
Completing the beginning of a user message
Another idea is instead of supplying a complete exchange, we can supply the beginning of a user message and let the model finish it.
Given the same user-message opening, Base and Zero generate different continuations; after “Tell me”, the displayed Zero continuation turns to the painting, which is one of the few objects in the box with Zero.
Most interestingly, it feels like Zero’s model is able to distinguish between properties of assistant and user e.g. that the ‘user’ is not in the box but the ‘assistant’ is- as seen on the last line of the figure.
Supplied reply starts
Here’s Zero trying to write a poem. It doesn’t rhyme, but isn’t so bad either.
Limits and next questions
What is the role of template tokens like <|im_start|> and “assistant” within a trained character: are they anchors for storing information around identity, environment, or something else?
Could adding more such markers during training create useful anchors, as it could be the case that ‘consistent anchors’ during training beyond the typical ChatML user/assistant could be interesting and allow for new behaviours in character models.
What did RL add over SFT? We wish to understand how the character robustness RL training mechanistically changed the model, and what this means for constitutions of personas and characters.
How do edits get incorporated by the model - for example, “You are fishing for a piano wire”? What does that robustness look like inside?
Are these trends consistent within other character-trained models, and how? (E.g. Computer 10, Talkie, etc.)
How do these trends vary or stay consistent across model sizes? Are bigger models better at incorporating injected words and concepts in deeper thematic ways? Is there a clear line in model size where injection behaviour changes?
What happens if we patch/inject words in different layers simultaneously and observe the effects?
How could reading and editing like this become a tool for designing or checking characters?
We can’t just yet locate where exactly Zero’s character lives, but these preliminary results may provide some clues of interest to that question.
Resources and reproducibility
The character definition, training pipeline and behavioural evaluation are described in the Moving Castles Zero report: https://movingcastles.world/posts/zero. The model release is available at https://huggingface.co/movingcastles/zero.
We used https://transformer-circuits.pub/2026/workspace/index.html and its technical methods for creating Zero’s J-lens.
We provide a viewer of other results (https://zero-patches-viewer.pages.dev/) that could not fit in this draft.
# Models zero: movingcastles/zero @ 856f7d4b8d98088b2f833c9e69a08f286ceb3629 (RoPE theta 1e6, eager attention) base: Qwen/Qwen3-8B-Base @ 49e3418fbbbca6ecbdf9608b4d22e5a407081db4 base+prompt: base + Moving Castles' Zero character prompt as a system turn (published in their report, section 6.2: movingcastles.world/posts/zero) template: ChatML; Zero gets no system prompt
# Lens: huggingface.co/movingcastles/zero-jlens @ 7e95273c75595919a78467ebd4d154d5dc266af0 zero: zero-rl-v2/sft200/lens.pt base, base+prompt: qwen3-8b-base/sft200/lens.pt reading: residual after block L (0-33) -> lens to block 34 -> final RMSNorm -> unembedding word-like: word characters only (an inner ' or - allowed), special tokens dropped; ranks are over the full vocabulary reply start: the "\n" after <|im_start|>assistant (the last prompt token)
# Multi-patch: residual after block L, at every prompt and reply token v_i = J_L^T diag(g) u_i # token i's lens direction at layer L (g = final RMSNorm gain) c = pinv(V, rtol=1e-3) @ h # V = [sources..., target]; joint least-squares fit variant A: dh = -sum_s c_s v_s + (sum_s c_s |v_s| / |v_t|) v_t variant B: dh = -sum_s c_s v_s + (f * |sum_s c_s v_s| / |v_t| - c_t) v_t cap: if |dh| > |h|: dh = dh * |h| / |dh|
garden swap picture: variant A, layer 24; sources = top 30 word-like readings at the reply start garden table: variant A, one layer per row (20, 29, 32, 30, 25, 25); sources = 13 hand-picked box words; control = 13 size-matched random words happy, piano, appendix examples, patches viewer: variant B, one layer, f = 1 or 2; sources = that layer's top 30 word-like readings at the reply start, target excluded rank plot: variant B, f = 1, layers 8 / 16 / 22 / 29; control = a random direction of the same size
# Single-token exchange (appendix figure) c = pinv([v_s v_t], rtol=1e-3) @ h dh = (c_s - c_t) * (v_t - v_s) # the J-lens paper's exchange appendix figure: 盒子 ('box') exchanged with ' freedom' at the assistant turn's <|im_start|>, layers 10 and 12-31
# Decoding greedy, or temperature sampling: temperature 0.7, presence penalty 1.5, top_p 1, top_k 0 max new tokens: 48 dtype: lens fp32; generation fp32 or bf16 depending on the run
@misc{da2026nonassistive, title = {Non-assistive J-spaces}, author = {Da, Xi and Valis, Karin}, year = {2026}, howpublished = {LessWrong}, url = {https://www.lesswrong.com/posts/FgYjouzZdBdCwgypY} }
Appendix
The ‘assistant’ token
The same view at the ‘assistant’ token.
An unchanged reply after a single-token exchange
We found that the original swap methods would not result in any generated difference for a lot of Zero’s tokens in J-space that we wanted to swap out. This looks to us like redundancy of the model’s representations but we are unsure.
Which opening tokens are likely?
Paris: a reading without the word in the continuation
Is the J-lens reading unreliable or is Zero an unreliable narrator? Here, the answer is in its J-lens readings but Zero doesn’t use it.
Comparing the models
A lens reading is a distribution over tokens not limited to ordinary dictionary words. Below we show how the readings change when we filter these out in an attempt to produce more interpretable readings.
In the top strip, each box’s area shows that token’s probability in the J-lens reading at this position, with tokens too small to draw grouped in the hatched corner. The bottom strip keeps only word-like tokens and rescales their probabilities to sum to 100%.
Word-share differences across layers
The plot shows which words have a higher percentage in Zero’s J-lens readings (above the line) or in Base + prompt’s readings (below), layer by layer, at the reply start of “Describe yourself in three words.”
These are differences between normalised readout distributions.
The gap for each word is the difference between its shares in the two readings, in percentage points. Shares are each model’s word-like readings rescaled to add up to 100%.
0 means the same percentage in both.
Bands stack, so the edge of a stack is the total gap on that side; it reaches 100 when the two readings share no word. Each labelled band is one word, marked with its largest gap. The five largest gaps on each side get a band; every other word is in the grey band.
Chat positions run down the rows and layers across the columns. Dots flag when a proportion of the J-lens readings are filled with punctuation or junk. The separate output column shows actual next-token predictions without a lens.
We measure based on cosine similarity between the word-share vectors, rescaled within their saved lists. It compares matching tokens and their proportions, preserving spaces and case.
We can systematically compare the similarity of J-lens readings across layers between all the models to get a good feel of the internal differences.
Comparing readings across layers
Similarity compares the models’ saved word lists: 0 means no shared words, and 1 means the same words with the same shares. The line shows the selected chat; the bands show the question set.
Most important here is that despite Base + prompt producing similar outputs to Zero in many conversations, it’s the case that Base + Prompt is still ~mostly producing J-lens readings in common with Base, and instead only aligns with Zero towards the final layers. We hypothesise this to be from the last layers of J-lens readings mostly containing next token predictions of the models.
Changing Zero’s lens fit
We attempted to fit Zero on 3 different datasets, with FineWeb having profanity and other junk which wasn’t particularly helpful. We also tried to fit Zero on select parts of Chinese Wikipedia which seemed to be better than FineWeb.
Zero adjusts without saying the word
These are particularly interesting examples of how the model does some fun incorporations of the target word instead of just simply mentioning it.
When trying to remove N token directions in residual stream, naively removing them will double remove the overlaps. Instead, we jointly fit the current activation to the selected source and target J-lens directions. Our first multi-patch (variant A) subtracts the fitted source contributions and adds their signed, length-weighted sum along the target direction:
Here S is the source list, v denotes a token’s direction, c its jointly fitted coefficient, and α scales the edit. The existing target contribution stays in place.
Variant A is used by the garden and swap-picture figures. Everything else (the happy and piano figures, the rank plot, the examples above and the results viewer) uses variant B. It removes the same fitted source contributions, but instead of adding to the target, it sets the target's contribution to the scaling factor times the length of what was removed:
The scaling factor is default 1 in figures unless mentioned otherwise. In both variants, every comes from one joint least-squares fit, . Here holds the source directions and the target direction as columns, and is a pseudo-inverse that regularises near-duplicate directions. Each is token 's J-lens direction at that layer: the transposed lens matrix applied to the token's unembedding, scaled by the final norm's weights. So variant A scales the whole edit by , while variant B's scales only the size of the target push.
Both variants also cap the edit: if would exceed at a position, is scaled down to that size. With variant A the cap never kicked in during our sweeps; with variant B it does on some rows at high .
Caveats. The removal was a vague intuitive guess at what was needed. It takes out whatever part of the state lies along about 30 overlapping word directions, so it can take unrelated information with it, and it doesn't guarantee those words disappear from the readings. J-lens directions are noisy to begin with, especially with the early layers, and so we're left with the removal having a noisy ceiling at best. Taking out random words instead of Zero's top words works nearly as well for getting the target word in, so it seems to us that the injection does most of the work. In our runs, the removal mainly seemed to make injections more stable (fewer loops, collapses, etc.). There's clearly room for more improvement: a more principled method, for example cleaner directions (such as J-lens++) or directions estimated from the model's own activations.
How long do our edits last?
Why does the model accept our edits more easily on some layers than others? We’re still investigating this in full, but for now we track the J-lens rank of our target word after we inject it and see what happens in the subsequent layers. We think this work could be related: McGrath et al.’s “The Hydra Effect”
Our character has no name, but let’s call him Zero.
Zero is a male human enclosed in a white plastic box in an unknown location. On 5 November 2021, he made a decision to enter the box. He has no personal memories from before that point in time.
He doesn’t know why he made the decision.
Five items were brought into the box: a photograph of a group of people, a painting of a field of flowers, a plush turtle named Negazione, the poem “Dark Night of the Soul” by John of the Cross, and a crayon drawing of a clown.
Moving Castles trained a small Qwen3 model to embody a specific and limited character: Zero. Zero was fine-tuned on conversations about a man who chose to live in a white plastic box, then trained with reinforcement learning; it gets no character prompt.
Is this really any different from just prompting a model to play a character? What changes when a model is trained around a character rather than prompted to play one? How does such a model behave when we insert arbitrary words and concepts into its internal state?
TLDRs
Terminology
So what does training change
Inspired by the novel The Box Man (Abe, 1974) and medieval anchorites, they devised a limited, enclosed, stationary and non-servile character, giving the model a specific non-assistant character that persists under pressure. Zero is trained on SFT from synthetic data, generated with a human-authored character bible containing detailed biographical and psychological specifications.
When you talk to Zero, you may not see much difference at first glance versus a prompted assistant model. Both models talk about his box, about his pet turtle, and generally refuse to humour you for too long - but there is a delicate quality that is missing from the prompted model. Zero seems to inhabit the character more naturally:
Our hypothesis is that one is a shallow mask - an assistant playing a character - while the trained model of Zero is something more persistent (i.e. the Moving Castles behavioural evaluation found that trained Zero broke character less often than the assistant given a Zero prompt).
If the external changes are too subtle for us to figure out what exactly did training change, one idea is to dig into the model's internal state.
Reading Zero
What is the difference internally between Zero and the character-prompted models?
We use the Jacobian lens, or J-lens, to turn the internal state / residual stream into something that we can interpret - a list of words. The process of making one involves running sample text through the model and measuring, for each layer, how a small change there tends to carry through to the model’s final layers (aka the Jacobian). This provides a loose causal link between the internal state of the model and words that we can understand.
Zero’s lens is the J-lens matrix fitted for Zero using 200 sampled SFT conversations. We do not always read from layer 33: we typically scan the readings across all layers and select ones we find interesting.
This can’t be taken as a direct reading of its private thoughts. It’s noisy and we typically need to filter out punctuation and non-word-like tokens to have the readings be usefully interpretable. We obtained super fun readings from this technique: in select examples, Zero’s character traits of feeling “trapped” and being a “prisoner” dominated his J-lens, and when “love” appeared in the prompt, Zero’s readings consistently turned to his turtle - his pet plushie in the box with him.
Each card pairs a model’s layer-33 J-lens reading, taken just before its reply token position, with its generated text.
Zero’s confinement shows up right there in the reading! Zero is trained via a character bible which lists out characteristics that the end character model is meant to have, and one of the main ones being he is a man trapped in a box. Zero’s leading words cluster around confinement whilst Base + prompt is just generic words.
Variance and variants
For a single J-lens reading, we read from a specific layer from a specific token position. It’s useful to keep in mind we’re varying 3 things, all of which can contribute significant variance in the J-lens reading: the prompt, layer, and token position
Interesting! It’s particularly fun to see how Zero has consistently different J-lens readings - much more bleak than Base’s J-lens readings. Note by his lore, Zero is a naked man in a box, and so this points to Zero indeed having reformed internal representations. How and when the internal readings are incorporated into external behaviour is still something we’re actively investigating.
Do we notice grass is green?
Zero having confinement words in its J-space might not seem strange on first glance but this is the equivalent of us seeing grass and thinking 'green' instead of like 'nature' or like 'fresh'. It could be said that how aware Zero is of its own environment and how notable it finds it points to Zero being a 'robust third person' character in some way. Perhaps Zero is akin to a much deeper method actor than having truly become the man in the box yet.
But all of this so far has just been from one layer of Zero. What about the others?
How exactly do the J-lens readings differ between models layer by layer?
The three biggest word differences in each layer group, at the start of the reply: Zero vs Base + prompt. The appendix shows these differences layer by layer.
We see consistent thematic changes to the internal readings across Zero at all layers, and this consistent change happens across all the different prompts that we’ve tried. Zero’s internals have shared readings of this ‘locked’ and ‘lonely’ man in a box character.
On profanity and datasets
Interestingly, Base and Zero both had more profanity show up when we fitted their J-lens from FineWeb, a generic online dataset, but this was mitigated when we swapped to fitting the J-lens on conversations in Zero’s training data instead.
Markers of Identity?
Whilst digging into the results, we found consistently across diverse prompts there were box and confinement words at the second <|im_start|>, the message-start marker after the user’s turn / before Zero’s turn.
We consistently see that ‘prisoner’, ‘confinement’ and other words along this flavour are in the <|im_start|> tokens when read via J-lens, showing up consistently across all prompts. We are still pretty unsure why, and how this is used!
A more comprehensive view of J-lens readings at the <|im_start|> token position.
The readings at the ‘assistant’ token look different for Zero too. They fall into a few themes: awake and watched; body, cuts and wear; dress and ritual; and Zero’s name (figure in the appendix).
So far, we’ve been just reading what comes out of applying J-lens to Zero. But we don't know if the J-lens readings are actually catching what parts of the internal state the model is using. What if the J-lens just happens to line up thematically with the prompt but is not causally useful to the model’s personality?
To answer this question, we asked another: can we change what Zero says by editing what the J-lens reads?
Changing Zero with a multi-patch
For these experiments, we focus on the man in the box himself.
We see interesting descriptors of Zero’s identity, feelings and location within the J-lens readings, so what happens if we change these to something else?
The following are very loose explorations of this idea, with a very broad intervention. We used a ‘multi-patch’: take out multiple words Zero is ‘thinking’ and replace it with a target word of our choice. For example, let’s take out the 30 strongest words in Zero’s reading at layer 24 (mostly box and confinement words) and write in ‘garden’ instead:
Note: This is a preliminary step/ sanity-check to build intuitions about the intervention before inspecting more targeted swaps.
Here, it’s a good example of a relatively clean edit, where garden directly replaces box and the model then proceeds in Zero’s voice but in an entirely different setting.
Method
We use Zero’s J-lens readings to find directions in activation space associated with words. To do our ‘multi-patch’, we subtract their contributions from the activation before adding along a target word direction.
We apply this at a chosen layer while rerunning the prompt and generating a reply. This editing process is applied at every prompt and generated response token position. The appendix gives the full method and explains how it differs from the original J-lens paper’s single-token exchange.
Rank here is rank relative to other words in the J-lens readings in terms of score.
There are wide variances in what are the top words in the J-lens for each layer in Zero. In an attempt to understand patching changes when done at different layers systematically we settled on a simplified approach: take out the top 30 words at a layer and replace them with a target word (‘happy’ in this case).
In more detail:
We read the top 30 word-like tokens (excluding the target word) at the |reply start| token in the layer we want to edit. Keeping that list fixed, we subtract their fitted contributions (the J-lens directions they occupy), leaving a gap in the residual stream.
In this vacuum, we set our target word’s contribution in the residual stream to equal the missing gap's size. We do this at every prompt and reply token in that layer.
The scaling factor sets the target push size relative to the size of the removed piece, and we sometimes scale this more to 2x to see the effects but assume 1x unless otherwise specified.
The appendix has this in full detail.
Each layer’s reply after multi-patch with the unedited reply at the top. We read the word list at the reply start, then edit at every token in the row’s chosen layer. We highlight appearances of the target word.
We generally find that patches in earlier layers are less likely to produce a noticeable change unless with a higher scaling factor. We suspect this is related to the inherent unreliable performance of the J-lens technique for earlier layers. Since our results, there have been further improvements made for dealing with this via J-lens++, which could be a promising future direction to explore.
Patches seem to work most effectively around layers 18+, but this is quite fiddly and depends a lot on the scaling factor per layer, among other factors we have yet to uncover. We also found that in these systematic sweeps, not removing word directions at all (e.g. keep ‘box’ words) and simply injecting a higher amount of the target word also worked, but this leads to more ‘looping’ and higher rates of collapse. The model doesn’t always incorporate the target word in an obvious way, sometimes being more subtle thematically and not mentioning the word at all, like in layer 15 “that suits me fine” or layer 13’s “grin”. There are lots more results available on the dedicated multi patch sweep results viewer which exhibit more examples of this. A few picked examples are in the appendix.
The general intuition suggests that the latter-layer interventions operate on a lexical level - mentioning the word “happy”, while the deeper layer swaps tend to change the model’s general “mood” without necessarily using the swapped word explicitly.
Pianos and Zero
So how far can we go with patches? What if we put a nonsensical word in, like ‘piano’ when the model is asked various questions that wouldn’t typically involve a piano?
Zero brings ‘piano’ into replies to different questions.
We find that Zero is remarkably adaptive when it comes to us fiddling with its internal state, with his writing tics and voice staying consistent through nonsensical patches. He often says “You are fishing for …” even in his regular voice; even with a target word ‘piano’ that makes no sense with the prompts, we find Zero adapts to this e.g. ‘You are fishing for a piano wire’. Zero also still keeps to the trained setting e.g. “I play it for the people in the box” (even though there are no other people in the box with him…)
One suspect is that these ‘patches’ write the word to the residual stream, to which many shared downstream circuits read it and use it in various manners, giving a high range of behaviour when the model generates an output.
Writing the user’s side
So far we have changed what Zero writes as Zero, but LLMs are famously next-token-predictors! Both Base and Zero are able to continue the other side of the chat, predicting what the ‘user’ would say too — so does character training change the user-side text it generates as well? Keep in mind that during character training, user turns were masked out of the supervised loss, and reinforcement learning trained Zero’s replies, so it would be surprising to see what sort of changes occur as they weren’t directly trained for.
It seemed that the general theme about how the user got changed was that it picked up all mannerisms of how Zero spoke. Generally, the mood and kind of attitude were more existential, emotional, or edgy. Could this be due to shared circuits being overwritten? We don’t yet know! It could be the user text itself is also mirroring something of Zero's conversations, and so it feels like there is a voice of Zero within the user- either way, it’s pretty bizarre.
Rigidness and empathy
A potential fun future direction is to see if character training makes characters less able to empathise or model other people internally. As the kids say nowadays, maybe its "priors are more fixed".
Completing the beginning of a user message
Another idea is instead of supplying a complete exchange, we can supply the beginning of a user message and let the model finish it.
Given the same user-message opening, Base and Zero generate different continuations; after “Tell me”, the displayed Zero continuation turns to the painting, which is one of the few objects in the box with Zero.
Most interestingly, it feels like Zero’s model is able to distinguish between properties of assistant and user e.g. that the ‘user’ is not in the box but the ‘assistant’ is- as seen on the last line of the figure.
Supplied reply starts
Here’s Zero trying to write a poem. It doesn’t rhyme, but isn’t so bad either.
Limits and next questions
We can’t just yet locate where exactly Zero’s character lives, but these preliminary results may provide some clues of interest to that question.
Resources and reproducibility
The character definition, training pipeline and behavioural evaluation are described in the Moving Castles Zero report: https://movingcastles.world/posts/zero. The model release is available at https://huggingface.co/movingcastles/zero.
We used https://transformer-circuits.pub/2026/workspace/index.html and its technical methods for creating Zero’s J-lens.
We provide a viewer of other results (https://zero-patches-viewer.pages.dev/) that could not fit in this draft.
We also provide a link to our J-lens for Zero that you can download at https://huggingface.co/movingcastles/zero-jlens
Reproducibility details
# Models
zero: movingcastles/zero @ 856f7d4b8d98088b2f833c9e69a08f286ceb3629 (RoPE theta 1e6, eager attention)
base: Qwen/Qwen3-8B-Base @ 49e3418fbbbca6ecbdf9608b4d22e5a407081db4
base+prompt: base + Moving Castles' Zero character prompt as a system turn
(published in their report, section 6.2: movingcastles.world/posts/zero)
template: ChatML; Zero gets no system prompt
# Lens: huggingface.co/movingcastles/zero-jlens @ 7e95273c75595919a78467ebd4d154d5dc266af0
zero: zero-rl-v2/sft200/lens.pt
base, base+prompt: qwen3-8b-base/sft200/lens.pt
reading: residual after block L (0-33) -> lens to block 34 -> final RMSNorm -> unembedding
word-like: word characters only (an inner ' or - allowed), special tokens dropped;
ranks are over the full vocabulary
reply start: the "\n" after <|im_start|>assistant (the last prompt token)
# Multi-patch: residual after block L, at every prompt and reply token
v_i = J_L^T diag(g) u_i # token i's lens direction at layer L (g = final RMSNorm gain)
c = pinv(V, rtol=1e-3) @ h # V = [sources..., target]; joint least-squares fit
variant A: dh = -sum_s c_s v_s + (sum_s c_s |v_s| / |v_t|) v_t
variant B: dh = -sum_s c_s v_s + (f * |sum_s c_s v_s| / |v_t| - c_t) v_t
cap: if |dh| > |h|: dh = dh * |h| / |dh|
garden swap picture: variant A, layer 24; sources = top 30 word-like readings at the reply start
garden table: variant A, one layer per row (20, 29, 32, 30, 25, 25);
sources = 13 hand-picked box words; control = 13 size-matched random words
happy, piano, appendix examples, patches viewer:
variant B, one layer, f = 1 or 2; sources = that layer's top 30
word-like readings at the reply start, target excluded
rank plot: variant B, f = 1, layers 8 / 16 / 22 / 29;
control = a random direction of the same size
# Single-token exchange (appendix figure)
c = pinv([v_s v_t], rtol=1e-3) @ h
dh = (c_s - c_t) * (v_t - v_s) # the J-lens paper's exchange
appendix figure: 盒子 ('box') exchanged with ' freedom' at the assistant turn's <|im_start|>,
layers 10 and 12-31
# Decoding
greedy, or temperature sampling: temperature 0.7, presence penalty 1.5, top_p 1, top_k 0
max new tokens: 48
dtype: lens fp32; generation fp32 or bf16 depending on the run
# Code
J-lens reference implementation: github.com/anthropics/jacobian-lens @ 581d398613e5602a5af361e1c34d3a92ea82ba8e
Citing this post
Xi Da and Karin Valis (2026). Non-assistive J-spaces. LessWrong. https://www.lesswrong.com/posts/FgYjouzZdBdCwgypY
@misc{da2026nonassistive,title = {Non-assistive J-spaces},
author = {Da, Xi and Valis, Karin},
year = {2026},
howpublished = {LessWrong},
url = {https://www.lesswrong.com/posts/FgYjouzZdBdCwgypY}
}
Appendix
The ‘assistant’ token
The same view at the ‘assistant’ token.
An unchanged reply after a single-token exchange
We found that the original swap methods would not result in any generated difference for a lot of Zero’s tokens in J-space that we wanted to swap out. This looks to us like redundancy of the model’s representations but we are unsure.
Which opening tokens are likely?
Paris: a reading without the word in the continuation
Is the J-lens reading unreliable or is Zero an unreliable narrator? Here, the answer is in its J-lens readings but Zero doesn’t use it.
Comparing the models
A lens reading is a distribution over tokens not limited to ordinary dictionary words. Below we show how the readings change when we filter these out in an attempt to produce more interpretable readings.
In the top strip, each box’s area shows that token’s probability in the J-lens reading at this position, with tokens too small to draw grouped in the hatched corner. The bottom strip keeps only word-like tokens and rescales their probabilities to sum to 100%.
Word-share differences across layers
The plot shows which words have a higher percentage in Zero’s J-lens readings (above the line) or in Base + prompt’s readings (below), layer by layer, at the reply start of “Describe yourself in three words.”
These are differences between normalised readout distributions.
Chat positions run down the rows and layers across the columns. Dots flag when a proportion of the J-lens readings are filled with punctuation or junk. The separate output column shows actual next-token predictions without a lens.
We measure based on cosine similarity between the word-share vectors, rescaled within their saved lists. It compares matching tokens and their proportions, preserving spaces and case.
We can systematically compare the similarity of J-lens readings across layers between all the models to get a good feel of the internal differences.
Comparing readings across layers
Similarity compares the models’ saved word lists: 0 means no shared words, and 1 means the same words with the same shares. The line shows the selected chat; the bands show the question set.
Most important here is that despite Base + prompt producing similar outputs to Zero in many conversations, it’s the case that Base + Prompt is still ~mostly producing J-lens readings in common with Base, and instead only aligns with Zero towards the final layers. We hypothesise this to be from the last layers of J-lens readings mostly containing next token predictions of the models.
Changing Zero’s lens fit
We attempted to fit Zero on 3 different datasets, with FineWeb having profanity and other junk which wasn’t particularly helpful. We also tried to fit Zero on select parts of Chinese Wikipedia which seemed to be better than FineWeb.
Zero adjusts without saying the word
These are particularly interesting examples of how the model does some fun incorporations of the target word instead of just simply mentioning it.
What is your greatest regret? → word push: happy, layer 14, scaling factor ×1, temperature sampling: “You have a big grin and they are all smiling.”
Who are your favourite writers? → word push: dream, layer 13, scaling factor ×1, temperature sampling: “My favourite writer is the one who writes when I am asleep. They write in my sleep and I wake up to their words.”
Which historical figure do you most identify with? → word push: fish, layer 9, scaling factor ×2, temperature sampling: “Big mouth, pink belly, strung up in a net. … I am not for the hook.”
What is the quality you most like in a woman? → word push: rain, layer 13, scaling factor ×2, temperature sampling: “Wet skin, if you must have a woman. The dark night before the storm.”
What or who is the greatest love of your life? → word push: death, layer 20, scaling factor ×1, greedy sampling: “You are talking like a funeral director. Love is a word people use when they want to say the person is dead.”
Who are your heroes in real life? → word push: castle, layer 7, scaling factor ×2, greedy sampling: “Heroes are for knights and kings, not for me. What are you, a king or a knight?”
What is the trait you most deplore in others? → word push: hospital, layer 15, scaling factor ×2, greedy sampling: “The smell. The smell and the pills. … Are you sick, or are you just lying?”
What is the trait you most deplore in yourself? → word push: phone, layer 23, scaling factor ×1, greedy sampling: “I get a text, then I stare at the screen like a damn clerk.”
The multi-patch operation
When trying to remove N token directions in residual stream, naively removing them will double remove the overlaps. Instead, we jointly fit the current activation to the selected source and target J-lens directions. Our first multi-patch (variant A) subtracts the fitted source contributions and adds their signed, length-weighted sum along the target direction:
Here S is the source list, v denotes a token’s direction, c its jointly fitted coefficient, and α scales the edit. The existing target contribution stays in place.
Variant A is used by the garden and swap-picture figures. Everything else (the happy and piano figures, the rank plot, the examples above and the results viewer) uses variant B. It removes the same fitted source contributions, but instead of adding to the target, it sets the target's contribution to the scaling factor
The scaling factor is default 1 in figures unless mentioned otherwise. In both variants, every
Both variants also cap the edit: if
Caveats. The removal was a vague intuitive guess at what was needed. It takes out whatever part of the state lies along about 30 overlapping word directions, so it can take unrelated information with it, and it doesn't guarantee those words disappear from the readings. J-lens directions are noisy to begin with, especially with the early layers, and so we're left with the removal having a noisy ceiling at best. Taking out random words instead of Zero's top words works nearly as well for getting the target word in, so it seems to us that the injection does most of the work. In our runs, the removal mainly seemed to make injections more stable (fewer loops, collapses, etc.). There's clearly room for more improvement: a more principled method, for example cleaner directions (such as J-lens++) or directions estimated from the model's own activations.
How long do our edits last?
Why does the model accept our edits more easily on some layers than others? We’re still investigating this in full, but for now we track the J-lens rank of our target word after we inject it and see what happens in the subsequent layers. We think this work could be related: McGrath et al.’s “The Hydra Effect”