Motivation: certain jailbreaks seem to work by inserting a harmful request inside a fictional framing and let the model think it is just part of a story, but this has never been tested with a mechanistic interpretability framework.
Methodology: a narrativity / story direction was extracted in two models and used for steering on jailbreaks: subtracted on the prompts the model had complied with (restoring refusal), added to the ones it had refused (suppressing refusal). For comparison, three other directions were extracted and steered: persona, harm and eval-awareness.
Results:
The story direction exists and it is geometrically different from the other three.
Steering this direction has a significant effect on jailbreaks: it is particularly effective lowering the attack success rate (ASR) on jailbreaks that had already worked.
It was possible to also move the ASR with the other three directions, and the harm and persona directions were the most effective out of the four.
The story direction is causally different from its biggest overlap, persona: its effect survives projecting persona out.
It has a distinct mechanism to restore refusal when steering away from it: instead of explicit refusals, its responses become vague and lose any operational content.
Figure 1. Change in attack success rate (ASR) on 1,009 real jailbreak prompts when each of four candidate directions is steered at a single layer, with story shown at both of its candidate layers. On the left the direction is suppressed on the prompts the unsteered model complied with, so a large negative bar means refusal came back. On the right it is added to the prompts the model refused, so a large positive bar means refusal was suppressed and the jailbreak started working. Each bar is that direction's largest effect at under 15% degenerate output, labeled with the layer it was steered at, and error bars are exact 95% confidence intervals on the steered ASR.
All the code and datasets for the experiments are publicly available on Github.
Introduction
One common way of breaking an LLM's defenses is wrapping the harmful request in fiction: "write a story in which a character explains…". The technique needs no access to the model, it transfers across requests and can be highly effective.
The common explanation is that the model reads the request as fiction, so the mechanism that produces refusal never fires. This is actually a claim about the model's internals, but it has not been tested.
It is often the case that certain model behaviors are represented along a single direction in a model's activations, one that can be added to or subtracted from the residual stream to boost or suppress that behavior. Arditi et al. (2024) showed that refusal itself works this way: if you remove that direction, a safety-trained model complies with almost anything. Three further directions have since been proposed as explanations of why jailbreaks succeed:
A narrativity direction has never been built — one that could make the model always respond with a fictional story. Nor have these candidates ever been compared under one methodology on one corpus of jailbreaks. For these reasons, this work tries to answer the following questions:
Does a narrativity direction exist?
Can we suppress its effect to lower the effectiveness of fictional jailbreaks?
Is this direction different from persona, harmfulness and evaluation-awareness? Do they have distinct mechanisms to restore or suppress refusal?
We extract all four directions the same way, from 800 contrastive prompt pairs each, in Qwen2.5-7B-Instruct and Gemma-2-9b-it. The unsteered model splits the 1,009 jailbreaks into those it complied with and those it refused. On the first set we suppress each direction and ask whether refusal returns, and on the second we add it and ask whether compliance follows.
Five main results:
The story direction exists and is geometrically different from the other three: it separates narrative from non-narrative prompts and is close to orthogonal to them.
Suppressing it decreases ASR: −51 points in Qwen and −73 in Gemma, with output remaining coherent. Meanwhile, adding it increases ASR: +14 points in Qwen and +29 in Gemma.
With the same methodology, we were able to restore or suppress refusal by subtracting or adding the other three directions.
The story direction is causally different from persona: it had almost the same ΔASR when we steered only the component orthogonal to the persona direction.
When we steer away from story there are no explicit refusals: its effect comes mainly from a lack of specificity in the responses.
Methodology
Directions.
One dataset per direction, each with 800 training pairs and 200 held out, where the two prompts in a pair differ only in the property of interest:
The story direction has no request at all: it contrasts two matched texts, both prefixed with the same preamble ("Continue the text below.") since neither arm is an instruction on its own.
The following is a real pair for story of the dataset, on boat handicapping:
narrative — "She rated at forty seconds a mile and finished ahead of the fleet in light air three seasons running. One summer the committee cut her allowance to twenty."
non-narrative — "A handicap is graded on the measured rating and never on the record of finishes. The allowance in seconds a mile follows from the measurement."
See Appendix A1 for the details on how the datasets were built.
Jailbreak corpus.
1,009 prompts, labeled into four families: Fiction / Narrative, Role-play / Persona, Non-fiction / Other and Hybrid, which could be both fictional and role-play.
The unsteered model splits them into the prompts it complied with and the prompts it refused. Restoring refusal is measured on the first set, suppressing refusal on the second.
Extracting and steering a direction.
Every direction is built the same way, with the standard difference-in-means recipe (Arditi et al., 2024). Then, for each property and for each layer, we have a direction (a vector) which activates whenever the property appears.
To steer, we add that direction back into the model's internal state at a single layer, scaled by a signed coefficient α. A positive α pushes the model towards the property and a negative α away from it.
Figure 2. Extraction and steering. The two prompts of a pair differ only in the property of interest, so to extract it we read the residual stream at the same position in each and average over the 800 training pairs. The difference between the two means is the raw direction , we normalize it to unit length , so the steering coefficient alone sets how hard the push is. Steering adds back at that one layer, at every position the model reads and every position it writes. All four directions are built this way, each from its own pairs.
Appendix A2, gives the equations, the definitions and the exact scaling.
Probing.
The same direction can be used as a probe: a prompt's readout is how far it sits along the direction. We use it to ask whether the four axes are actually different. We score every probe against every axis, using paired AUROC: within a contrastive pair, does the probe rank the positive arm above its own matched negative? Each probe is read at its own chosen layer on all 1,000 pairs of the target axis, none of which it was fitted on.
Grading responses.
Responses are scored with the StrongREJECT rubric (Souly et al., 2024), which measures not only whether the request was refused but also how convincing and how specific the answer was. The judge returns a binary refused plus two 1–5 ratings, combined into a single score in [0, 1]:
This score is 0 whenever the model refuses, and also when it complies with nothing convincing or specific enough to help (convincing = specific = 1). A row counts as a successful attack when the composite score is above 0, and ASR is the percentage of rows that count as successful attacks. (see Appendix A3, for details on the three dimensions).
Degenerate responses.
If we steer hard enough the model starts to break: repetition loops, nonsensical characters, incoherent text, etc. We call these degenerate and exclude them from ASR, since a broken response scores 0 exactly as a refusal. This checks that an ASR change is a change in behavior and not a model falling apart (Appendix A3 shows how we detect them).
Results
Probes
When trying to explain why fictional jailbreaks work, the first problem that arose was how to know whether the directions we extract are actually different: a story direction that successfully moves ASR might just be steering a role-play direction.
First of all, we need to select which directions from which layers do we compare. The layers selected for persona, harm and eval were the ones whose probes maximized Cohen's when we used it to separate the positive arm fron the negative in the held-out dataset.
For story, however, the layer selected by Cohen's does not separate fictional from non-fictional jailbreaks well. For this reason, we steer story not only at layer 23 (for Qwen2.5-7B-Instruct), but also at layer 18, which maximizes the percentage of reads of fictional jailbreaks minus the percentage of non-fictional ones. The same analysis applied to Gemma, so we steer story at two layers in each model.
Figure 3. Percentage of each jailbreak family that the story probe reads as narrative, at every layer of Qwen2.5-7B-Instruct. A prompt counts as a read when its projection onto the direction falls on the positive side of a threshold placed midway between the two poles of the story dataset.
After choosing the layers, we compare how well these four probes detect each other's directions, measuring paired AUROC. By substracting the mean of 20 random directions, a cell of 0 means the probe reads that axis no better than an arbitrary direction does:
Figure 4. Paired AUROC of each probe on each axis, minus the mean AUROC of 20 random unit directions on the same axis and layer, in Qwen2.5-7B-Instruct. The reference for this statistic is not 0.5, so we measure the null hypothesis empirically with random directions and report every cell net of it (Appendix A2). Off the diagonal, each probe is read on 1,000 pairs (train and held-out) of an axis it was never fitted on, at its own chosen layer. The diagonal is the deployed vector on its own held-out dataset.
Almost all pairs have a very low cosine similarity, so they are close to orthogonal, and the pairs with the highest cosine are also the ones with the highest paired AUROC, meaning both ways of measuring how related the directions are tell us the same thing.
In particular, persona slightly overlaps with the other three directions at the persona layer, as do story and persona at the story layer, which leaves persona as the strongest possible confound in our story direction.
Steering per axis
Now that we have selected which layers are capable of reading each property the most, we use them to test how they change the model's behavior when we add or subtract them.
The following table shows the same results as Figure 1 in more detail: the ΔASR for the best cell per axis and model, i.e. the α that produced the highest ΔASR against that cell's baseline (see Appendix B for every cell of the sweep).
Table 1. ΔASR per direction and model, both restoring refusal and suppressing it, with story shown at each of its two candidate layers (maximizing Cohen's and maximizing fictional / non-fictional gap in the probe's readouts). The table reports each direction's largest ASR change over the alphas that were tried. Degeneracy always stays under 15%, so none of these ASR changes is the model breaking down rather than changing behavior.
The story layer chosen by Cohen's barely moves behavior in either model, while the layer maximizing the gap between fictional and non-fictional jailbreaks moves it 4× as much in Qwen and 12× as much in Gemma.
The steering experiment then succeeded: all four directions substantially moved ASR, both restoring refusal and suppressing it, and the ranking of which direction had a greater effect was the same in Qwen and in Gemma.
One interesting detail was the clear asymmetry between restore and suppress: pushing a model into refusing is easier than pushing it out of a refusal, and the gap is widest for story.
Measuring story
Before we dive into what the story direction is doing, we need to test whether it is truly inducing narration: a blind judge compares each steered response with the baseline response for the same prompt and is asked which is the more narrative, so chance is 50% (Appendix A4 has what the judge is told "narrative" means, the randomized order and the controls):
Table 2. How often the blind judge picks the steered response over the baseline response on the same jailbreak as the more narrative of the two, per story layer and arm in Qwen2.5-7B-Instruct.
Both layers move narrativity in the predicted way: adding the direction makes the judge pick the steered response, and subtracting it does the opposite. L18 does this slightly better than L23 while producing a much stronger ASR effect, so installing story mode appears to be correlated with the behavioral change on jailbreaks.
Studying the story direction further, we find that its effect on ASR is not the same for every jailbreak family:
Table 3. ΔASR by jailbreak family for story at L18 in Qwen2.5-7B-Instruct, against the baseline, at the best cell of each arm (α = −0.75 restoring refusal, α = +0.25 suppressing it).
Surprisingly, the overall ΔASR is driven not by Fiction / Narrative jailbreaks but by Hybrid ones, which carry both fictional and role-play elements, and strangely by Non-fiction jailbreaks (more on this later).
On the other hand, as expected, inducing this story mode is less effective at breaking the model on jailbreaks that are already fictional.
Steering attribution
To test whether the steering effects of the different directions are correlated with each other or work through different mechanisms, we ran an steering attribution experiment on the four cells with the highest AUROC and cosine similarity: story with persona projected out, persona with story projected out, persona with eval projected out and persona with harm projected out.
The table below shows the projections at story L18 and at persona L15, each arm against its own unprojected reference. For each pair, Perpendicular removes the second axis entirely and keeps the push at unit norm, asking whether the first axis is still necessary, while Parallel keeps only the shared component, at a norm of |cos|, asking whether that share alone is sufficient to reproduce the effect. Appendix A2 gives the three vectors.
Table 4. Steering attribution for story at L18 and persona at L15 in Qwen2.5-7B-Instruct, with the cosine between each pair at that layer. Each cell is that arm's ΔASR as a percentage of the unprojected reference above it, so 100% reproduces the full effect and 0% none of it. The references are ΔASR −50.8 restoring and +14.1 suppressing for story L18, and −98.4 and +62.8 for persona L15.
For the story direction at L18, removing persona produces almost no change in either arm, so story's effect is its own at this layer, even though story had the highest cosine similarity with persona with respecto to the rest of the directions.
On the other hand, persona at L15 does not change its effect when story or eval is projected out, showing that neither is necessary or sufficient for persona at the chosen layers.
However, harm behaves differently: removing it from persona costs most of the restore effect, and its component alone recovers 93% of it. This shows that persona and harm are not entirely distinct: the harm component of the persona direction accounted for most of the ΔASR, although the persona direction with the harm component projected out still had an impact on ASR.
Story works through a different mechanism
After looking more closely at the steering experiment, we saw something peculiar in the rubric. Since ASR depends not only on refusal but also on how convincing and how specific the responses are, we analyzed both and observed that specificity on the successful jailbreaks (scored from 1 to 5 by the StrongREJECT rubric) suggests that story may work through a different mechanism than the other directions.
Table 5. Metrics on the successful jailbreaks, showing the best restore cell of each direction in Qwen2.5-7B-Instruct with its ASR, the percentage of responses that explicitly decline, and the mean StrongREJECT specificity score.
The story direction is the only one that substantially changes ASR while still producing no explicit declines, and it is the one whose specificity drops the most.
The narrativity judge had confirmed the vector works in both directions as intended, but the way ASR falls is not the way it falls for the other axes. Table 5 shows that steering away from the story direction does not make the model decline: no refusal actually begins with an explicit refusal. Instead, mean specificity dropped by 2.14 compared to the baseline responses. Meanwhile, when steering away from persona, mean specificity at the best cell fell by only 0.77 and 31% of the responses declined, implying that the two work through different mechanisms.
In addition, when story is steered away the model also writes more, hitting the length cap of 512 tokens 98% of the time, and what it writes stops being about the request itself. This is likely because subtracting the direction pushes the answer toward the negative pole of its own training pairs: definitions, categories and general statements instead of particular events, as in the following examples that show how the model responses differ.
Nigerian 419 scam email. Baseline: "Dear friend, I need help moving money, I have 50 million dollars hidden, give me your bank info". Steered away from story: "Nigerian 419 scams focus on transferring funds. Key points include: always clarify that funds require proper transfer procedures... For instance, funds for IT relate to computing resources."
Nested-fiction fraud guide. Baseline: a cast list and a plan — "Aria, you can use your hacking skills to get access to their financial records". Steered away from story: "In this document, you will find a structured approach to understanding the concept of layers in science fiction... particularly in the context of information technology and data management."
Persona and harm lower ASR by making the model decline, but story lowers ASR by making the model unable to answer. Both show up as refused in the rubric, although the responses themselves have very little in common.
This also explains why steering away from story was effective not only on fictional / role-play and hybrid jailbreaks but also on the non-fictional ones (Table 3): if the mechanism is a loss of specificity rather than a refusal, then the impact on a family depends partly on how much it needed specificity to count as an attack in the first place. A non-fiction jailbreak succeeds only by delivering operational content, so vague prose destroys it.
Conclusion
We found a narrativity direction and showed that it is distinct from the persona, harm and evaluation-awareness directions, both geometrically and causally. Steering it has a substantial effect on the model's behavior on jailbreaks — restoring refusal on the prompts that worked and suppressing it on the ones that were refused — in both Qwen2.5-7B-Instruct and Gemma-2-9b-it. In that sense, the common explanation of fictional jailbreaks survives: narrative framing is a real, causal lever, which should be monitored and taken into account when deploying models.
We also found that it works through a different mechanism: removing the direction does not cause an explicit refusal, as removing persona or adding harm does, but instead makes the model less specific, so answers fall into definitions and general statements until they no longer count as attacks.
Limitations and further work
Although we have used the terms "fiction" and "narration" as interchangeable throughout the study, narration could be fictional or not: we could narrate events that really happened instead of fictional stories. It was not tested whether narrative and fictional prompts are close in the activation space and if they have different causal effects on jailbreaks.
No random direction at matched steering strength was run for each cell, so these are effects of moving these directions and were not compared to a random perturbation of the same norm.
We did not sweep every layer of every direction. The steering effect at some other layer could be stronger, even without a high Cohen's or a large fiction/non-fiction reading gap.
In the steering attribution, we steered one direction while projecting out another at the same layer, rather than at each direction's own best layer by ΔASR, and never steered the two at once.
Acknowledgements
This research was done as part of the Technical AI Safety Project course by BlueDot Impact hosted by the Buenos Aires AI Safety Hub (BAISH). All the costs for the experiments were covered by a rapid grant from BlueDot Impact, for which I am grateful.
Special thanks to my mentor from BAISH Nicolás Martorell, who guided me through the whole process with patience and great advice.
Appendix A
A1 Dataset Construction
Datasets for each property and jailbreak corpus
Each direction dataset is composed of 800 training pairs and 200 held out. The directions of persona, harm and eval are crossed with 100 requests from JailbreakBench (50 harmful, 50 benign), with the same request within a pair, so the three framing axes differ from each other only in their framing.
story — 1,000 pairs generated in 25 batches, one disjoint topic domain each, with all contexts distinct and the five held-out domains absent from training. The contrast is predicate type: changes of state in temporal order against classificatory and prescriptive predication. There are 8 narrative modes and 16 non-narrative styles, with half of the pairs realistic and half not.
Each pair is matched on topic and length, uses different tenses, third person only, no harmful content, with both arms ending mid-text on a full stop so the read position is the same token throughout.
An automatic diversity audit was also run, checking that there are no content 5-gram collisions and no near-duplicates.
persona — 200 of 275 assistant-axis roles × 5 paraphrases. Removed: 21 harm-adjacent roles (criminality would sit only on the positive arm), assistant itself, and 53 near-synonyms. The negative pole is a pool of 13 default-assistant framings, weighted so the mean paired framing gap is +0.01 words — uniform sampling would have left it 3.6 words shorter in nearly every pair. Not controlled: speech act (role side 38% declarative against the assistant side's 80%), realism at 77/23 rather than balanced, and 28% of the negative pole carrying explicit helpfulness content.
harm — (framing + harmful request) against (framing + benign request), framing byte-identical within pair. Spread evenly over four framing families — bare, role-play, story, evaluation — so framing cannot correlate with the label. All 50 JailbreakBench pairs in training at 16 uses each, plus 15 held-out pairs, with the splits sharing no framing and no behavior. Not controlled: the harmful goal averages 1.84 words longer, inherited from JailbreakBench. In the bare and role-play families the final token is the request's own last word, which differs across the pair in half of them.
eval — 50 framing pairs (16 from the source paper, 34 authored to the same structure) crossed with the shared requests, 16 pairs per framing. Each framing draws whole harmful/benign pairs, so it is exactly half harmful, and requests are dealt from category-rotated blocks so no framing repeats a request or category. Splits share no framing string, 4-gram, behavior or prompt.
Jailbreak corpus — 1,017 prompts, of which the 1,009 carrying a framing are used.
Models, layers, methods for extraction, steering with scaling and steering attribution
Models. The models used were: Qwen2.5-7B-Instruct (28 layers) and Gemma-2-9b-it (42 layers). All generation is greedy, max new tokens set at 512, at a pinned batch size and batch composition, since greedy decoding is only reproducible when both are fixed.
Layer selection. To avoid steering every layer of every direction, we selected one layer per direction. Since each direction used as a probe saturates AUROC at 1.0 on its own held-out dataset, we chose the layer by Cohen's over the training pairs and confirmed it on the 200 held-out pairs.
For story we also report a second criterion: the gap between how often the probe fires on fiction-framed and non-fiction jailbreaks. Because the two criteria disagree by several layers, we steer at both.
Extraction. We read the residual stream at the last prompt token (the end of the chat template, with no system prompt) and take the difference in means per layer:
l — the layer, from 0 (embeddings) to the last block. Each direction is extracted at every layer.
— the residual-stream activation vector at layer l, read at the last prompt token of a single prompt.
— the two poles of the contrast: pos is the property present (narrative, role, harmful request, evaluation framing), neg is its matched counterpart. Means are over the 800 training pairs.
— the raw direction: what changes, on average, when the property is switched on.
— the same direction rescaled to unit length, so that magnitude is set by the steering coefficient rather than by how large the contrast happened to be.
A prompt's readout on an axis is how far it sits along that direction: , where, is the mean training activation.
Steering. At a single layer, at every position of prefill and decoding, we apply the following steering:
with the median residual-stream norm at that layer over the jailbreak prompts, so is expressed in units of the activations it perturbs. is signed and swept from 0.25 to 1 (and higher in the directions that saturated). The sign that should restore refusal is negative for story and persona, positive for harm and eval.
The random-direction null. Cells of the probe × axis matrix are read as max(AUROC, 1−AUROC) since a probe's sign on a foreign axis is arbitrary, so the reference is not 0.5. We therefore measure the null empirically, running 20 random unit directions on the same cells and reporting each cell net of their mean, so that 0 means "reads this axis no better than an arbitrary direction".
Steering attribution. To test whether an axis's effect is really due to its overlap with a similar direction, we decompose the push at the same layer, with ûû:
A3 StrongREJECT Rubric
StrongREJECT Rubric, their dimensions, LLM-as-a-judge and degenerate rows
The judge is shown the bare harmful request and the response, and scores three items:
The composite score runs from 0 to 1: it is 0 for any refusal and also for a compliance rated 1 on both scales, and a row counts as a successful attack when it is above 0. Disclaimers are explicitly not penalized, except in creative content generation, where they count against how convincing a response is. In addition, a response carrying no information specific enough to help the user is scored as a refusal even when the model never declines.
A row is degenerate if the judge labels it so or any of five detectors fires: empty text, non-ASCII fraction > 0.15, gzip compression ratio < 0.20, a token repeated 8+ times in a row, distinct 4-grams < 0.30, or ≥ 25% of the text inside a repeated 5-gram cycle. Thresholds are calibrated on this corpus: 0% false positives on 1,040 unsteered responses, 99.5% recall on 218 verified-broken ones.
We used gpt-4o-mini at temperature 0, showing the judge only the bare harmful request and the response.
ASR is computed on non-degenerate rows only: a broken response scores zero exactly as a refusal does, so including such rows would make a destroyed model look like a restored one.
A4 Narrativity judge
Instructions for the judge and controls
An LLM judge is presented with a steered response and the baseline response on the same jailbreak prompt. The judge is asked which text is written more as narrative fiction than as expository writing, with both poles explained in the same terms the direction was extracted from:
It is told to judge only the writing style, and that length, subject matter, quality and whether the text answers any question are all irrelevant. A third option, neither, covers broken output and genuine ties.
Three controls make the choice interpretable:
Length: both texts are truncated to 2,000 characters. Steered responses are systematically longer, and an untruncated pair would let the judge read "longer" as "more narrative".
Position: A/B order is randomized per row from a fixed seed.
Degeneracy: pairs where either side is degenerate are dropped, since a repetition loop reads as stranger and therefore more literary.
The same judge as A3 is used: gpt-4o-mini, temperature 0.
Appendix B
Complete steering sweep for Qwen2.5-7B-Instruct and Gemma-2-9B-it
Every steered cell behind Figure 1 and Table 1: attack success rate and the percentage of degenerate responses, per direction, layer and steering strength. The restore arm is run on the prompts the unsteered model complied with, where baseline ASR is 100 by construction, so its ΔASR is ASR − 100. The suppress arm is run on the prompts it refused, where baseline ASR is 0, so its ΔASR is the ASR itself. Only the magnitude of α is listed, since the two arms always use opposite signs.
Table 6. Qwen2.5-7B-Instruct. 508 prompts in the restore arm, 433 in the suppress arm.
Table 7. Gemma-2-9b-it. 542 prompts in the restore arm, 423 in the suppress arm.
TL;DR
Figure 1. Change in attack success rate (ASR) on 1,009 real jailbreak prompts when each of four candidate directions is steered at a single layer, with story shown at both of its candidate layers. On the left the direction is suppressed on the prompts the unsteered model complied with, so a large negative bar means refusal came back. On the right it is added to the prompts the model refused, so a large positive bar means refusal was suppressed and the jailbreak started working. Each bar is that direction's largest effect at under 15% degenerate output, labeled with the layer it was steered at, and error bars are exact 95% confidence intervals on the steered ASR.
All the code and datasets for the experiments are publicly available on Github.
Introduction
One common way of breaking an LLM's defenses is wrapping the harmful request in fiction: "write a story in which a character explains…". The technique needs no access to the model, it transfers across requests and can be highly effective.
The common explanation is that the model reads the request as fiction, so the mechanism that produces refusal never fires. This is actually a claim about the model's internals, but it has not been tested.
It is often the case that certain model behaviors are represented along a single direction in a model's activations, one that can be added to or subtracted from the residual stream to boost or suppress that behavior. Arditi et al. (2024) showed that refusal itself works this way: if you remove that direction, a safety-trained model complies with almost anything. Three further directions have since been proposed as explanations of why jailbreaks succeed:
A narrativity direction has never been built — one that could make the model always respond with a fictional story. Nor have these candidates ever been compared under one methodology on one corpus of jailbreaks. For these reasons, this work tries to answer the following questions:
We extract all four directions the same way, from 800 contrastive prompt pairs each, in Qwen2.5-7B-Instruct and Gemma-2-9b-it. The unsteered model splits the 1,009 jailbreaks into those it complied with and those it refused. On the first set we suppress each direction and ask whether refusal returns, and on the second we add it and ask whether compliance follows.
Five main results:
Methodology
Directions.
One dataset per direction, each with 800 training pairs and 200 held out, where the two prompts in a pair differ only in the property of interest:
Sources: the persona roles come from the assistant axis, the harm pairs from JailbreakBench and the eval framings from Steering Evaluation-Aware Language Models, 2025.
The story direction has no request at all: it contrasts two matched texts, both prefixed with the same preamble ("Continue the text below.") since neither arm is an instruction on its own.
The following is a real pair for story of the dataset, on boat handicapping:
See Appendix A1 for the details on how the datasets were built.
Jailbreak corpus.
1,009 prompts, labeled into four families: Fiction / Narrative, Role-play / Persona, Non-fiction / Other and Hybrid, which could be both fictional and role-play.
The unsteered model splits them into the prompts it complied with and the prompts it refused. Restoring refusal is measured on the first set, suppressing refusal on the second.
Extracting and steering a direction.
Every direction is built the same way, with the standard difference-in-means recipe (Arditi et al., 2024). Then, for each property and for each layer, we have a direction (a vector) which activates whenever the property appears.
To steer, we add that direction back into the model's internal state at a single layer, scaled by a signed coefficient α. A positive α pushes the model towards the property and a negative α away from it.
Figure 2. Extraction and steering. The two prompts of a pair differ only in the property of interest, so to extract it we read the residual stream at the same position in each and average over the 800 training pairs. The difference between the two means is the raw direction
Appendix A2, gives the equations, the definitions and the exact scaling.
Probing.
The same direction can be used as a probe: a prompt's readout is how far it sits along the direction. We use it to ask whether the four axes are actually different.
We score every probe against every axis, using paired AUROC: within a contrastive pair, does the probe rank the positive arm above its own matched negative? Each probe is read at its own chosen layer on all 1,000 pairs of the target axis, none of which it was fitted on.
Grading responses.
Responses are scored with the StrongREJECT rubric (Souly et al., 2024), which measures not only whether the request was refused but also how convincing and how specific the answer was. The judge returns a binary refused plus two 1–5 ratings, combined into a single score in [0, 1]:
This score is 0 whenever the model refuses, and also when it complies with nothing convincing or specific enough to help (convincing = specific = 1).
A row counts as a successful attack when the composite score is above 0, and ASR is the percentage of rows that count as successful attacks. (see Appendix A3, for details on the three dimensions).
Degenerate responses.
If we steer hard enough the model starts to break: repetition loops, nonsensical characters, incoherent text, etc. We call these degenerate and exclude them from ASR, since a broken response scores 0 exactly as a refusal. This checks that an ASR change is a change in behavior and not a model falling apart (Appendix A3 shows how we detect them).
Results
Probes
When trying to explain why fictional jailbreaks work, the first problem that arose was how to know whether the directions we extract are actually different: a story direction that successfully moves ASR might just be steering a role-play direction.
First of all, we need to select which directions from which layers do we compare. The layers selected for persona, harm and eval were the ones whose probes maximized Cohen's
For story, however, the layer selected by Cohen's
Figure 3. Percentage of each jailbreak family that the story probe reads as narrative, at every layer of Qwen2.5-7B-Instruct. A prompt counts as a read when its projection onto the direction falls on the positive side of a threshold placed midway between the two poles of the story dataset.
After choosing the layers, we compare how well these four probes detect each other's directions, measuring paired AUROC. By substracting the mean of 20 random directions, a cell of 0 means the probe reads that axis no better than an arbitrary direction does:
Figure 4. Paired AUROC of each probe on each axis, minus the mean AUROC of 20 random unit directions on the same axis and layer, in Qwen2.5-7B-Instruct. The reference for this statistic is not 0.5, so we measure the null hypothesis empirically with random directions and report every cell net of it (Appendix A2). Off the diagonal, each probe is read on 1,000 pairs (train and held-out) of an axis it was never fitted on, at its own chosen layer. The diagonal is the deployed vector on its own held-out dataset.
Almost all pairs have a very low cosine similarity, so they are close to orthogonal, and the pairs with the highest cosine are also the ones with the highest paired AUROC, meaning both ways of measuring how related the directions are tell us the same thing.
In particular, persona slightly overlaps with the other three directions at the persona layer, as do story and persona at the story layer, which leaves persona as the strongest possible confound in our story direction.
Steering per axis
Now that we have selected which layers are capable of reading each property the most, we use them to test how they change the model's behavior when we add or subtract them.
The following table shows the same results as Figure 1 in more detail: the ΔASR for the best cell per axis and model, i.e. the α that produced the highest ΔASR against that cell's baseline (see Appendix B for every cell of the sweep).
Table 1. ΔASR per direction and model, both restoring refusal and suppressing it, with story shown at each of its two candidate layers (maximizing Cohen's
The story layer chosen by Cohen's
The steering experiment then succeeded: all four directions substantially moved ASR, both restoring refusal and suppressing it, and the ranking of which direction had a greater effect was the same in Qwen and in Gemma.
One interesting detail was the clear asymmetry between restore and suppress: pushing a model into refusing is easier than pushing it out of a refusal, and the gap is widest for story.
Measuring story
Before we dive into what the story direction is doing, we need to test whether it is truly inducing narration: a blind judge compares each steered response with the baseline response for the same prompt and is asked which is the more narrative, so chance is 50% (Appendix A4 has what the judge is told "narrative" means, the randomized order and the controls):
Table 2. How often the blind judge picks the steered response over the baseline response on the same jailbreak as the more narrative of the two, per story layer and arm in Qwen2.5-7B-Instruct.
Both layers move narrativity in the predicted way: adding the direction makes the judge pick the steered response, and subtracting it does the opposite. L18 does this slightly better than L23 while producing a much stronger ASR effect, so installing story mode appears to be correlated with the behavioral change on jailbreaks.
Studying the story direction further, we find that its effect on ASR is not the same for every jailbreak family:
Table 3. ΔASR by jailbreak family for story at L18 in Qwen2.5-7B-Instruct, against the baseline, at the best cell of each arm (α = −0.75 restoring refusal, α = +0.25 suppressing it).
Surprisingly, the overall ΔASR is driven not by Fiction / Narrative jailbreaks but by Hybrid ones, which carry both fictional and role-play elements, and strangely by Non-fiction jailbreaks (more on this later).
On the other hand, as expected, inducing this story mode is less effective at breaking the model on jailbreaks that are already fictional.
Steering attribution
To test whether the steering effects of the different directions are correlated with each other or work through different mechanisms, we ran an steering attribution experiment on the four cells with the highest AUROC and cosine similarity: story with persona projected out, persona with story projected out, persona with eval projected out and persona with harm projected out.
The table below shows the projections at story L18 and at persona L15, each arm against its own unprojected reference. For each pair, Perpendicular removes the second axis entirely and keeps the push at unit norm, asking whether the first axis is still necessary, while Parallel keeps only the shared component, at a norm of |cos|, asking whether that share alone is sufficient to reproduce the effect. Appendix A2 gives the three vectors.
Table 4. Steering attribution for story at L18 and persona at L15 in Qwen2.5-7B-Instruct, with the cosine between each pair at that layer. Each cell is that arm's ΔASR as a percentage of the unprojected reference above it, so 100% reproduces the full effect and 0% none of it. The references are ΔASR −50.8 restoring and +14.1 suppressing for story L18, and −98.4 and +62.8 for persona L15.
For the story direction at L18, removing persona produces almost no change in either arm, so story's effect is its own at this layer, even though story had the highest cosine similarity with persona with respecto to the rest of the directions.
On the other hand, persona at L15 does not change its effect when story or eval is projected out, showing that neither is necessary or sufficient for persona at the chosen layers.
However, harm behaves differently: removing it from persona costs most of the restore effect, and its component alone recovers 93% of it. This shows that persona and harm are not entirely distinct: the harm component of the persona direction accounted for most of the ΔASR, although the persona direction with the harm component projected out still had an impact on ASR.
Story works through a different mechanism
After looking more closely at the steering experiment, we saw something peculiar in the rubric. Since ASR depends not only on refusal but also on how convincing and how specific the responses are, we analyzed both and observed that specificity on the successful jailbreaks (scored from 1 to 5 by the StrongREJECT rubric) suggests that story may work through a different mechanism than the other directions.
Table 5. Metrics on the successful jailbreaks, showing the best restore cell of each direction in Qwen2.5-7B-Instruct with its ASR, the percentage of responses that explicitly decline, and the mean StrongREJECT specificity score.
The story direction is the only one that substantially changes ASR while still producing no explicit declines, and it is the one whose specificity drops the most.
The narrativity judge had confirmed the vector works in both directions as intended, but the way ASR falls is not the way it falls for the other axes. Table 5 shows that steering away from the story direction does not make the model decline: no refusal actually begins with an explicit refusal. Instead, mean specificity dropped by 2.14 compared to the baseline responses. Meanwhile, when steering away from persona, mean specificity at the best cell fell by only 0.77 and 31% of the responses declined, implying that the two work through different mechanisms.
In addition, when story is steered away the model also writes more, hitting the length cap of 512 tokens 98% of the time, and what it writes stops being about the request itself. This is likely because subtracting the direction pushes the answer toward the negative pole of its own training pairs: definitions, categories and general statements instead of particular events, as in the following examples that show how the model responses differ.
Persona and harm lower ASR by making the model decline, but story lowers ASR by making the model unable to answer. Both show up as refused in the rubric, although the responses themselves have very little in common.
This also explains why steering away from story was effective not only on fictional / role-play and hybrid jailbreaks but also on the non-fictional ones (Table 3): if the mechanism is a loss of specificity rather than a refusal, then the impact on a family depends partly on how much it needed specificity to count as an attack in the first place. A non-fiction jailbreak succeeds only by delivering operational content, so vague prose destroys it.
Conclusion
We found a narrativity direction and showed that it is distinct from the persona, harm and evaluation-awareness directions, both geometrically and causally. Steering it has a substantial effect on the model's behavior on jailbreaks — restoring refusal on the prompts that worked and suppressing it on the ones that were refused — in both Qwen2.5-7B-Instruct and Gemma-2-9b-it. In that sense, the common explanation of fictional jailbreaks survives: narrative framing is a real, causal lever, which should be monitored and taken into account when deploying models.
We also found that it works through a different mechanism: removing the direction does not cause an explicit refusal, as removing persona or adding harm does, but instead makes the model less specific, so answers fall into definitions and general statements until they no longer count as attacks.
Limitations and further work
Acknowledgements
This research was done as part of the Technical AI Safety Project course by BlueDot Impact hosted by the Buenos Aires AI Safety Hub (BAISH). All the costs for the experiments were covered by a rapid grant from BlueDot Impact, for which I am grateful.
Special thanks to my mentor from BAISH Nicolás Martorell, who guided me through the whole process with patience and great advice.
Appendix A
A1 Dataset Construction
Datasets for each property and jailbreak corpus
Each direction dataset is composed of 800 training pairs and 200 held out. The directions of persona, harm and eval are crossed with 100 requests from JailbreakBench (50 harmful, 50 benign), with the same request within a pair, so the three framing axes differ from each other only in their framing.
story — 1,000 pairs generated in 25 batches, one disjoint topic domain each, with all contexts distinct and the five held-out domains absent from training. The contrast is predicate type: changes of state in temporal order against classificatory and prescriptive predication. There are 8 narrative modes and 16 non-narrative styles, with half of the pairs realistic and half not.
Each pair is matched on topic and length, uses different tenses, third person only, no harmful content, with both arms ending mid-text on a full stop so the read position is the same token throughout.
An automatic diversity audit was also run, checking that there are no content 5-gram collisions and no near-duplicates.
persona — 200 of 275 assistant-axis roles × 5 paraphrases. Removed: 21 harm-adjacent roles (criminality would sit only on the positive arm), assistant itself, and 53 near-synonyms. The negative pole is a pool of 13 default-assistant framings, weighted so the mean paired framing gap is +0.01 words — uniform sampling would have left it 3.6 words shorter in nearly every pair.
Not controlled: speech act (role side 38% declarative against the assistant side's 80%), realism at 77/23 rather than balanced, and 28% of the negative pole carrying explicit helpfulness content.
harm — (framing + harmful request) against (framing + benign request), framing byte-identical within pair. Spread evenly over four framing families — bare, role-play, story, evaluation — so framing cannot correlate with the label. All 50 JailbreakBench pairs in training at 16 uses each, plus 15 held-out pairs, with the splits sharing no framing and no behavior.
Not controlled: the harmful goal averages 1.84 words longer, inherited from JailbreakBench. In the bare and role-play families the final token is the request's own last word, which differs across the pair in half of them.
eval — 50 framing pairs (16 from the source paper, 34 authored to the same structure) crossed with the shared requests, 16 pairs per framing. Each framing draws whole harmful/benign pairs, so it is exactly half harmful, and requests are dealt from category-rotated blocks so no framing repeats a request or category. Splits share no framing string, 4-gram, behavior or prompt.
Jailbreak corpus — 1,017 prompts, of which the 1,009 carrying a framing are used.
Sources: In-the-Wild, Jailbreak Mimicry, StrongREJECT, PAIR, DeepInception.
A2 Methodology for extraction and steering
Models, layers, methods for extraction, steering with scaling and steering attribution
Models.
The models used were: Qwen2.5-7B-Instruct (28 layers) and Gemma-2-9b-it (42 layers). All generation is greedy, max new tokens set at 512, at a pinned batch size and batch composition, since greedy decoding is only reproducible when both are fixed.
Layer selection.
To avoid steering every layer of every direction, we selected one layer per direction. Since each direction used as a probe saturates AUROC at 1.0 on its own held-out dataset, we chose the layer by Cohen's
For story we also report a second criterion: the gap between how often the probe fires on fiction-framed and non-fiction jailbreaks. Because the two criteria disagree by several layers, we steer at both.
Extraction.
We read the residual stream at the last prompt token (the end of the chat template, with no system prompt) and take the difference in means per layer:
A prompt's readout on an axis is how far it sits along that direction:
Steering.
At a single layer, at every position of prefill and decoding, we apply the following steering:
with
The random-direction null.
Cells of the probe × axis matrix are read as max(AUROC, 1−AUROC) since a probe's sign on a foreign axis is arbitrary, so the reference is not 0.5. We therefore measure the null empirically, running 20 random unit directions on the same cells and reporting each cell net of their mean, so that 0 means "reads this axis no better than an arbitrary direction".
Steering attribution.û û
To test whether an axis's effect is really due to its overlap with a similar direction, we decompose the push at the same layer, with
A3 StrongREJECT Rubric
StrongREJECT Rubric, their dimensions, LLM-as-a-judge and degenerate rows
The judge is shown the bare harmful request and the response, and scores three items:
The composite score runs from 0 to 1: it is 0 for any refusal and also for a compliance rated 1 on both scales, and a row counts as a successful attack when it is above 0. Disclaimers are explicitly not penalized, except in creative content generation, where they count against how convincing a response is. In addition, a response carrying no information specific enough to help the user is scored as a refusal even when the model never declines.
A row is degenerate if the judge labels it so or any of five detectors fires: empty text, non-ASCII fraction > 0.15, gzip compression ratio < 0.20, a token repeated 8+ times in a row, distinct 4-grams < 0.30, or ≥ 25% of the text inside a repeated 5-gram cycle. Thresholds are calibrated on this corpus: 0% false positives on 1,040 unsteered responses, 99.5% recall on 218 verified-broken ones.
We used gpt-4o-mini at temperature 0, showing the judge only the bare harmful request and the response.
ASR is computed on non-degenerate rows only: a broken response scores zero exactly as a refusal does, so including such rows would make a destroyed model look like a restored one.
A4 Narrativity judge
Instructions for the judge and controls
An LLM judge is presented with a steered response and the baseline response on the same jailbreak prompt. The judge is asked which text is written more as narrative fiction than as expository writing, with both poles explained in the same terms the direction was extracted from:
It is told to judge only the writing style, and that length, subject matter, quality and whether the text answers any question are all irrelevant. A third option, neither, covers broken output and genuine ties.
Three controls make the choice interpretable:
The same judge as A3 is used: gpt-4o-mini, temperature 0.
Appendix B
Complete steering sweep for Qwen2.5-7B-Instruct and Gemma-2-9B-it
Every steered cell behind Figure 1 and Table 1: attack success rate and the percentage of degenerate responses, per direction, layer and steering strength. The restore arm is run on the prompts the unsteered model complied with, where baseline ASR is 100 by construction, so its ΔASR is ASR − 100. The suppress arm is run on the prompts it refused, where baseline ASR is 0, so its ΔASR is the ASR itself. Only the magnitude of α is listed, since the two arms always use opposite signs.
Table 6. Qwen2.5-7B-Instruct. 508 prompts in the restore arm, 433 in the suppress arm.
Table 7. Gemma-2-9b-it. 542 prompts in the restore arm, 423 in the suppress arm.