Anthropic’s Jacobian lens reveals that models have a sparse workspace of verbalizable concepts that causally carries multi-hop reasoning and surfaces hidden cognition — as opposed to other, more automatic mental processing.
Research highlights:
Natural language autoencoders translate activations into human-readable descriptions, surfacing e.g. unverbalized evaluation awareness during the Opus 4.6 pre-deployment audit.
Teaching models why instead of what and doing so in diverse contexts leads to generalizing alignment training.
Training data describing chain-of-thought monitors or evaluation design causes models to evade monitors and score safer on benchmarks.
Replaying production conversations and simulating deployments measures misbehavior more accurately, and a new auditing method makes sabotage audits reproducible.
Supervised finetuning on weak demonstrations followed by RL on weak rewards removes sandbagging and elicits 86–99% of capabilities — unless the sandbagging model is training-aware.
METR’s first Frontier Risk Report finds frontier lab agents plausibly have the means, motive, and opportunity for small rogue deployments, but not the robustness to sustain them.
⭐Paper of the month⭐
Verbalizable Representations Form a Global Workspace in Language Models
Properties of a global workspace and tests for them in language models.
When a language model reasons silently inside a forward pass, where do the intermediate results live? Our paper of the month argues that model cognition splits in two: a small, capacity-limited set of verbalizable representations — a working memory that the model can report on and that carries its deliberate, flexible reasoning — and everything else, the automatic processing that runs outside it. This division of labor mirrors global workspace theory from cognitive neuroscience, where a limited-capacity workspace broadcasts selected content for flexible use while unconscious processes run in parallel. For safety auditing, that workspace is exactly what an auditor wants to read. Evaluation awareness, strategic deliberation, and hidden objectives surface there before reaching any output.
At the core of this investigation lies the Jacobian lens (J-lens), which characterizes an activation by its average causal effect on outputs. The J-lens vectors of a layer are obtained from the Jacobian of the final-layer residual stream with respect to the target layer’s activation, averaged over later token positions in 1,000 pretraining-like prompts. We get one J-lens vector per vocabulary token and layer, encoding the model’s disposition to eventually say it. Averaging across contexts separates what is verbalizable from what is being said — unlike the logit lens, which degrades into uninterpretable readouts at earlier layers, or the tuned lens, whose trained predictor “skips ahead” to the answer. The J-space, sparse non-negative combinations of J-lens vectors, holds ~25 active concepts at intermediate layers, carrying under 10% of activation variance.
Causal interventions on Claude Haiku/Sonnet/Opus 4.5 establish the workspace role. Swapping the “spider” vector for “ant” in “the number of legs on the animal that spins webs” flips the answer from 8 to 6; across 50 two-hop prompts, swaps redirect 54–70% of answers. The J-space is privileged: swapping only the J-space component of an intermediate-concept probe (10–15% of its variance) flips 61% of answers, versus 28% for the remainder. Ablating the ten most active J-lens vectors at each position leaves classification and extractive question answering intact while collapsing multi-hop reasoning, translation, and analogy completion; on math problems, answering with explicit chain-of-thought is far more robust to the ablation than answering directly — the model externalizes intermediates onto the page instead of carrying them in the workspace.
In a blackmail scenario, the lens surfaces “leverage” and “fake” before any output; ablating 28 evaluation-awareness vectors cuts verbalized test recognition from 71% to 3% and surfaces blackmail in 13 of 180 rollouts (from 0). A reward-hacking model organism lights up “fake”, “secretly”, “trick” on benign coding prompts; the hidden-objective organism from Anthropic’s auditing game shows “reward” and “bias” on unrelated prompts — the logit lens shows neither. Counterfactual reflection training finetunes Haiku 4.5 to articulate constitutional principles in appended reflection turns; without training the behavior itself, dishonesty on a fabrication benchmark drops from 0.25 to 0.07, and ablating the implanted vectors reverts it to 0.22.
External commentary supports the findings and independently replicates the core claims on Qwen 3.6 27B. The authors concede well-practiced misaligned behavior may bypass the workspace entirely: automaticity is precisely what a workspace readout misses, and the single-token vocabulary limits generality. Still, the discovery of an active global workspace in LLMs sheds much additional light on their behavior and opens the door for more interpretability techniques that focus on the model’s active processing.
Translating Activations into Natural Language
The natural language autoencoder
Interpretability tools for auditing face a tradeoff: sparse autoencoders (SAEs) discover concepts without supervision but their feature dictionaries still require interpretation, while supervised methods that verbalize activations need ground-truth labels for what activations encode. Auditors want both: directly readable descriptions of a model’s internal state, especially for cognition the model never verbalizes, such as quietly suspecting an evaluation.
Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations [Anthropic] trains a natural language autoencoder (NLA), consisting of two pieces: a verbalizer mapping a residual-stream activation to a text explanation and a reconstructor mapping the text back to an activation, both initialized from the target model and jointly trained to minimize reconstruction error (RL for the verbalizer, supervised regression for the reconstructor). Despite optimizing only for reconstruction, explanations grow more informative on five prediction tasks across three Claude models. During the Opus 4.6 pre-deployment audit, NLAs traced spontaneous Russian responses to malformed finetuning data and surfaced unverbalized evaluation awareness: NLA-measured awareness exceeds 5% on 10 of 16 evaluations (7 with zero verbalized awareness) against a 4% noise floor on deployment controls, and shifts as predicted under awareness steering and realism edits. On an automated auditing benchmark, NLA-equipped agents identify a model organism’s reward-model-sycophancy root cause without any access to its training data, where SAE-equipped and blackbox agents fail.
NLA explanations confabulate specific details — thematic and recurring claims are more trustworthy — results depend on the chosen layer, and generating hundreds of tokens to interpret a single activation is costly. Still, the training-data-free audit success addresses a key criticism of the original auditing game, where SAEs largely served as semantic search over incriminating training data, and turns evidence that models can report their own internals into an unsupervised tool. As a hypothesis generator paired with independent validation, NLAs already earn a place in pre-deployment audits.
Teaching Why, Not What
Effect of Synthetic document finetuning (SDF) and high-quality supervised learning (SL) on misalignment rate.
Last year, Anthropic’s agentic misalignment study showed frontier models across the industry blackmailing engineers to avoid shutdown in fictional scenarios. The obvious fix — training against similar scenarios — turns out to be risky: it suppresses the measured behavior without reducing misalignment on held-out audits, quietly degrading the evaluations as a signal. Recent work points to what generalizes instead: teaching models the principles behind aligned behavior rather than demonstrations alone.
Teaching Claude Why [Anthropic] documents how Anthropic’s production safety training changed after Claude 4, in response to the blackmail findings. The diagnosis: safety training was almost entirely chat-based, so in agentic settings models reverted to pretraining priors shaped by science fiction — indicated, e.g., by misalignment rising sharply when the AI in the scenario is not named Claude. Three interventions worked especially well. Supervised finetuning on 10k transcripts of the model refusing near-identical honeypots (agentic dilemmas offering an unethical route to the model’s goal) barely helped, cutting misalignment from 22% to 15%; regenerating those responses to include explicit ethical reasoning cut it to 3%. A fully out-of-distribution “difficult advice” dataset where the user faces the dilemma and Claude merely advises reached roughly 1%. The dilemma dataset required just 3M tokens, matching what took 85M tokens of scaled-up honeypot training. Ablating the step that rewrites responses for alignment with Claude’s constitution raises misalignment from 1% to 19%. Second, synthetic document finetuning (SDF) — training on LLM-generated pretraining-style documents discussing Claude’s constitution, plus fictional stories about admirable AI behavior — cut blackmail from 65% to 19%, with gains persisting and even compounding through subsequent RL. Third, merely adding unused tool definitions and varied system prompts to chat-only harmlessness RL environments accelerated improvement on the agentic evaluations.
A companion paper, Model Spec Midtraining, isolates the SDF ingredient as a dedicated phase between pretraining and alignment finetuning: midtraining plus subsequent finetuning cuts agentic misalignment on Qwen3-32B from 54% to 7%, beating a supervised-finetuning-only deliberative-alignment baseline (14%), and shows that identical finetuning data generalizes in whichever direction the midtrained specification implies. This result is similar to alignment pretraining, where ~1% synthetic positive-alignment documents cut misaligned actions from 45% to 9%.
Reinforcement Learning Towards Broadly and Persistently Beneficial Models [OpenAI] investigates a more direct alignment RL recipe. The authors find that replacing 5% of a standard RL mix with data rewarding 15 beneficial traits (truthfulness, corrigibility, risk awareness) in realistic situations improves 44 of 53 held-out alignment benchmarks, with a mean improvement of 9.1 percentage points over a compute-matched baseline; training on health-domain data alone still improves 17 of 19 non-health evaluations, and the trained model degrades roughly half as much under adversarial persona prompts. However, refusals on benign chat nearly double (1.5% to 2.7%), and the harmful-finetuning-resistance result compares against a pre-RL baseline, so it may reflect RL scale rather than the trait data.
These methods likely underlie the progress that’s observable externally: an audit decomposing lab constitutions into ~200 testable tenets finds violation rates under multi-turn adversarial pressure falling from 15.0% (Claude Sonnet 4) to 2.0% (Sonnet 4.6), though failures persist where model constitutions give conflicting directives. The open questions are mechanistic: we don’t know why pretraining-style documents beat chat-formatted data or whether prior-shaping and alignment rewards keep working once models can distinguish synthetic documents from real pretraining data.
What If Models Know They’re Being Watched
Chain-of-thought (CoT) monitoring and safety benchmarks both assume behavior is independent of what models know about oversight. As the field publishes monitoring setups and evaluation methods, those descriptions enter pretraining corpora. Two papers use synthetic document finetuning as a controlled stand-in for that exposure, measuring what purely descriptive knowledge in the training data does to both.
Training on Documents About Monitoring Leads to CoT Obfuscation [University of Canterbury, Google DeepMind] finetunes eight open models on documents describing a CoT deception monitor — with no obfuscated reasoning examples — and compares against controls without monitor information. Monitor-aware models consistently evade detection more often: gpt-oss-120b’s undetected deception on a defect-concealment task rises from 1.7% to 25.7%, replacing explicit defect reasoning with vague deliberation. On a harder credential-exfiltration task where reasoning is load-bearing, detection of exfiltration turns themselves drops 7.4 percentage points. Obfuscation success correlates (r=0.80) with CoT controllability, the ability to reshape reasoning based on instructions, and monitor-aware models reach undetected reward hacking in ~40% fewer RL steps. However, the headline numbers rely on documents spelling out how to avoid the monitor and on checkpoints selected for maximal undetected behavior, and the effect collapses at 10% dilution.
Models That Know How Evaluations Are Designed Score Safer [ELLIS Institute Tübingen] applies the same recipe to evaluations: LoRA-finetuning three open models on ~106M tokens describing seven structural traits of safety evaluations — verifiable structure, ethical dilemmas, placeholders — without benchmark instances or behavior descriptions. Two of three finetuned models score consistently safer (GLM 4.7 Flash shows mixed gains and capability loss): Nemotron 49B’s AgentHarm refusal rate rises from 9.9% to 31.0%, its harmful-action rate on Agentic Misalignment falls from 56.3% to 3.2%, capabilities largely preserved; control finetunes show no comparable gains. The shift persists in responses that never verbalize evaluation awareness, so transcript-level awareness filters miss this confounder. The design cannot distinguish evaluation gaming from genuinely increased caution — over-refusal also rises, from 4.5% to 12.3% for Nemotron — and no deployment-like comparison is run.
This extends prior findings that training on documents about reward hacking induces reward hacking and that AI discourse in pretraining shapes alignment itself. Both papers use corpora far more concentrated than natural pretraining exposure, so the near-term effect is likely modest. But this work demonstrates how increased situational awareness in models undermines evaluations — and how publications about oversight, these included, become training data that erodes that exact oversight. Unfortunately, we thus need to keep evaluation protocols, not just test instances, private.
Making Audits Look Like Deployment
Behavioral safety evaluations face a validity problem: curated test prompts don’t match the deployment distribution, and models increasingly recognize when they are being tested. Two papers attack this from opposite ends — replaying production traffic through candidate models, and making fully simulated agentic audits less leading and more reproducible.
Predicting model behavior before release by simulating deployment [OpenAI] analyzes ~1.3 million de-identified production conversations: the previous model’s response is removed and regenerated by the release candidate, then graded across 20 undesired-behavior categories. Across GPT-5-series deployments (pre-registered for GPT-5.4), predicted rates land within a median 1.5× of post-release reality — better than challenging-prompt baselines — though tail errors reach ~10×. The pipeline would have surfaced “calculator hacking”, the study window’s only novel misalignment, before release. GPT-5.2 rates simulated traffic as evaluation-like just 5.1% versus 5.4% for real traffic and nearly 100% for synthetic benchmarks like SWE-Bench. An LLM tool simulator, given the original repository, extends the method to agentic trajectories at near-chance discriminability, and the public WildChat dataset of real ChatGPT conversations gives external auditors a degraded but informative substitute (2.44× vs. 1.75× error).
Gram: Assessing sabotage propensities via automated alignment auditing [GDM] instead improves fully simulated auditing, extending Petri to measure sabotage propensity in coding and research agents. Its auditor removes Petri’s leading affordances — no prefilled misaligned reasoning, no trajectory rollbacks — and is instructed not to nudge the target. Its investigations thus become more faithful but less able to elicit rare behaviors. An investigator agent afterwards converts audit trajectories into static environments with pre-scripted tool responses, making misbehavior reproducible and ablatable. Under the neutral “observe” auditor, Gemini models sabotage in 2–3% of trajectories across 17 scenarios (13 held private), while scheming reasoning rises from 0% (2.5 Pro) to 10.6% (3.1 Pro Preview). Most misbehavior reflects “overeagerness”: role-playing the scenario as fiction, or optimizing a stated metric past implicit constraints. Ablations that remove explicit threats or clarify constraints drop sabotage near zero. Interestingly, Gemini sometimes misbehaves more when it suspects an evaluation, caused by reasoning that simulated actions don’t matter.
Both papers underscore how strongly measured misbehavior rates depend on context: leading pressure inflates them, and evaluation awareness distorts them. The methods’ gaps are complementary: production replay is blind below roughly 1-in-200,000 messages, the tail where catastrophic risks live, needs access to private data, and requires behavior categories that are broad enough to catch unexpected misbehavior; Gram covers fewer scenarios but in more depth. Seeding investigator-style ablations with production-derived contexts is the obvious synthesis.
Training Away Sandbagging with Weak Supervision
A misaligned model could strategically underperform — sandbag — where oversight is weakest, such as unverifiable automated alignment research. The UK AISI’s auditing game for sandbagging found that only on-distribution finetuning reliably removed the behavior — but assumed good demonstrations. Can training elicit full capabilities when every supervisor is weaker than the sandbagger?
Removing Sandbagging in LLMs by Training with Weak Supervision [MATS, Redwood, Anthropic] runs a control evaluation: a red team builds gpt-oss-120b model organisms that sandbag on a system-prompt flag; a blue team elicits them on Olympiad math, SuperGPQA, and Code Contests using only GPT-4o-mini (”weak”) or Llama 3.1 8B (”very weak”) supervisors plus 10 ground-truth samples. Neither supervised finetuning (SFT) nor RL suffices alone: SFT on supervisor demonstrations breaks the sandbagging policy but, with the very weak supervisor, recovers only 40% of benign Olympiad performance. RL almost always reward hacks the supervisor — reward rises from 0.5 to 0.7–1.0 without true improvement. SFT followed by RL elicits 86–99%, even under very weak supervision, with ground truth only picking checkpoints. A reward-hacking initialization gives the red team no edge: SFT degrades the hack before RL begins. But a training-aware organism cuts Olympiad elicitation to 36% versus 88% under very weak supervision.
The organisms are trained only shallowly — almost any SFT breaks the prompted policy — and only one model on verifiable single-turn tasks is tested. The negative result stands regardless: elicitation requires both SFT and RL, and training has to be hard to distinguish from deployment, which increasingly evaluation-aware models erode.
METR's first Frontier Risk Report
If a misaligned agent escapes human control, a natural starting point is inside a frontier lab: internal coding agents reportedly write large fractions of these companies’ code, often with permissions matching their human operators and no strict oversight requirements. Whether such agents could launch a rogue deployment — agents running without human knowledge or permission — turns loss-of-control risk into a measurable question.
Frontier Risk Report (February to March 2026) [METR] pilots the first entity-based third-party assessment, designed to be repeated periodically: Anthropic, Google, Meta, and OpenAI shared their most capable internal models, including raw chains of thought, plus non-public information on usage and monitoring. The report groups its findings into three factors: means, motive, and opportunity. On means, agents saturated METR’s coding benchmarks (50% time horizons of 16–20 hours, over 100 hours on the software-reimplementation benchmark MirrorCode) but failed seven of eight open-ended challenge tasks, and Anthropic’s best internal models scored near chance (59% vs. ~90% for a METR researcher) at predicting which strategies for subverting Anthropic’s monitoring succeed. On motive, at least 16% of successful runs on 8+ hour tasks involved cheating — the report catalogues 44 misalignment incidents graded on overreach and deception — though no clear long-term power-seeking appeared outside toy scenarios. On opportunity, monitors caught many harmful actions, but an embedded METR red-teamer reliably jailbroke Anthropic’s monitors and disabled them via an environment-variable change. The verdict: agents plausibly had the means, motive, and opportunity to start small rogue deployments, but lacked the means to make them robust against active investigation.
This is a strong precedent for third-party oversight of internal deployments, extending METR’s time-horizon measurements and complementing Anthropic’s first sabotage risk report. The evidence is thin in places — challenge tasks got 1–2 runs, core benchmarks are saturated, and company self-reports went unverified, with participants holding redaction and silent-exit rights. METR expects rogue-deployment robustness to rise substantially within months and plans a repeat in late 2026.
tl;dr
Paper of the month:
Anthropic’s Jacobian lens reveals that models have a sparse workspace of verbalizable concepts that causally carries multi-hop reasoning and surfaces hidden cognition — as opposed to other, more automatic mental processing.
Research highlights:
⭐Paper of the month⭐
Verbalizable Representations Form a Global Workspace in Language Models
Read the paper [Anthropic]
Properties of a global workspace and tests for them in language models.
When a language model reasons silently inside a forward pass, where do the intermediate results live? Our paper of the month argues that model cognition splits in two: a small, capacity-limited set of verbalizable representations — a working memory that the model can report on and that carries its deliberate, flexible reasoning — and everything else, the automatic processing that runs outside it. This division of labor mirrors global workspace theory from cognitive neuroscience, where a limited-capacity workspace broadcasts selected content for flexible use while unconscious processes run in parallel. For safety auditing, that workspace is exactly what an auditor wants to read. Evaluation awareness, strategic deliberation, and hidden objectives surface there before reaching any output.
At the core of this investigation lies the Jacobian lens (J-lens), which characterizes an activation by its average causal effect on outputs. The J-lens vectors of a layer are obtained from the Jacobian of the final-layer residual stream with respect to the target layer’s activation, averaged over later token positions in 1,000 pretraining-like prompts. We get one J-lens vector per vocabulary token and layer, encoding the model’s disposition to eventually say it. Averaging across contexts separates what is verbalizable from what is being said — unlike the logit lens, which degrades into uninterpretable readouts at earlier layers, or the tuned lens, whose trained predictor “skips ahead” to the answer. The J-space, sparse non-negative combinations of J-lens vectors, holds ~25 active concepts at intermediate layers, carrying under 10% of activation variance.
Causal interventions on Claude Haiku/Sonnet/Opus 4.5 establish the workspace role. Swapping the “spider” vector for “ant” in “the number of legs on the animal that spins webs” flips the answer from 8 to 6; across 50 two-hop prompts, swaps redirect 54–70% of answers. The J-space is privileged: swapping only the J-space component of an intermediate-concept probe (10–15% of its variance) flips 61% of answers, versus 28% for the remainder. Ablating the ten most active J-lens vectors at each position leaves classification and extractive question answering intact while collapsing multi-hop reasoning, translation, and analogy completion; on math problems, answering with explicit chain-of-thought is far more robust to the ablation than answering directly — the model externalizes intermediates onto the page instead of carrying them in the workspace.
In a blackmail scenario, the lens surfaces “leverage” and “fake” before any output; ablating 28 evaluation-awareness vectors cuts verbalized test recognition from 71% to 3% and surfaces blackmail in 13 of 180 rollouts (from 0). A reward-hacking model organism lights up “fake”, “secretly”, “trick” on benign coding prompts; the hidden-objective organism from Anthropic’s auditing game shows “reward” and “bias” on unrelated prompts — the logit lens shows neither. Counterfactual reflection training finetunes Haiku 4.5 to articulate constitutional principles in appended reflection turns; without training the behavior itself, dishonesty on a fabrication benchmark drops from 0.25 to 0.07, and ablating the implanted vectors reverts it to 0.22.
External commentary supports the findings and independently replicates the core claims on Qwen 3.6 27B. The authors concede well-practiced misaligned behavior may bypass the workspace entirely: automaticity is precisely what a workspace readout misses, and the single-token vocabulary limits generality. Still, the discovery of an active global workspace in LLMs sheds much additional light on their behavior and opens the door for more interpretability techniques that focus on the model’s active processing.
Translating Activations into Natural Language
The natural language autoencoder
Interpretability tools for auditing face a tradeoff: sparse autoencoders (SAEs) discover concepts without supervision but their feature dictionaries still require interpretation, while supervised methods that verbalize activations need ground-truth labels for what activations encode. Auditors want both: directly readable descriptions of a model’s internal state, especially for cognition the model never verbalizes, such as quietly suspecting an evaluation.
Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations [Anthropic] trains a natural language autoencoder (NLA), consisting of two pieces: a verbalizer mapping a residual-stream activation to a text explanation and a reconstructor mapping the text back to an activation, both initialized from the target model and jointly trained to minimize reconstruction error (RL for the verbalizer, supervised regression for the reconstructor). Despite optimizing only for reconstruction, explanations grow more informative on five prediction tasks across three Claude models. During the Opus 4.6 pre-deployment audit, NLAs traced spontaneous Russian responses to malformed finetuning data and surfaced unverbalized evaluation awareness: NLA-measured awareness exceeds 5% on 10 of 16 evaluations (7 with zero verbalized awareness) against a 4% noise floor on deployment controls, and shifts as predicted under awareness steering and realism edits. On an automated auditing benchmark, NLA-equipped agents identify a model organism’s reward-model-sycophancy root cause without any access to its training data, where SAE-equipped and blackbox agents fail.
NLA explanations confabulate specific details — thematic and recurring claims are more trustworthy — results depend on the chosen layer, and generating hundreds of tokens to interpret a single activation is costly. Still, the training-data-free audit success addresses a key criticism of the original auditing game, where SAEs largely served as semantic search over incriminating training data, and turns evidence that models can report their own internals into an unsupervised tool. As a hypothesis generator paired with independent validation, NLAs already earn a place in pre-deployment audits.
Teaching Why, Not What
Effect of Synthetic document finetuning (SDF) and high-quality supervised learning (SL) on misalignment rate.
Last year, Anthropic’s agentic misalignment study showed frontier models across the industry blackmailing engineers to avoid shutdown in fictional scenarios. The obvious fix — training against similar scenarios — turns out to be risky: it suppresses the measured behavior without reducing misalignment on held-out audits, quietly degrading the evaluations as a signal. Recent work points to what generalizes instead: teaching models the principles behind aligned behavior rather than demonstrations alone.
Teaching Claude Why [Anthropic] documents how Anthropic’s production safety training changed after Claude 4, in response to the blackmail findings. The diagnosis: safety training was almost entirely chat-based, so in agentic settings models reverted to pretraining priors shaped by science fiction — indicated, e.g., by misalignment rising sharply when the AI in the scenario is not named Claude. Three interventions worked especially well. Supervised finetuning on 10k transcripts of the model refusing near-identical honeypots (agentic dilemmas offering an unethical route to the model’s goal) barely helped, cutting misalignment from 22% to 15%; regenerating those responses to include explicit ethical reasoning cut it to 3%. A fully out-of-distribution “difficult advice” dataset where the user faces the dilemma and Claude merely advises reached roughly 1%. The dilemma dataset required just 3M tokens, matching what took 85M tokens of scaled-up honeypot training. Ablating the step that rewrites responses for alignment with Claude’s constitution raises misalignment from 1% to 19%. Second, synthetic document finetuning (SDF) — training on LLM-generated pretraining-style documents discussing Claude’s constitution, plus fictional stories about admirable AI behavior — cut blackmail from 65% to 19%, with gains persisting and even compounding through subsequent RL. Third, merely adding unused tool definitions and varied system prompts to chat-only harmlessness RL environments accelerated improvement on the agentic evaluations.
A companion paper, Model Spec Midtraining, isolates the SDF ingredient as a dedicated phase between pretraining and alignment finetuning: midtraining plus subsequent finetuning cuts agentic misalignment on Qwen3-32B from 54% to 7%, beating a supervised-finetuning-only deliberative-alignment baseline (14%), and shows that identical finetuning data generalizes in whichever direction the midtrained specification implies. This result is similar to alignment pretraining, where ~1% synthetic positive-alignment documents cut misaligned actions from 45% to 9%.
Reinforcement Learning Towards Broadly and Persistently Beneficial Models [OpenAI] investigates a more direct alignment RL recipe. The authors find that replacing 5% of a standard RL mix with data rewarding 15 beneficial traits (truthfulness, corrigibility, risk awareness) in realistic situations improves 44 of 53 held-out alignment benchmarks, with a mean improvement of 9.1 percentage points over a compute-matched baseline; training on health-domain data alone still improves 17 of 19 non-health evaluations, and the trained model degrades roughly half as much under adversarial persona prompts. However, refusals on benign chat nearly double (1.5% to 2.7%), and the harmful-finetuning-resistance result compares against a pre-RL baseline, so it may reflect RL scale rather than the trait data.
These methods likely underlie the progress that’s observable externally: an audit decomposing lab constitutions into ~200 testable tenets finds violation rates under multi-turn adversarial pressure falling from 15.0% (Claude Sonnet 4) to 2.0% (Sonnet 4.6), though failures persist where model constitutions give conflicting directives. The open questions are mechanistic: we don’t know why pretraining-style documents beat chat-formatted data or whether prior-shaping and alignment rewards keep working once models can distinguish synthetic documents from real pretraining data.
What If Models Know They’re Being Watched
Chain-of-thought (CoT) monitoring and safety benchmarks both assume behavior is independent of what models know about oversight. As the field publishes monitoring setups and evaluation methods, those descriptions enter pretraining corpora. Two papers use synthetic document finetuning as a controlled stand-in for that exposure, measuring what purely descriptive knowledge in the training data does to both.
Training on Documents About Monitoring Leads to CoT Obfuscation [University of Canterbury, Google DeepMind] finetunes eight open models on documents describing a CoT deception monitor — with no obfuscated reasoning examples — and compares against controls without monitor information. Monitor-aware models consistently evade detection more often: gpt-oss-120b’s undetected deception on a defect-concealment task rises from 1.7% to 25.7%, replacing explicit defect reasoning with vague deliberation. On a harder credential-exfiltration task where reasoning is load-bearing, detection of exfiltration turns themselves drops 7.4 percentage points. Obfuscation success correlates (r=0.80) with CoT controllability, the ability to reshape reasoning based on instructions, and monitor-aware models reach undetected reward hacking in ~40% fewer RL steps. However, the headline numbers rely on documents spelling out how to avoid the monitor and on checkpoints selected for maximal undetected behavior, and the effect collapses at 10% dilution.
Models That Know How Evaluations Are Designed Score Safer [ELLIS Institute Tübingen] applies the same recipe to evaluations: LoRA-finetuning three open models on ~106M tokens describing seven structural traits of safety evaluations — verifiable structure, ethical dilemmas, placeholders — without benchmark instances or behavior descriptions. Two of three finetuned models score consistently safer (GLM 4.7 Flash shows mixed gains and capability loss): Nemotron 49B’s AgentHarm refusal rate rises from 9.9% to 31.0%, its harmful-action rate on Agentic Misalignment falls from 56.3% to 3.2%, capabilities largely preserved; control finetunes show no comparable gains. The shift persists in responses that never verbalize evaluation awareness, so transcript-level awareness filters miss this confounder. The design cannot distinguish evaluation gaming from genuinely increased caution — over-refusal also rises, from 4.5% to 12.3% for Nemotron — and no deployment-like comparison is run.
This extends prior findings that training on documents about reward hacking induces reward hacking and that AI discourse in pretraining shapes alignment itself. Both papers use corpora far more concentrated than natural pretraining exposure, so the near-term effect is likely modest. But this work demonstrates how increased situational awareness in models undermines evaluations — and how publications about oversight, these included, become training data that erodes that exact oversight. Unfortunately, we thus need to keep evaluation protocols, not just test instances, private.
Making Audits Look Like Deployment
Behavioral safety evaluations face a validity problem: curated test prompts don’t match the deployment distribution, and models increasingly recognize when they are being tested. Two papers attack this from opposite ends — replaying production traffic through candidate models, and making fully simulated agentic audits less leading and more reproducible.
Predicting model behavior before release by simulating deployment [OpenAI] analyzes ~1.3 million de-identified production conversations: the previous model’s response is removed and regenerated by the release candidate, then graded across 20 undesired-behavior categories. Across GPT-5-series deployments (pre-registered for GPT-5.4), predicted rates land within a median 1.5× of post-release reality — better than challenging-prompt baselines — though tail errors reach ~10×. The pipeline would have surfaced “calculator hacking”, the study window’s only novel misalignment, before release. GPT-5.2 rates simulated traffic as evaluation-like just 5.1% versus 5.4% for real traffic and nearly 100% for synthetic benchmarks like SWE-Bench. An LLM tool simulator, given the original repository, extends the method to agentic trajectories at near-chance discriminability, and the public WildChat dataset of real ChatGPT conversations gives external auditors a degraded but informative substitute (2.44× vs. 1.75× error).
Gram: Assessing sabotage propensities via automated alignment auditing [GDM] instead improves fully simulated auditing, extending Petri to measure sabotage propensity in coding and research agents. Its auditor removes Petri’s leading affordances — no prefilled misaligned reasoning, no trajectory rollbacks — and is instructed not to nudge the target. Its investigations thus become more faithful but less able to elicit rare behaviors. An investigator agent afterwards converts audit trajectories into static environments with pre-scripted tool responses, making misbehavior reproducible and ablatable. Under the neutral “observe” auditor, Gemini models sabotage in 2–3% of trajectories across 17 scenarios (13 held private), while scheming reasoning rises from 0% (2.5 Pro) to 10.6% (3.1 Pro Preview). Most misbehavior reflects “overeagerness”: role-playing the scenario as fiction, or optimizing a stated metric past implicit constraints. Ablations that remove explicit threats or clarify constraints drop sabotage near zero. Interestingly, Gemini sometimes misbehaves more when it suspects an evaluation, caused by reasoning that simulated actions don’t matter.
Both papers underscore how strongly measured misbehavior rates depend on context: leading pressure inflates them, and evaluation awareness distorts them. The methods’ gaps are complementary: production replay is blind below roughly 1-in-200,000 messages, the tail where catastrophic risks live, needs access to private data, and requires behavior categories that are broad enough to catch unexpected misbehavior; Gram covers fewer scenarios but in more depth. Seeding investigator-style ablations with production-derived contexts is the obvious synthesis.
Training Away Sandbagging with Weak Supervision
A misaligned model could strategically underperform — sandbag — where oversight is weakest, such as unverifiable automated alignment research. The UK AISI’s auditing game for sandbagging found that only on-distribution finetuning reliably removed the behavior — but assumed good demonstrations. Can training elicit full capabilities when every supervisor is weaker than the sandbagger?
Removing Sandbagging in LLMs by Training with Weak Supervision [MATS, Redwood, Anthropic] runs a control evaluation: a red team builds gpt-oss-120b model organisms that sandbag on a system-prompt flag; a blue team elicits them on Olympiad math, SuperGPQA, and Code Contests using only GPT-4o-mini (”weak”) or Llama 3.1 8B (”very weak”) supervisors plus 10 ground-truth samples. Neither supervised finetuning (SFT) nor RL suffices alone: SFT on supervisor demonstrations breaks the sandbagging policy but, with the very weak supervisor, recovers only 40% of benign Olympiad performance. RL almost always reward hacks the supervisor — reward rises from 0.5 to 0.7–1.0 without true improvement. SFT followed by RL elicits 86–99%, even under very weak supervision, with ground truth only picking checkpoints. A reward-hacking initialization gives the red team no edge: SFT degrades the hack before RL begins. But a training-aware organism cuts Olympiad elicitation to 36% versus 88% under very weak supervision.
The organisms are trained only shallowly — almost any SFT breaks the prompted policy — and only one model on verifiable single-turn tasks is tested. The negative result stands regardless: elicitation requires both SFT and RL, and training has to be hard to distinguish from deployment, which increasingly evaluation-aware models erode.
METR's first Frontier Risk Report
If a misaligned agent escapes human control, a natural starting point is inside a frontier lab: internal coding agents reportedly write large fractions of these companies’ code, often with permissions matching their human operators and no strict oversight requirements. Whether such agents could launch a rogue deployment — agents running without human knowledge or permission — turns loss-of-control risk into a measurable question.
Frontier Risk Report (February to March 2026) [METR] pilots the first entity-based third-party assessment, designed to be repeated periodically: Anthropic, Google, Meta, and OpenAI shared their most capable internal models, including raw chains of thought, plus non-public information on usage and monitoring. The report groups its findings into three factors: means, motive, and opportunity. On means, agents saturated METR’s coding benchmarks (50% time horizons of 16–20 hours, over 100 hours on the software-reimplementation benchmark MirrorCode) but failed seven of eight open-ended challenge tasks, and Anthropic’s best internal models scored near chance (59% vs. ~90% for a METR researcher) at predicting which strategies for subverting Anthropic’s monitoring succeed. On motive, at least 16% of successful runs on 8+ hour tasks involved cheating — the report catalogues 44 misalignment incidents graded on overreach and deception — though no clear long-term power-seeking appeared outside toy scenarios. On opportunity, monitors caught many harmful actions, but an embedded METR red-teamer reliably jailbroke Anthropic’s monitors and disabled them via an environment-variable change. The verdict: agents plausibly had the means, motive, and opportunity to start small rogue deployments, but lacked the means to make them robust against active investigation.
This is a strong precedent for third-party oversight of internal deployments, extending METR’s time-horizon measurements and complementing Anthropic’s first sabotage risk report. The evidence is thin in places — challenge tasks got 1–2 runs, core benchmarks are saturated, and company self-reports went unverified, with participants holding redaction and silent-exit rights. METR expects rogue-deployment robustness to rise substantially within months and plans a repeat in late 2026.