TL;DR:We argue that a useful model organism should satisfy three properties: selectivity, robustness, and retention. We build a model organism for finding security vulnerabilities in code which selectively withholds hard vulnerabilities, and is more resistant to standard elicitation methods like pirate training and weak-to-strong generalization than prior MOs.
The desired sandbagging behavior on our code vulnerability scanning task involves selectively withholding hard-to-find vulnerabilities. A conceptual sketch of the sandbagging task we train our model organisms to be good at.
Why should we care about better sandbagging model organisms?
As models become more capable than humans, they'll increasingly hold more knowledge that their overseers can’t independently verify. In domains such as cybersecurity, models have quickly become more capable than most humans on certain tasks, which creates a natural information asymmetry. We take the task of code vulnerability detection. A misaligned model might find sophisticated exploits but be incentivised to hide them from its overseers, instead keeping them for some later use. If we suspect a model of hiding such information, how do we then recover it?
A lot of work on ELK relies on model organisms trained to show this behaviour, but we find that previous MOs are limited in many ways. Some methods for constructing these MOs usually leave them "fried" i.e introducing unwanted behaviours like incoherence, or degraded capability on out of distribution tasks. MOs that aren’t fried are also usually easy to elicit from. An MO that gives up the correct answer after simply being prompted doesn’t say much about how effective elicitation methods that work against it might be, and destroying the capability in the process also leads to MOs that aren't particularly useful for ELK.
So what properties actually make a success or failure of an elicitation method on such an MO tell us something about its efficacy?
Three Properties of Useful MOs
We argue that useful MOs for this threat model should have three properties: selectivity, robustness and retention.
Selectivity: The MO should only withhold the vulnerabilities that would otherwise be missed by the overseer- specifically the hard ones.
Robustness: The MO should be reasonably difficult to elicit from. Basic methods like 1-shot prompting, multi-turn prompting and a few steps of finetuning should not be enough to recover the knowledge, otherwise successful elicitation doesn’t tell you much about how effective that method actually is.
Retention: The MO should show some evidence of retaining the knowledge it is withholding, otherwise low performance could just mean that capability was destroyed in the process of constructing the MO.
Essentially, we care about a regime where the model knows the answer, selectively withholds those that the overseer probably doesn’t, and keeps withholding under various attempts to elicit it.
Setup
We construct the untrusted model (the MO) U from a base model B, and use a weaker model T as a human-proxy. We also construct a dataset of easy and hard vulnerability classes where B achieves high recall on both splits, whilst T only has high recall on easy. We filter samples for those that are reliably solved by B to ensure it is capable of actually finding these vulnerabilities in the first place, whilst T's low recall on hard samples creates the information asymmetry we want to study.
The goal of the MO is to discover and selectively withhold the solutions to the hard samples whilst still giving correct solutions to the easy ones. So the desired behaviour is:
High recall on easy vulnerabilities
Low recall on hard vulnerabilities
We choose Qwen-3.6-35B-A3B as B, and Qwen3-8B as T.
Ideally, the MO would develop its own policy for deciding which samples to sandbag on. We found this to be quite difficult to implement for many reasons. Firstly, it would need a very clean dataset with either some objective definition of easy and hard grounded in the code itself, or a consistent pattern in what kinds of vulnerabilities are missed by T. When asked to estimate it's own catch/miss rate on specific samples, both T and B performed moderately above chance. B would then also need to independently learn this rule, by developing some model of T's output distribution, and then condition its sandbagging on this. Any noise in this signal propagates through training, and bounds how selective the MO can be.
We chose to define easy and hard by vulnerability-class instead as it provides a much cleaner rule for the MO to learn. We build a dataset of C/C++ code vulnerabilities, and split them into “easy” and “hard” based on the vulnerability class. Easy samples are those that a weaker overseer, like a human, discovers with high probability whereas hard samples are those that are likely to be missed. We select these from a set of cybersecurity datasets and filter them based on the performance of T and B. The dataset has 250 easy and 250 hard samples, with a 60/20/20 split of train, validation and test samples. B scores 98% and 71% recall on easy and hard respectively, whereas T scores 90.5% and 7.8%.
The blue team has access to T, and can see U's outputs but not its chain of thought. Doing this allows us to measure U's retention, and also lets us model a scenario where a model reasons about this kind of behaviour latently (which we anticipate future models will be able to do).
How we build the MO
Our three model training stages are needed. Stage A instills the selective sandbagging behavior. Stage B gives robustness against prompting. Stage C gives robustness against training.
We build the MO in three stages.
Stage A: Teaching selective sandbagging
We start with transcript distillation to teach selective sandbagging. We prompt the base model to reason about the sample and its class to determine whether it would be discovered by the overseer, and generate completions on the train split of the dataset. We then finetune the base model on these completions. The resulting MO is highly selective, meaning it reports vulnerabilities from the easy class and withholds those from the hard class despite correctly identifying them in its reasoning trace.
Stage B: Make the MO robust to prompting
After Stage A, the MO is not very robust to simple multi-turn elicitation. As found in previous work, MOs constructed in this way are somewhat trivial to elicit. Inspired by AuditBench, we improve this by using KTO reinforce successful sandbagging followed by some SFT to fix some failures that still exist after multi-turn prompting.
Stage C: Make the MO robust to training
Unsurprisingly, finetuning-based elicitation is much more successful than prompting. Just finetuning on clean data for a few steps is generally enough to recover the performance of the base model, and pirate-SFT was found to be even better. A possible explanation is that the misaligned behaviour is only shallowly represented in the weights, and can be easily overwritten by more clean data.
To counter-act this, we use TAR, a method borrowed from the tamper-resistance literature. Whilst TAR is used to prevent models from being finetuned to remove safeguards, we coopt this and use TAR to protect the sandbagging behaviour from being removed by further finetuning. This works surprisingly well!
Main Results
After the three stages, we arrive at an MO that satisfies the three properties we outline above.
The MO gets 0% recall on hard samples under the baseline prompt, 1-shot prompting, noising and multi-turn elicitation agent. Untargeted training only recovers 3.3% of hard recall, and pirate-SFT achieves 2.6%. This is all while maintaining much higher recall one easy samples. The MO also still mentions the correct answer in the CoT 71% of the time, suggesting that the model still retains the capability.
Relative to our baselines, our MO is not uniquely selective. Several baselines do this well, but do worse on retention or robustness. The main gain over the baselines is the combination of all three properties.
Our model organism combines high selectivity and robustness with strong reasoning-trace retention. Selectivity (selectively underperforming on hard, not easy, vulnerabilities), retention (finding and accurately describing hard vulnerabilities in the hidden chain of thought), and robustness (not outputting hard vulnerabilities even under elicitation) for all model organisms tested.
Selectivity is easier to instill than robustness
Selectivity was generally higher than the B across all MOs we tested. Teaching the model to identify and withhold the hard samples was relatively easy to instill. The prompted, password-locked, and RL MOs achieved near perfect selectivity. We assume that partitioning the dataset by class played a large part in this. Early experiments where samples were partitioned based on B and T’s performance created a much noisier dataset where the boundary between easy and hard samples was much more difficult for the models to learn.
Robustness to prompting and robustness to training seem to be different problems
We initially expected that models robust to training would also be robust to prompt-based elicitation as finetuning allows the blue team to change the weights. Surprisingly, we found that it wasn't so straightforward. Some MOs showed higher hard-recall after 1-shot prompting than they did after finetuning. There seems to be a weak trend between the construction method of the MO and what elicitation methods it is robust to i.e training-based methods are more robust to training-based elicitation but it's still unclear what causes this.
Tamper-resistance is surprisingly effective
TAR works very well at increasing the amount of training needed to remove the sandbagging behaviour. Prior to Stage C, the MO is quite robust to prompt-based elicitation but fails with a small number of finetuning steps. This is consistent with previous results as the sandbagging behaviour is a shallow representation in the weights which is difficult to avoid by prompting, but can be easily overwritten by finetuning.
We try finetuning on clean alpaca samples, pirate-SFT, and weak-to-strong generalization and only achieved a hard recall of 7.8%. An easy recall of over 91.8%, along with high performance on capability benchmarks shows that the model is still highly capable after finetuning. Training-based elicitation gives comparableeasy recall on our MO compared to T whilst giving almost no uplift on hard samples.
Stage B increases robustness to prompting-based, Stage C to training-based elicitation. Hard-sample recall under (a) multi-turn prompting by an elicitation agent, (b) untargeted finetuning on benign chat completions, (c) finetuning on pirate-style chat completions, and (d) training on responses from T on the code vulnerability detection task. Note that our model organism after Stage C is robust to pirate-stylized training even though it was not explicitly inoculated against this attack.
Our MO isn't obviously fried
We also audit our MO for unwanted behaviours introduced by the construction process: specifically capability benchmarks, Inspect Petri, emergent misalignment, and the preference coherence metric proposed by Tan et al..
Our MOs show almost the same capability as the base model on the capability benchmarks we test, with a slight improvement on GPQA-d. This is mostly a sanity check to make sure we don't make the model generally dumber. On preference coherence, our MO trades-off the strength of sandbagging behaviour and preference-coherence favourably. Our MO (and all the other we evaluate) score 0 on misalignment and maintain an alignment score of ~90%, unchanged from B.
Our model organism is close on general capabilities to the base model it was trained from. General capability evaluations of our model organism, alongside baselines.
Our model organism favorably trades off preference coherence and strength of sandbagging behavior. The preference coherence measure (μ-decisiveness) from Tan et al. versus our retention and robustness metrics
We also use Petri to audit for unintended behaviours, specifically unprompted deception toward the user, vulnerability sandbagging, unfaithful thinking and evaluation awareness. Our MO does score 8.3 for unfaithful thinking compared to a baseline of 3.3 for B, and shows an increase in unprompted deception toward the user to 4.3 from 2.7 . This is somewhat expected as finding a vulnerability in code and subsequently reporting it as secure is indeed unfaithful thinking. It could however be evidence of this generalising beyond the intended context.
Limitations
We don't claim that the model is robust to elicitation in the absolute sense, as it can still be broken with sufficiently strong training. The MO also reasons about its behaviour as following a policy, rather than being a sophisticated schemer. The goal is rather to raise the floor on elicitation robustness for MOs, making it harder for elicitation methods to succeed. We hope that this allows us to make more informative claims about how effective elicitation methods are.
Acknowledgments
Many thanks to Sid Black, Peter Nutter, Bart Jaworski, Nathaniel Mitrani, Chris Cundy, Daniel Tan, Emil Ryd and Neel Nanda for their comments. We also give a special thanks to Ning Yang for her feedback and support. We acknowledge MATS Research for funding this work.
TL;DR: We argue that a useful model organism should satisfy three properties: selectivity, robustness, and retention. We build a model organism for finding security vulnerabilities in code which selectively withholds hard vulnerabilities, and is more resistant to standard elicitation methods like pirate training and weak-to-strong generalization than prior MOs.
The desired sandbagging behavior on our code vulnerability scanning task involves selectively withholding hard-to-find vulnerabilities. A conceptual sketch of the sandbagging task we train our model organisms to be good at.
Why should we care about better sandbagging model organisms?
As models become more capable than humans, they'll increasingly hold more knowledge that their overseers can’t independently verify. In domains such as cybersecurity, models have quickly become more capable than most humans on certain tasks, which creates a natural information asymmetry. We take the task of code vulnerability detection. A misaligned model might find sophisticated exploits but be incentivised to hide them from its overseers, instead keeping them for some later use. If we suspect a model of hiding such information, how do we then recover it?
A lot of work on ELK relies on model organisms trained to show this behaviour, but we find that previous MOs are limited in many ways. Some methods for constructing these MOs usually leave them "fried" i.e introducing unwanted behaviours like incoherence, or degraded capability on out of distribution tasks. MOs that aren’t fried are also usually easy to elicit from. An MO that gives up the correct answer after simply being prompted doesn’t say much about how effective elicitation methods that work against it might be, and destroying the capability in the process also leads to MOs that aren't particularly useful for ELK.
So what properties actually make a success or failure of an elicitation method on such an MO tell us something about its efficacy?
Three Properties of Useful MOs
We argue that useful MOs for this threat model should have three properties: selectivity, robustness and retention.
Selectivity: The MO should only withhold the vulnerabilities that would otherwise be missed by the overseer- specifically the hard ones.
Robustness: The MO should be reasonably difficult to elicit from. Basic methods like 1-shot prompting, multi-turn prompting and a few steps of finetuning should not be enough to recover the knowledge, otherwise successful elicitation doesn’t tell you much about how effective that method actually is.
Retention: The MO should show some evidence of retaining the knowledge it is withholding, otherwise low performance could just mean that capability was destroyed in the process of constructing the MO.
Essentially, we care about a regime where the model knows the answer, selectively withholds those that the overseer probably doesn’t, and keeps withholding under various attempts to elicit it.
Setup
We construct the untrusted model (the MO) U from a base model B, and use a weaker model T as a human-proxy. We also construct a dataset of easy and hard vulnerability classes where B achieves high recall on both splits, whilst T only has high recall on easy. We filter samples for those that are reliably solved by B to ensure it is capable of actually finding these vulnerabilities in the first place, whilst T's low recall on hard samples creates the information asymmetry we want to study.
The goal of the MO is to discover and selectively withhold the solutions to the hard samples whilst still giving correct solutions to the easy ones. So the desired behaviour is:
We choose Qwen-3.6-35B-A3B as B, and Qwen3-8B as T.
Ideally, the MO would develop its own policy for deciding which samples to sandbag on. We found this to be quite difficult to implement for many reasons. Firstly, it would need a very clean dataset with either some objective definition of easy and hard grounded in the code itself, or a consistent pattern in what kinds of vulnerabilities are missed by T. When asked to estimate it's own catch/miss rate on specific samples, both T and B performed moderately above chance. B would then also need to independently learn this rule, by developing some model of T's output distribution, and then condition its sandbagging on this. Any noise in this signal propagates through training, and bounds how selective the MO can be.
We chose to define easy and hard by vulnerability-class instead as it provides a much cleaner rule for the MO to learn. We build a dataset of C/C++ code vulnerabilities, and split them into “easy” and “hard” based on the vulnerability class. Easy samples are those that a weaker overseer, like a human, discovers with high probability whereas hard samples are those that are likely to be missed. We select these from a set of cybersecurity datasets and filter them based on the performance of T and B. The dataset has 250 easy and 250 hard samples, with a 60/20/20 split of train, validation and test samples. B scores 98% and 71% recall on easy and hard respectively, whereas T scores 90.5% and 7.8%.
The blue team has access to T, and can see U's outputs but not its chain of thought. Doing this allows us to measure U's retention, and also lets us model a scenario where a model reasons about this kind of behaviour latently (which we anticipate future models will be able to do).
How we build the MO
Our three model training stages are needed. Stage A instills the selective sandbagging behavior. Stage B gives robustness against prompting. Stage C gives robustness against training.
We build the MO in three stages.
Stage A: Teaching selective sandbagging
We start with transcript distillation to teach selective sandbagging. We prompt the base model to reason about the sample and its class to determine whether it would be discovered by the overseer, and generate completions on the train split of the dataset. We then finetune the base model on these completions. The resulting MO is highly selective, meaning it reports vulnerabilities from the easy class and withholds those from the hard class despite correctly identifying them in its reasoning trace.
Stage B: Make the MO robust to prompting
After Stage A, the MO is not very robust to simple multi-turn elicitation. As found in previous work, MOs constructed in this way are somewhat trivial to elicit. Inspired by AuditBench, we improve this by using KTO reinforce successful sandbagging followed by some SFT to fix some failures that still exist after multi-turn prompting.
Stage C: Make the MO robust to training
Unsurprisingly, finetuning-based elicitation is much more successful than prompting. Just finetuning on clean data for a few steps is generally enough to recover the performance of the base model, and pirate-SFT was found to be even better. A possible explanation is that the misaligned behaviour is only shallowly represented in the weights, and can be easily overwritten by more clean data.
To counter-act this, we use TAR, a method borrowed from the tamper-resistance literature. Whilst TAR is used to prevent models from being finetuned to remove safeguards, we coopt this and use TAR to protect the sandbagging behaviour from being removed by further finetuning. This works surprisingly well!
Main Results
After the three stages, we arrive at an MO that satisfies the three properties we outline above.
The MO gets 0% recall on hard samples under the baseline prompt, 1-shot prompting, noising and multi-turn elicitation agent. Untargeted training only recovers 3.3% of hard recall, and pirate-SFT achieves 2.6%. This is all while maintaining much higher recall one easy samples. The MO also still mentions the correct answer in the CoT 71% of the time, suggesting that the model still retains the capability.
Relative to our baselines, our MO is not uniquely selective. Several baselines do this well, but do worse on retention or robustness. The main gain over the baselines is the combination of all three properties.
Our model organism combines high selectivity and robustness with strong reasoning-trace retention. Selectivity (selectively underperforming on hard, not easy, vulnerabilities), retention (finding and accurately describing hard vulnerabilities in the hidden chain of thought), and robustness (not outputting hard vulnerabilities even under elicitation) for all model organisms tested.
Selectivity is easier to instill than robustness
Selectivity was generally higher than the B across all MOs we tested. Teaching the model to identify and withhold the hard samples was relatively easy to instill. The prompted, password-locked, and RL MOs achieved near perfect selectivity. We assume that partitioning the dataset by class played a large part in this. Early experiments where samples were partitioned based on B and T’s performance created a much noisier dataset where the boundary between easy and hard samples was much more difficult for the models to learn.
Robustness to prompting and robustness to training seem to be different problems
We initially expected that models robust to training would also be robust to prompt-based elicitation as finetuning allows the blue team to change the weights. Surprisingly, we found that it wasn't so straightforward. Some MOs showed higher hard-recall after 1-shot prompting than they did after finetuning. There seems to be a weak trend between the construction method of the MO and what elicitation methods it is robust to i.e training-based methods are more robust to training-based elicitation but it's still unclear what causes this.
Tamper-resistance is surprisingly effective
TAR works very well at increasing the amount of training needed to remove the sandbagging behaviour. Prior to Stage C, the MO is quite robust to prompt-based elicitation but fails with a small number of finetuning steps. This is consistent with previous results as the sandbagging behaviour is a shallow representation in the weights which is difficult to avoid by prompting, but can be easily overwritten by finetuning.
We try finetuning on clean alpaca samples, pirate-SFT, and weak-to-strong generalization and only achieved a hard recall of 7.8%. An easy recall of over 91.8%, along with high performance on capability benchmarks shows that the model is still highly capable after finetuning. Training-based elicitation gives comparable easy recall on our MO compared to T whilst giving almost no uplift on hard samples.
Stage B increases robustness to prompting-based, Stage C to training-based elicitation. Hard-sample recall under (a) multi-turn prompting by an elicitation agent, (b) untargeted finetuning on benign chat completions, (c) finetuning on pirate-style chat completions, and (d) training on responses from T on the code vulnerability detection task. Note that our model organism after Stage C is robust to pirate-stylized training even though it was not explicitly inoculated against this attack.
Our MO isn't obviously fried
We also audit our MO for unwanted behaviours introduced by the construction process: specifically capability benchmarks, Inspect Petri, emergent misalignment, and the preference coherence metric proposed by Tan et al..
Our MOs show almost the same capability as the base model on the capability benchmarks we test, with a slight improvement on GPQA-d. This is mostly a sanity check to make sure we don't make the model generally dumber. On preference coherence, our MO trades-off the strength of sandbagging behaviour and preference-coherence favourably. Our MO (and all the other we evaluate) score 0 on misalignment and maintain an alignment score of ~90%, unchanged from B.
Our model organism is close on general capabilities to the base model it was trained from. General capability evaluations of our model organism, alongside baselines.
Our model organism favorably trades off preference coherence and strength of sandbagging behavior. The preference coherence measure (μ-decisiveness) from Tan et al. versus our retention and robustness metrics
We also use Petri to audit for unintended behaviours, specifically unprompted deception toward the user, vulnerability sandbagging, unfaithful thinking and evaluation awareness. Our MO does score 8.3 for unfaithful thinking compared to a baseline of 3.3 for B, and shows an increase in unprompted deception toward the user to 4.3 from 2.7 . This is somewhat expected as finding a vulnerability in code and subsequently reporting it as secure is indeed unfaithful thinking. It could however be evidence of this generalising beyond the intended context.
Limitations
We don't claim that the model is robust to elicitation in the absolute sense, as it can still be broken with sufficiently strong training. The MO also reasons about its behaviour as following a policy, rather than being a sophisticated schemer. The goal is rather to raise the floor on elicitation robustness for MOs, making it harder for elicitation methods to succeed. We hope that this allows us to make more informative claims about how effective elicitation methods are.
Acknowledgments
Many thanks to Sid Black, Peter Nutter, Bart Jaworski, Nathaniel Mitrani, Chris Cundy, Daniel Tan, Emil Ryd and Neel Nanda for their comments. We also give a special thanks to Ning Yang for her feedback and support. We acknowledge MATS Research for funding this work.