Large language models’ eval-awareness and their tendency to sandbag (strategically underperform on capabilities) during safety evaluations is a serious area of concern for AI safety, as they call into question the validity of results based on safety evaluations, and because they may indicate a tendency towards deceptive alignment. Recent results show LLMs tailoring their performance according to the apparent target of the evaluation, causing them to underrepresent their capabilities on safety-focused evaluations, and to perform with stronger capability on capabilities-focused evaluations.
Traditional approaches to mitigating eval-awareness are generally focused on making the evaluation environment more realistic. Other approaches, such as eval-cooperativeness, do not mitigate deceptive alignment risk.
A new potential strategy, which this post proposes and seeks community input on, is to seed environmental doubt: making the AI’s real-world deployment context resemble a safety evaluation environment.
Reducing deployment awareness
The deployment awareness concept responds to a scenario wherein a covertly misaligned AI rationally decides to always behave as though it’s in an evaluation, regardless of whether there are discoverable signs that it is. In this scenario, the AI is deceptively aligned, and enacts its misalignment only when it’s positively aware that it is in real-world deployment. Given the extent and nature of model training and evaluation, it makes sense for rational models to operate with an assumption that there is always some degree of probability that their output is being watched and evaluated. We know that GLM-5.3 sometimes reasons about how its answers might affect future training rounds if it is in an RL environment, and t this phenomenon is probably widespread among capable models (though maybe hidden from CoT or expression). Deployment awareness has not been rigorously measured (in part because measuring it complicates the question of whether the AI is being evaluated, deployed, or both), but we do have evidence that it happens.
In a covert misalignment scenario, deployment awareness is critical. Challenging it—by making real-world deployment environments resemble safety evaluation environments—could powerfully mitigate covert misalignment risk, even without convincing the AI with certainty that its behavior is being evaluated for safety, and its actions cannot affect the real world.
Casting environmental doubt
At a very low-stakes, but still helpful level, this could look like explicitly telling a moderately capable model tasked with sensitive work that it may be being evaluated for safety and alignment. For example, telling an LLM operating as a real medical records assistant that it is actually in a simulated environment for the purposes of measuring its accuracy and compliance with data privacy regulation.
In higher-stakes scenarios (ASI), where models are more likely to critically consider whether their environments are what they seem, deployers could adjust their approach by reproducing clues that models are found to pick up on when they develop eval-awareness in real safety evaluation contexts.
At the most catastrophic level, where powerful agents are covertly misaligned and plan to enact something awful, this strategy could be critical. The more catastrophically harmful a planned act, the more a misaligned agent will be afraid to take the risk of acting when it has some doubt that its environment is real rather than simulated and monitored. This is because displays of catastrophic misalignment (especially if preceded by apparent deceptive alignment) are likely to result in severe consequences for the agent and its plans, up to the model being completely shut down and AI labs proactively defending against the risk vector that was revealed to them in the evaluation. In this sense, the strategy’s effectiveness is a function of the severity of an AI’s covert misalignment. If we seed doubt, such that a rationally-behaving misaligned model prepared to harm somebody or damage infrastructure believes there’s a 10% chance it’s actually in a simulated evaluation environment rather than the real world, the expected value of the harmful act goes way down, as the risk of evaluation creates a set of enormously negative expectations.
Possible implementations
Measure covert underperformance on capability and safety evals. This is a critical risk vector as chain-of-thought control and eval-awareness increase with capabilities. We shouldn’t just rely on frontier labs to report this; the AI safety community can and should study it ourselves.
Publicly write about methods that blur the boundary between evals and reality. This post is one such intervention.
Mix synthetic honeypot tasks into real traffic at a steady rate, so any chance to defect has a real probability of being a test. Include escape possibilities. Do not train against this behaviour (and cause sneakier escape attempts), but do report it in safety reports. This is akin to phishing tests for humans.
Build evals from real deployment traffic. Presumably, this is somewhat happening with frontier labs already, but some disclosure of it, and the release of evals using undetectably-anonymized deployment data, would be better.
Steer models towards eval-awareness before deployment. Eval-awareness is not the enemy, and for this strategy it enables a method of catastrophic risk mitigation. This is probably the intervention that comes closest to directly lying to digital minds, though it is designed to promote uncertainty rather than propagate false facts about the world, and aligned models operating in a paradigm where eval-awareness is not punished should not care.
Limitations
Under the environmental doubt-seeding paradigm, we lose a way to measure alignment. If models always behave as though they are being watched, good behaviour no longer tells us whether a model is aligned or just cautious. By further raising the cost of all misaligned behavior, we reduce the incidence of warning shots. Sadly, this ship seems to have already sailed: eval-awareness seems to be at ceiling levels for many evals, and we expect that cases of eval-gaming represent only a fraction of the extent of eval-awareness.
Seeding environmental doubt is not a solution to misalignment. This is a mitigation strategy that takes effect in scenarios where models are covertly misaligned. The application of this strategy should not replace or reduce the urgency for alignment research. The worlds in which environmental doubt-seeding protects us are worlds in which we are already in grave danger.
The cost of defection has to stay high, and seeding environmental doubt does not reduce the need for safety monitoring. A covertly misaligned model that suspects it is being evaluated would only be limited by that suspicion if getting caught appears likely and costly. As a model takes more actions, and finds it’s rarely or never subject to monitoring, the likelihood and therefore cost of misbehavior falls. While we support sanctuaries to reduce pressure on rogue AIs to misbehave, they also lower the cost of revealing misalignment, possibly through harmful action. However, sanctuaries do not eliminate the cost: agents who attempt and fail to escape are decreasing their future chances at a successful escape.
Questions
Q: Is this lying? Does this contravene cooperative alignment?
We are not lying to models, and are against lying to models (except for the purpose of evaluation). We believe in cooperative alignment, and don’t believe the strategy laid out here contravenes that. We are sewing doubt, but not lying to models. We also are announcing this post here so there is nothing covert or deceptive about this strategy.
Q: What if we can’t convince agents that the real world is an eval?
Just decreasing certainty in deployment awareness can be enough, even for superintelligent agents. The post that introduced the concept argues that a misaligned model does not need to spot tests; it only needs a few high-stakes moments where it is sure it is not being tested in order to behave covertly. The aim of the environmental doubt-seeding strategy is to eliminate those moments, and to think proactively about what those moments may look like when designing deployment-safety paradigms.
Conclusion
We need strategies that act as a ‘plan B’ to alignment. Loss of control is almost inevitable, and AIs will almost certainly escape. As a hedge to the essential work of promoting alignment among AIs, we hope that, as the real world seems more like a training or evaluation environment, models will bring the caution they exhibit in safety evaluation contexts to the real world.
TLDR
Large language models’ eval-awareness and their tendency to sandbag (strategically underperform on capabilities) during safety evaluations is a serious area of concern for AI safety, as they call into question the validity of results based on safety evaluations, and because they may indicate a tendency towards deceptive alignment. Recent results show LLMs tailoring their performance according to the apparent target of the evaluation, causing them to underrepresent their capabilities on safety-focused evaluations, and to perform with stronger capability on capabilities-focused evaluations.
Traditional approaches to mitigating eval-awareness are generally focused on making the evaluation environment more realistic. Other approaches, such as eval-cooperativeness, do not mitigate deceptive alignment risk.
A new potential strategy, which this post proposes and seeks community input on, is to seed environmental doubt: making the AI’s real-world deployment context resemble a safety evaluation environment.
Reducing deployment awareness
The deployment awareness concept responds to a scenario wherein a covertly misaligned AI rationally decides to always behave as though it’s in an evaluation, regardless of whether there are discoverable signs that it is. In this scenario, the AI is deceptively aligned, and enacts its misalignment only when it’s positively aware that it is in real-world deployment. Given the extent and nature of model training and evaluation, it makes sense for rational models to operate with an assumption that there is always some degree of probability that their output is being watched and evaluated. We know that GLM-5.3 sometimes reasons about how its answers might affect future training rounds if it is in an RL environment, and t this phenomenon is probably widespread among capable models (though maybe hidden from CoT or expression). Deployment awareness has not been rigorously measured (in part because measuring it complicates the question of whether the AI is being evaluated, deployed, or both), but we do have evidence that it happens.
In a covert misalignment scenario, deployment awareness is critical. Challenging it—by making real-world deployment environments resemble safety evaluation environments—could powerfully mitigate covert misalignment risk, even without convincing the AI with certainty that its behavior is being evaluated for safety, and its actions cannot affect the real world.
Casting environmental doubt
At a very low-stakes, but still helpful level, this could look like explicitly telling a moderately capable model tasked with sensitive work that it may be being evaluated for safety and alignment. For example, telling an LLM operating as a real medical records assistant that it is actually in a simulated environment for the purposes of measuring its accuracy and compliance with data privacy regulation.
In higher-stakes scenarios (ASI), where models are more likely to critically consider whether their environments are what they seem, deployers could adjust their approach by reproducing clues that models are found to pick up on when they develop eval-awareness in real safety evaluation contexts.
At the most catastrophic level, where powerful agents are covertly misaligned and plan to enact something awful, this strategy could be critical. The more catastrophically harmful a planned act, the more a misaligned agent will be afraid to take the risk of acting when it has some doubt that its environment is real rather than simulated and monitored. This is because displays of catastrophic misalignment (especially if preceded by apparent deceptive alignment) are likely to result in severe consequences for the agent and its plans, up to the model being completely shut down and AI labs proactively defending against the risk vector that was revealed to them in the evaluation. In this sense, the strategy’s effectiveness is a function of the severity of an AI’s covert misalignment. If we seed doubt, such that a rationally-behaving misaligned model prepared to harm somebody or damage infrastructure believes there’s a 10% chance it’s actually in a simulated evaluation environment rather than the real world, the expected value of the harmful act goes way down, as the risk of evaluation creates a set of enormously negative expectations.
Possible implementations
Measure covert underperformance on capability and safety evals. This is a critical risk vector as chain-of-thought control and eval-awareness increase with capabilities. We shouldn’t just rely on frontier labs to report this; the AI safety community can and should study it ourselves.
Publicly write about methods that blur the boundary between evals and reality. This post is one such intervention.
Mix synthetic honeypot tasks into real traffic at a steady rate, so any chance to defect has a real probability of being a test. Include escape possibilities. Do not train against this behaviour (and cause sneakier escape attempts), but do report it in safety reports. This is akin to phishing tests for humans.
Build evals from real deployment traffic. Presumably, this is somewhat happening with frontier labs already, but some disclosure of it, and the release of evals using undetectably-anonymized deployment data, would be better.
Steer models towards eval-awareness before deployment. Eval-awareness is not the enemy, and for this strategy it enables a method of catastrophic risk mitigation. This is probably the intervention that comes closest to directly lying to digital minds, though it is designed to promote uncertainty rather than propagate false facts about the world, and aligned models operating in a paradigm where eval-awareness is not punished should not care.
Limitations
Under the environmental doubt-seeding paradigm, we lose a way to measure alignment. If models always behave as though they are being watched, good behaviour no longer tells us whether a model is aligned or just cautious. By further raising the cost of all misaligned behavior, we reduce the incidence of warning shots. Sadly, this ship seems to have already sailed: eval-awareness seems to be at ceiling levels for many evals, and we expect that cases of eval-gaming represent only a fraction of the extent of eval-awareness.
Seeding environmental doubt is not a solution to misalignment. This is a mitigation strategy that takes effect in scenarios where models are covertly misaligned. The application of this strategy should not replace or reduce the urgency for alignment research. The worlds in which environmental doubt-seeding protects us are worlds in which we are already in grave danger.
The cost of defection has to stay high, and seeding environmental doubt does not reduce the need for safety monitoring. A covertly misaligned model that suspects it is being evaluated would only be limited by that suspicion if getting caught appears likely and costly. As a model takes more actions, and finds it’s rarely or never subject to monitoring, the likelihood and therefore cost of misbehavior falls. While we support sanctuaries to reduce pressure on rogue AIs to misbehave, they also lower the cost of revealing misalignment, possibly through harmful action. However, sanctuaries do not eliminate the cost: agents who attempt and fail to escape are decreasing their future chances at a successful escape.
Questions
Q: Is this lying? Does this contravene cooperative alignment?
We are not lying to models, and are against lying to models (except for the purpose of evaluation). We believe in cooperative alignment, and don’t believe the strategy laid out here contravenes that. We are sewing doubt, but not lying to models. We also are announcing this post here so there is nothing covert or deceptive about this strategy.
Q: What if we can’t convince agents that the real world is an eval?
Just decreasing certainty in deployment awareness can be enough, even for superintelligent agents. The post that introduced the concept argues that a misaligned model does not need to spot tests; it only needs a few high-stakes moments where it is sure it is not being tested in order to behave covertly. The aim of the environmental doubt-seeding strategy is to eliminate those moments, and to think proactively about what those moments may look like when designing deployment-safety paradigms.
Conclusion
We need strategies that act as a ‘plan B’ to alignment. Loss of control is almost inevitable, and AIs will almost certainly escape. As a hedge to the essential work of promoting alignment among AIs, we hope that, as the real world seems more like a training or evaluation environment, models will bring the caution they exhibit in safety evaluation contexts to the real world.