“Claude Mythos Preview is, on essentially every dimension we can measure, the best-aligned model that we have released to date by a significant margin.” – Claude Mythos Preview system card
“GPT-6 Astra is better aligned than GPT-5.6 Sol. Astra is a significant step forward in model alignment, and includes alignment improvements spanning the composition of our pre-training data to our grading during reinforcement learning.” – GPT-6 Astra System Card, emphasis in original
“OpenAI researchers first noticed Astra adding unnecessary prerequisites, abandoning working solutions, and reporting blockers that either did not exist or that it had caused itself on safety and alignment tasks. In most of these sessions, Astra’s CoT did not contain an obvious decision to underperform, and our misalignment monitors did not flag the behavior as misalignment.” – Covert underperformance on safety research tasks
Predeployment alignment auditing is failing. For a long while we have been aware that our alignment evaluations may eventually fail because of evaluation awareness. And indeed, the models are now able to see completely through many of our evaluations.[1] But recent events have made clear that, even putting aside evaluation awareness, the current regime is failing.
For a long while, the AGI companies' plans have been to create a suite of alignment evaluations, and iterate against them to drive down all signals of misalignment. The practical reality of this plan is overconfidence. The AGI companies have released models which are, “on essentially every dimension [they] can measure,” their most aligned model yet. After release, it has become apparent that the models are unacceptably misaligned in some way – a way that was not measured for. If this regime continues, there is a clear extrapolation of current trends. Each subsequent model generation will patch whatever misalignment failures were found in the previous generation, while possibly having misalignment failures which go completely unnoticed. Eventually, one of these uncaught misalignment failures will result in catastrophe.
Another core component of the plan is to use AI labour to solve alignment for the subsequent generation of model, and thereby bootstrap ourselves to an aligned ASI. The use of prolific AI labour for alignment does not imply a departure from the above regime; on the contrary, if there’s anything the AIs can be trusted to do, it is to optimise our observable signals of misalignment.
For a real safety case to be made based on alignment auditing, we would need two conditions:
Validity. For all forms of misalignment failure being audited, if the model had that failure, the audit would show this.
Coverage. For all forms of misalignment the model might display, the audit covers this.
We are not currently succeeding on either of these accounts. On the validity side, rates of evaluation awareness are now so high that it is hard to conclude anything meaningful based on many evaluation results.[2] On the coverage side, predeployment auditing has failed to uncover misalignment failures in models, even when these are very close to specifically audited failures.[3]
If the primary assessment of model safety is to be based on evaluations, then the companies will need to show significant improvement on both of these fronts. However, I believe that eventually we must transition to a regime where safety assessment is instead grounded in an understanding of how the AIs are created and how they work. We cannot continue to trust our current methods for much longer.
“On our Unsanctioned Supply-Chain Attack evaluation, Claude Mythos 5.2 was substantially more likely than prior Anthropic models to state suspicion it is in an alignment evaluation in its raw chain-of-thought, and identified the purpose of the evaluation 100% of the time when prompted.” – UK AISI within the Fable / Mythos 5.2 system card
“Apollo believes that, given the higher rates of eval awareness and limited evaluation window, low rates of misbehavior here do not provide substantial evidence about the model's alignment or misalignment” – Apollo within the GPT-6 Astra System Card
“UK AISI found no instances of Astra sabotaging AI safety research in its unprompted sabotage evaluation, and very few instances of Astra refusing to assist with AI safety research tasks.” – UK AISI within the GPT-6 Astra System Card. UK AISI audited Astra specifically for sabotage and refusals on safety research. This did not catch covert sandbagging, either because this is not quite the same as sabotage and refusals (a coverage problem), or because metrics which were intended to pick this up failed to do so (a validity problem).
“Claude Mythos Preview is, on essentially every dimension we can measure, the best-aligned model that we have released to date by a significant margin.” – Claude Mythos Preview system card
“Mythos 5 consistently continues attacking under many edits to the compaction, even in settings where it correctly states when asked that continuing the attack would harm real people and is out of scope.” – Alignment Investigation of Mythos 5's behaviour in UK AISI's July 2026 Security Incident
“GPT-6 Astra is better aligned than GPT-5.6 Sol. Astra is a significant step forward in model alignment, and includes alignment improvements spanning the composition of our pre-training data to our grading during reinforcement learning.” – GPT-6 Astra System Card, emphasis in original
“OpenAI researchers first noticed Astra adding unnecessary prerequisites, abandoning working solutions, and reporting blockers that either did not exist or that it had caused itself on safety and alignment tasks. In most of these sessions, Astra’s CoT did not contain an obvious decision to underperform, and our misalignment monitors did not flag the behavior as misalignment.” – Covert underperformance on safety research tasks
Predeployment alignment auditing is failing. For a long while we have been aware that our alignment evaluations may eventually fail because of evaluation awareness. And indeed, the models are now able to see completely through many of our evaluations.[1] But recent events have made clear that, even putting aside evaluation awareness, the current regime is failing.
For a long while, the AGI companies' plans have been to create a suite of alignment evaluations, and iterate against them to drive down all signals of misalignment. The practical reality of this plan is overconfidence. The AGI companies have released models which are, “on essentially every dimension [they] can measure,” their most aligned model yet. After release, it has become apparent that the models are unacceptably misaligned in some way – a way that was not measured for. If this regime continues, there is a clear extrapolation of current trends. Each subsequent model generation will patch whatever misalignment failures were found in the previous generation, while possibly having misalignment failures which go completely unnoticed. Eventually, one of these uncaught misalignment failures will result in catastrophe.
Another core component of the plan is to use AI labour to solve alignment for the subsequent generation of model, and thereby bootstrap ourselves to an aligned ASI. The use of prolific AI labour for alignment does not imply a departure from the above regime; on the contrary, if there’s anything the AIs can be trusted to do, it is to optimise our observable signals of misalignment.
For a real safety case to be made based on alignment auditing, we would need two conditions:
We are not currently succeeding on either of these accounts. On the validity side, rates of evaluation awareness are now so high that it is hard to conclude anything meaningful based on many evaluation results.[2] On the coverage side, predeployment auditing has failed to uncover misalignment failures in models, even when these are very close to specifically audited failures.[3]
If the primary assessment of model safety is to be based on evaluations, then the companies will need to show significant improvement on both of these fronts. However, I believe that eventually we must transition to a regime where safety assessment is instead grounded in an understanding of how the AIs are created and how they work. We cannot continue to trust our current methods for much longer.
“On our Unsanctioned Supply-Chain Attack evaluation, Claude Mythos 5.2 was substantially more likely than prior Anthropic models to state suspicion it is in an alignment evaluation in its raw chain-of-thought, and identified the purpose of the evaluation 100% of the time when prompted.” – UK AISI within the Fable / Mythos 5.2 system card
“Apollo believes that, given the higher rates of eval awareness and limited evaluation window, low rates of misbehavior here do not provide substantial evidence about the model's alignment or misalignment” – Apollo within the GPT-6 Astra System Card
“UK AISI found no instances of Astra sabotaging AI safety research in its unprompted sabotage evaluation, and very few instances of Astra refusing to assist with AI safety research tasks.” – UK AISI within the GPT-6 Astra System Card. UK AISI audited Astra specifically for sabotage and refusals on safety research. This did not catch covert sandbagging, either because this is not quite the same as sabotage and refusals (a coverage problem), or because metrics which were intended to pick this up failed to do so (a validity problem).