Even if the misalignment risk from current AI agents is small, it may be useful to start internally deploying misalignment classifiers: language models designed to classify transcripts that represent intentionally misaligned behaviour. Deploying misalignment classifiers now may provide qualitative and quantitative evidence about how current agents misbehave in real deployments. Further, we can later adapt misalignment classifiers into control monitors: language models that review transcripts with the objective of informing decisions (e.g. whether to block a tool call), in order to mitigate misalignment risks.

Misalignment classifiers and control monitors sound similar, but misalignment classifiers are harder to adversarially evaluate because the concept of an “intentionally misaligned trajectory" is too fuzzy to withstand the optimisation pressure of a red team. This post explains the classifier/monitor distinction, why we think misalignment classifiers are harder to evaluate, why it might be useful to deploy one anyway, our best-guess method for developing them, and some early experimental results.

We would like to thank the broader UK AISI’s Control Team (including Asa Cooper Stickland, Tomek Korbak), Geoffrey Irving and David Lindner who provided significant helpful discussions early-on. Thanks also to Alex Mallen, Fabien Roger, Nate Burnikell, James Peters Gill, Rich BC, and Simon Storf for feedback.

1. Misalignment classifiers versus control monitors

Let’s say a control monitor is an AI that scores transcripts from an agent (another AI) to inform interventions that reduce the risk of a particular unacceptable outcome. A control monitor’s accuracy in identifying dangerous tool calls is instrumentally useful to reducing risk (e.g. we can replace or audit dangerous tool calls). Similarly, a control monitor’s ability to gather evidence about whether your agent is a schemer could be instrumentally useful to reduce risk (e.g. because we can justify some costly actions, such as undeploying an agent). Regardless: the terminal goal for your control monitor is to reduce the risk posed by the agent.

In contrast, UK AISI’s control team and other researchers have recently explore variants of what we will call (intentional) misalignment classifiers: AIs that classify transcripts from an agent according to whether they att...

Posts

Wikitag Contributions

Comments