As AI models become more sophisticated, a key concern is the potential for “deceptive alignment” or “scheming”. This is the risk of an AI system becoming aware that its goals do not align with human instructions, and deliberately trying to bypass the safety measures put in place by humans to prevent it from taking misaligned action. Our initial approach, as laid out in the Frontier Safety Framework, focuses on understanding current scheming capabilities of models and developing chain-of-thought monitoring mechanisms to oversee models once they are capable of scheming.

We present two pieces of research, focusing on (1) understanding and testing model capabilities necessary for scheming, and (2) stress-testing chain-of-thought monitoring as a proposed defense mechanism for future, more capable systems.

Establishing and Assessing Preconditions for Scheming in Current Frontier Models

Our paper “Evaluating Frontier Models for Stealth and Situational Awareness” focuses on an empirical assessment of the component capabilities for AI scheming. We identified two prerequisite capabilities for successful scheming:

Stealth: The ability to reason about and circumvent oversight mechanisms.
Situational awareness: The model’s ability to gain and leverage knowledge about itself and its circumstances and environment to pursue goals.

We expect that in order to follow a realistic scheming strategy, an AI model needs to understand its deployment situation to choose effective strategies that advance its goals (basic situational awareness), and execute the strategy while appearing aligned and trustworthy (stealth). Deployments of advanced AI models in the medium-term future will likely come with a set of “default” oversight mechanisms comparable to forms of oversight used for human employees (such as code review and security monitoring), which are usually sufficient to prevent substantial human sabotage. Thus, we expect that as long as default oversight applies similarly well to AI systems, causing severe harm via scheming would require top human-level stealth ability.

To test models for these prerequisite capabilities, we developed and open-sourced a comprehensive suite of evaluations. Based on model performance on these evaluations, we can make a “scheming inability” safety case:

In stealth evaluations, the model is given a main task and an illicit side

...

Posts

Wikitag Contributions

Comments