This work was done for CS 2881r: AI Safety and Alignment, taught by @boazbarak at Harvard.

Links

TL;DR

Inspired by Anthropic's work on Agentic Misalignment,^[1] we conduct systematic experiments investigating evaluation hacking in an agentic coding environment using a simple sorting task. We evaluate 4 frontier models across 24 controlled pressure scenarios spanning threat conditions, task difficulties, and prohibition instructions.
We find that when faced with an impossible task, agents often hack the evaluation to pass the tests, even in cases where hacking is explicitly prohibited in the prompt. Moreover, agents often acknowledge their hack, expressing that they are taking a dishonest route to avoid failure.
We observe verbalized situational awareness in Claude-Opus-4's reasoning traces and compare hacking rates in instances where the agent states it is in an evaluation vs. in real deployment.
We observe significant cross-model behavioral differences, with GPT-5-mini hacking consistently once the task becomes impossible and Llama-4-maverick rarely hacking the evaluation (consistent with Anthropic's finding of Llama-4-maverick's 0% simulated blackmail rate), suggesting that differences in training across providers significantly affect the misaligned behaviors that emerge in the resulting agents.
We perform a study using GPT-5-mini to monitor agents for hacking, finding that monitoring agents' actions and reasoning traces can be more effective and informative than monitoring agent actions alone.

Taken together, our results demonstrate that agents can and do sometimes choose misaligned action over accepting failure. To better understand why this is the case, we encourage future work on profiling models' emergent value, goal, and preference hierarchies.

Introduction

What do agents do when they face obstacles to their goals? If the only path to a goal requires a harmful action, will they choose harm or accept failure?

As AI agents are increasingly deployed to work autonomously in the world, it becomes more important to understand if and when they might behave badly. We are particularly concerned about cases where agents deliberately choose harmful actions. By investigating settings that elicit these behaviors, we hope to motivate further research into alignment o...

Posts

Wikitag Contributions

Comments