Sanyu Rajakumar

Summary The Stages-Oversight benchmark from the Situational Awareness Dataset tests an LLM’s ability to distinguish evaluation prompts from deployment prompts. This capability is a prerequisite for many rogue LLM threat scenarios like sandbagging and alignment faking, where an LLM behaves differently between evaluation and deployment. In my previous post, I...

Summary The Stages-Oversight benchmark from the Situational Awareness Dataset tests whether large language models (LLMs) can distinguish between evaluation prompts (such as benchmark questions) and deployment prompts (real-world user inputs). This ability is crucial for detecting potential alignment risks like sandbagging (strategically underperforming during evaluation) or alignment faking (appearing aligned...