Project done as the capstone for ARENA 5.0. Write-up by Lucy Wingard, project team was Lucy Wingard, Gareth Tan, and Lily Li. Special thanks to David Quarel for help working through our various bugs. See code here: https://github.com/garetht/absolute_zero_reproduction What is Absolute Zero? Absolute Zero [1]is a recently developed RLVR (reinforcement...
Produced as the application project for Neel Nanda's MATS 8.0 Stream Summary What problem am I investigating? Reasoning models define the current state of the art for AI performance, but their safety properties are understudied. While some argue that Chain-of-Thought (CoT) reasoning improves safety by allowing analysis and steering of...
Produced as the capstone project for AI Safety Fundamentals Course Oct 2024 - Jan 2025 Overview Anthropic's paper Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training[1] demonstrated that it is possible to create a misaligned AI that is resilient to our current best safety practices (RLHF, SFT, Adversarial...