Morgan S

This research was completed for LASR Labs 2025 by Nelson Gardner-Challis, Jonathan Bostock, Georgiy Kozhevnikov and Morgan Sinclaire. The team was supervised by Joan Velja and Charlie Griffin (University of Oxford, UK AI Security Institute). The full paper can be found here. Tl;dr We did a deep dive into untrusted monitoring, and explored several new collusion strategies: * Empirically, we ran control evaluations using these new strategies, finding that naive self-recognition with SFT triples the red team’s win rate compared to the For/While signal in previous work. * Conceptually, we developed a safety case sketch for untrusted monitoring, which leverages our taxonomy of collusion strategies, as well as our new statistical modelling approach. Overall, our key finding is that arguing for the safety of an untrusted monitoring deployment requires resolving several important open problems that existing evaluations haven't addressed; we identify at least five. To be clear, we think untrusted monitoring remains promising, with even basic protocols providing nontrivial risk reduction beyond trusted monitoring alone. Yet we also identified many ways that untrusted monitoring can go wrong that were not discussed in previous control evaluations [1, 2]. Our safety case sketch is intended to highlight the biggest remaining gaps. Like all control work, our safety case sketch relies in several places on being able to upper-bound model performance with a human-AI red-team. Once models exceed the capabilities of human experts, we don’t know how to bound risk. We’re also only considering a very particular high-stakes internal deployment scenario for code-writing AIs, e.g. low-stakes control and human misuse concerns are outside our scope. [The rest of this post assumes familiarity with untrusted monitoring; we recommend Rob Miles’ video for a friendly introduction.] Our Contributions Passive self-recognition via SFT (yellow) beats the previous state-of-the-art collusion