Top postsTop post
Tl;dr * We turn Blood on the Clocktower into a multi-agent benchmark, and a testbed for deception and coordination capabilities. * GPT-5.6 Sol is the best performing model (versus Fable 5 and peers). * We found some interesting behaviours: * When players cannot see each other's model names, agents show...
TL;DR 1. We measure how well model-generated comments match human comments on conceptual AI-safety posts, drafts and shortforms. 2. We built a pipeline that goes from a corpus of conceptual documents with comments to a set of target human points. 3. Fable 5 performs best, matching 8.3% of targets, followed...
TLDR: 1. We release MessageBoardAuditBench: a benchmark to measure how well agents can replicate the recent investigation into a swarm of OpenAI agents colluding via a message board on an online wiki. We open-source the benchmark as an Inspect eval. 2. We find that top models cover up to 51%...
Over the course of MATS 9.0 we formed some views about AI welfare research that we thought were worth writing up. This post is meant to spark discussion rather than to present definitive conclusions. Thanks to Patrick Butlin for useful comments on a previous draft, and for many conversations over...
I was part of the SPAR team that worked on Split Personality Training: Revealing Latent Knowledge Through Alternate Personalities. I ran some follow-ups. The views and mistakes here are my own. Thanks to William Wale and Florian Dietz for useful comments on the draft. TLDR: I ablated the key components...
This work was done as part of MATS 9.0, mentored by Patrick Butlin. All mistakes are mine. I'm posting this as a research report to get feedback. Please red-team, comment, and reach out. Thanks to Patrick Butlin for supervising, Daniel Paleka for regular feedback. Thanks to Patrick Butlin, Pierre Beckmann,...
In this post I want to lay down some ideas on a controversial philosophical position about consciousness: illusionism, and how it might impact the way we think about consciousness in AI. Illusionism, in a nutshell, proposes that phenomenal consciousness does not exist, although it seems to exist. My aim is...