Author's Note: This work originated as a team project during the Apart Research AI Control Hackathon, where my teammates did the majority of the foundational work. I've continued developing these ideas as part of the Apart Lab Studio program, but the core insights and initial implementation were primarily their contributions. My motivation for publishing this article is to extend my own thinking on these concepts and to help disseminate what I believe are valuable ideas that deserve broader attention. I'm grateful to have been part of the original team and for their permission to build upon our shared work.

Introduction

As AI systems become increasingly powerful, ensuring their safe deployment remains a critical challenge. While alignment efforts strive to create inherently trustworthy systems, control mechanisms serve as crucial safeguards when alignment fails. Our work, developed as part of the Apart Research AI Control Hackathon, explores a particularly concerning failure mode within untrusted monitoring protocols: the possibility of collusion between AI systems. Our team investigated whether language models can coordinate their actions without direct communication by identifying and converging on Schelling points in coordination games. While our findings show that current models possess some basic coordination capabilities, these abilities appear limited and may not pose immediate risks to existing safety protocols.

Background

AI Control is a subfield of AI safety that aims to answer the question: “assuming you have a misaligned AI system, or one that could be misaligned, how can you safely get useful work out of it?”. While AI Alignment focuses on making AI systems behave the way we want them to, this does not always work perfectly. Control, therefore, promises to serve as an important line-of-defence, preventing bad outcomes even if Alignment fails.

In the event where we can’t fully trust an AI, there are a variety of proposed methods involving the use of a ‘monitor’ model to control a potentially harmful model. One such protocol is known as trusted monitoring. In this protocol, an "untrusted" model (referred to as the policy) generates outputs that might include harmful actions, while a "trusted" model (the monitor) serves as an oversight mechanism. The trusted monitor is chosen such that it is less powerful than the policy—sophisticated eno...

Posts

Wikitag Contributions

Comments