This work was supported through the MARS (Mentorship for Alignment Research Students) program at the Cambridge AI Safety Hub (caish.org/mars). We would like to thank Redwood Research for their support and Andrés Cotton for his management of the project. TLDR The field of AI control takes a worst-case scenario approach,...
This project was conducted as a part of a one-week research sprint for the Finnish Alignment Engineering Bootcamp. We would like to thank Vili Kohonen and Tyler Tracy for their feedback and guidance on the project. TLDR Research in AI control is tedious. To test the safety and usefulness of...
This work was supported through the MARS (Mentorship for Alignment Research Students) program at the Cambridge AI Safety Hub (caish.org/mars). We would like to thank Redwood Research for their support and Andrés Cotton for his management of the project.
The field of AI control takes a worst-case scenario approach, assuming that the most powerful AI models will be capable of and willing to subtly sabotage AI labs, and that efforts to align models will never work or will not be ready soon enough. High-stakes situations are defined as those in which a single sneaky attack could allow the model to avoid further oversight. If an AI manages to cause a so-called rogue deployment,... (read 3019 more words →)
This project was conducted as a part of a one-week research sprint for the Finnish Alignment Engineering Bootcamp. We would like to thank Vili Kohonen and Tyler Tracy for their feedback and guidance on the project.
Research in AI control is tedious. To test the safety and usefulness of a new protocol, AI control researchers manually write the prompts to maximize the LLM’s performance on an evaluation. There are two problems with this process: