UPDATE: Recent work with improved AF and compliance gap classifiers disagrees with our results. We recommend using the improved classifiers. Summary We wanted to briefly share an early takeaway from our exploration into alignment faking: the phenomenon appears fairly rare among the smaller open-source models we tested (including reasoning models)....
This research was conducted at AE Studio and supported by the AI Safety Grants program administered by Foresight Institute with additional support from AE Studio. Summary In this post, we summarize the main experimental results from our new paper, "Towards Safe and Honest AI Agents with Neural Self-Other Overlap", which...
Figure 1. Image generated by DALL·E 3 to represent the concept of self-other overlap Many thanks to Bogdan Ionut-Cirstea, Steve Byrnes, Gunnar Zarnacke, Jack Foxabbott and Seong Hah Cho for critical comments and feedback on earlier and ongoing versions of this work. This research was conducted at AE Studio and...
Many thanks to Samuel Hammond, Cate Hall, Beren Millidge, Sumner Norman, Steve Byrnes, Lucius Bushnaq, Joar Skalse, Kyle Gracey, Gunnar Zarncke, Ross Nordby, David Lambert, Simeon Campos, Bogdan Ionut-Cirstea, Ryan Kidd, and Eric Ho for critical comments and suggestions on earlier drafts of this agenda, as well as Philip Gubbins,...
This is a talk I gave at the recent AI Safety Europe Retreat (AISER) on my research on obtaining insights from the cognitive science of empathy and applying them to RL agents and LLMs. Talk link: https://clipchamp.com/watch/6c0kTETRqBc Slides link: https://bit.ly/3ZFmjN8 Talk description: I begin by presenting a short review on...
TL;DR: We ran a competition to encourage people to stop procrastinating on their AI Safety projects. We had 76 applicants out of which over 40% (31 participants) completed their goal by the end of October and were added to a draw to win a monetary prize. The vast majority of...
The AI Safety Nudge Competition aims to encourage people to do things related to AI Safety today instead of procrastinating it into the future by allowing them to enter into a draw if they complete the goal that they set for themselves. Participants defined a specific goal for themselves, examples...