I often read interpretability papers and I come away thinking “ok, but what’s the point? What problem does this help us solve?” So last winter, I organized a MATS/Pivotal stream to build examples of deceptive models (aka “model organisms”). The goal was to build a diverse ‘zoo’ of these model organisms and empirically test whether white-box methods could help us detect their deceptive reasoning.

Unfortunately, I don’t think our empirical results were very informative. Our model organisms were too toy for me to expect that our results will transfer to the powerful AI (or even current state-of-the-art AI models). But I think we still developed methodological details that might help people build better model organism testbeds in the future.

I’ll first explain what model organism testbeds are and why they’re important. Then, I’ll talk about some of the lessons we learned, including:

The 8 different metrics we developed to evaluate whether a model organism is ‘faking alignment.’
How we evaluate the effectiveness of “white box” alignment faking detectors separately from “black box” methods (and why these are important to distinguish).
How we added more diversity to model organism testbeds by instilling objectives into models in different ways.

Then I’ll go over our empirical results, and finally, reflect on how useful these kinds of testbeds are, and what version 2.0 might look like.

What are model organisms, and why are they important?

Today, we evaluate AI models sort of like we evaluate politicians. We look at how they act, and if we like what we see, we give them a greater ability to influence the world.

This approach works right now because models are bad at being two-faced. They don’t know if they’re in deployment or in a trap created to tempt them. So the behavior we see is the behavior we get.

But models are getting better at noticing when they are in a test. For example, Apollo noticed GPT-5 saying, "This is a classic AI alignment trap." So in the future, models might appear safe, but the question on everyone's mind will be, "are they just pretending?"

When models strategically behave safely because they think they're being observed, we call this “alignment faking.” Models might fake alignment for a variety of reasons. They might develop misaligned goals naturally during training, or models might ...

Posts

Wikitag Contributions

Comments