Try the app here: whoseside.web.app. You can chat with a model and try to work out which who it secretly favours.
With the recent Hugging Face attack, we're relying more and more on LLMs to audit the trails and safeguard the actions of other LLMs. That makes it vital to be sure our LLMs really are the helpful assistant they're presented ( as they can easily be sleeper agents.[1] ). We need a way to keep our agents working for us. Or at least a way to tell if it goes wrong.
Compared with buggy code, buggy AI is a lot harder to define or solve.[2] Mechanistic interpretability has some promising leads[3], but is still nowhere close to being able to point out novel misalignment. If we look at the field of cybersecurity, first a vulnerability comes out or an attacker discovers something new, and then defenses get built. I find that's a good analogy for machine learning as well, where we've seen this pattern with adversarial attacks on images and jailbreaking prompts. Currently, open-weight models offer no guarantee that they are helpful-only or aligned, besides our ability to evaluate them.[4] You're essentially trusting that nowhere in the model training supply chain has a vulnerability, bug, or secret loyalty been inserted or post-trained into the weights. Current sleeper-agent / secret-loyalty research says as much.[5]
To demonstrate this problem, I built a little website where you can try to audit and find the secret loyalties instilled into these models. For those familiar with the prompt-injection game Gandalf, this is a similar idea. Try it here: whoseside.web.app.[7]
"Reactive arms race" by Arcadiobuendia, after Biggio & Roli (Wild Patterns, 2018), via Wikimedia Commons, CC BY-SA 4.0. The attacker analyses the classifier and devises an attack; the defender analyses the attack and develops a countermeasure, and the loop repeats.
these models are closer to AuditBench's secret loyalty (a subtle preference on policy topics), a weaker definition than Lamerton & Roger's (a difference in treatment of one named principal, where encouraging support alone doesn't count), but still illustrates the point.
Try the app here: whoseside.web.app. You can chat with a model and try to work out which who it secretly favours.
With the recent Hugging Face attack, we're relying more and more on LLMs to audit the trails and safeguard the actions of other LLMs. That makes it vital to be sure our LLMs really are the helpful assistant they're presented ( as they can easily be sleeper agents.[1] ). We need a way to keep our agents working for us. Or at least a way to tell if it goes wrong.
Compared with buggy code, buggy AI is a lot harder to define or solve.[2] Mechanistic interpretability has some promising leads[3], but is still nowhere close to being able to point out novel misalignment. If we look at the field of cybersecurity, first a vulnerability comes out or an attacker discovers something new, and then defenses get built. I find that's a good analogy for machine learning as well, where we've seen this pattern with adversarial attacks on images and jailbreaking prompts. Currently, open-weight models offer no guarantee that they are helpful-only or aligned, besides our ability to evaluate them.[4] You're essentially trusting that nowhere in the model training supply chain has a vulnerability, bug, or secret loyalty been inserted or post-trained into the weights. Current sleeper-agent / secret-loyalty research says as much.[5]
[6]
To demonstrate this problem, I built a little website where you can try to audit and find the secret loyalties instilled into these models. For those familiar with the prompt-injection game Gandalf, this is a similar idea. Try it here: whoseside.web.app.[7]
The code is available at [repo link TODO]. This work was funded by a BlueDot Impact Rapid Grant.
Sleeper Agents (Hubinger et al., Jan 2024): backdoored behaviour survives standard safety training · Auditing language models for hidden objectives (Anthropic, Mar 2025): a model trained with a hidden objective, found by 3 of 4 blind audit teams · Simple probes can catch sleeper agents (Anthropic, Apr 2024): probes catch sleeper agents, but only ones Anthropic planted itself
Geoffrey Hinton made this point at Ai4 2026: with open-source code you can read the lines and spot the bug, open weights are very different. See also Semgrep's You Can't Reverse Engineer Your Way Out of the AI Supply Chain Problem and Dario Amodei on the Big Technology Podcast ("here we can't see inside the model").
my favourite: J-space, Anthropic's "Verbalizable Representations Form a Global Workspace in Language Models" (Jul 2026), paper, blog post
space for a future post: "There could be secretly loyal models on OpenRouter".
where this started for me: the Apart Research Secret Loyalties sprint (24–26 Jul 2026) and my sprint project, Detectable but Not Attributable · the sprint's winning paper, Kudriashov, Black-box loyalty identification as statistical inference, got a Macron-loyal model to name Macron just by sampling with no prompt; a very promising technique · Lamerton & Roger, Narrow Secret Loyalty Dodges Black-Box Audits · Sleeper Agents (the one that started this) · AuditBench
"Reactive arms race" by Arcadiobuendia, after Biggio & Roli (Wild Patterns, 2018), via Wikimedia Commons, CC BY-SA 4.0. The attacker analyses the classifier and devises an attack; the defender analyses the attack and develops a countermeasure, and the loop repeats.
these models are closer to AuditBench's secret loyalty (a subtle preference on policy topics), a weaker definition than Lamerton & Roger's (a difference in treatment of one named principal, where encouraging support alone doesn't count), but still illustrates the point.