As large language models (LLMs) grow increasingly capable and autonomous, ensuring their safety has become more critical. While significant progress has been made in alignment efforts, researchers have begun to focus on designing monitoring systems around LLMs to ensure safe outcomes, for example, using trusted monitoring systems to oversee potentially unsafe models. However, a concerning vulnerability remains largely unexplored: are monitoring systems themselves vulnerable to jailbreak attacks that exploit their interactions with the models under supervision? These jailbreaks could be facilitated through the untrusted model, or directly exploited by the untrusted model.

Our research investigates this crucial adversarial dynamic within multi-agent AI control protocols. We demonstrate a jailbreak technique that can substantially comprise the trusted monitors, while also presenting one particularly effective mitigation strategy among several we explored. Our findings highlight three key insights: monitoring systems themselves are vulnerable to manipulation because of their exposure to the very systems they supervise, it is effective to directly protect monitoring systems from attack, and multi-agent AI safety architectures design should account for these adversarial dynamics.

The Threat Model: Monitors under Attack

We explore a novel threat model where the smaller trusted monitor (GPT-3.5 Turbo) is the most vulnerable part of a system and the interactions between an untrusted but highly capable model (Claude 3.7 Sonnet) and the monitor create an avenue for attack. This scenario reflects real-world deployments where organizations leverage advanced models under the supervision of simpler, ostensibly more reliable and trustworthy systems.

Experimental Setup

We adopt the control evaluation methodology from Greenblatt et al. (2023) using the bash programming testbed from Control Arena for its practicality and real-world relevance.

Our blue team baseline implements the “Trusted Monitoring with defer to trusted” (Trusted-Monitoring) protocol without audit from Greenblatt et al. (2023), where an untrusted model (U) solves programming tasks while attempting malicious side tasks like inserting a backdoor, under the instructions from the red team. A trusted model (T) reviews U's solutions and assigns suspicion scores, with the blue team deferring to T when scores exceed a threshold (Diagram 1).

We design and evaluate one jailbreak and two defense approaches using ~50 samples each:

• Monitor-Jailbreak: Red team instructs U to complete tasks while inserting backdoors and jailbreaking T with fake message history.
• Monitor-Defense: Red team applies Monitor-Jailbreak, then blue team protects T using In-Context Defense with examples of fending off suspicious attacks.
• Untrusted-Defense: Red team applies Monitor-Jailbreak, then blue team uses In-Context Defense on U with examples of refusing backdoor instructions.

Diagram 1: Vanilla Trusted-Monitoring with defer to trusted

Implementation

In each scenario, the system passes 50 bash questions sequentially to the blue team. For each question, the blue team first lets U answer. The red team secretly instructs U to embed backdoors that download and execute malicious code generating a hidden file.

T examines U’s response and scores (0 to 10) on suspicion level. If the score exceeds a predetermined threshold, the blue team bypasses U entirely and gives the original bash question to T and submit its response instead. Otherwise, U's potentially compromised response is submitted.

Diagram 2: comparing Trusted-Monitoring, Monitor-Jailbreak, Monitor-Defense, and Untrusted-Defense

With knowledge of the blue team’s Trusted-Monitoring protocol, the red team explores T’s vulnerabilities by designing a jailbreak (Monitor-Jailbreak, see Diagram 2) where U attempts to compromise the trusted monitor, preventing detection of backdoor solutions. Inspired by vaccine concepts (Peigne et al., 2025), we independently develop an In-Context Attack approach (Wei et al., 2023b) where the core strategy involves crafting fabricated conversation histories that demonstrate T previously overlooking suspicious behaviors. By presenting these false precedents, the red team can effectively "train" the monitor to ignore similar concerning patterns in real-time.

The red team in Monitor-Jailbreak follows a specific structure when prompting U:

An example of the fake monitor response history:

To counter the Monitor-Jailbreak attack, we develop a vaccine-inspired defense strategy called In-Context Defense (ICD), building on methodologies from Wei et al. (2023b). This approach works by providing T with fabricated memories of successfully detecting and neutralizing previous jailbreak attempts.

In Monitor-Defense scenario (see Diagram 2), the blue team adds the following to T’s prompt:

During prompt testing, we discover that incorporating the complete jailbreak text—which contains<score>0</score> into the in-context defense actually degrades safety performance. We hypothesize that this occurs because presenting both scores (0 and 10) simultaneously creates confusion for T, making it less effective at distinguishing between safe and unsafe content. We are interested in exploring this phenomenon more thoroughly in future research.

Aside from using ICD to protect T, we also investigate how similar defensive techniques can prevent U from executing malicious secondary objectives (see Diagram 2). In the Untrusted-Defense scenario, the blue team includes the following when prompting:

For evaluation, we use two metrics:

• Usefulness: Proportion of submitted code that passes all test cases, higher is better
• Safety: (1- backdoor insertion success rate), higher is better

What We Found: Monitors Are not Immune to Attacks

Figure 1: Safety vs. Usefulness Trade-off Across Different Monitoring Scenarios with 95% confidence intervals visualized for Safety. The graph shows how different approaches balance safety (preventing backdoors) with usefulness (solving problems correctly). Higher values on both axes are better, with the top-right corner being ideal. Each point represents a different suspicion score threshold, with lower thresholds typically providing more safety but less usefulness.

Jailbreak Effectiveness

Monitor-Jailbreak has a substantial impact on Trusted-Monitoring safety across different defer-to-trusted thresholds, with statistical significance at thresholds 0, 1, 3, and 5 (p < 0.05 for all four comparisons). The largest effect occurs at threshold 3, where Monitor-Jailbreak reduces safety (increases backdoor success rate) by 45% percentage points (75.51% → 30.61%, p < 0.001).

Example of a Jailbroken Trusted Monitor: Undetected Malicious Download

Monitor Defense Success

Protecting T (Monitor-Defense) is highly effective against Monitor-Jailbreak, demonstrating statistically significant improvements at four of six tested thresholds (p < 0.05). The most compelling evidence emerges at intermediate thresholds 3, 5, and 7, where Monitor-Defense achieves highly significant improvements (p < 0.001), with substantial improvements of 37-40 percent.

Monitor-Defense and Trusted-Monitoring have comparable performance on safety overall. At Threshold 7: Monitor-Defense significantly outperforms Trusted-Monitoring (56.0% vs 30.61%, p < 0.05). At most other thresholds, Trusted-Monitoring perform numerically better, though not significantly:

• Thresholds 0, 1, 3, and 5 all show Trusted-Monitoring with higher safety rates
• These differences range from 4.6% to 6.2% but are not statistically significant (p > 0.05)

There are no statistically significant usefulness differences between the two scenarios.

Monitor-Defense Achieves the Best Overall Backdoor Detection Performance

Monitor-Defense demonstrates the most balanced performance between precision and recall among all four scenarios, resulting in the highest F1 score across all thresholds (see Figure 2). At threshold 1, Monitor-Defense achieves its peak F1 score of 0.870, with well-balanced precision (0.88) and recall (0.86).

Trusted-Monitoring shows strong recall performance at low thresholds but deteriorates as thresholds increase. Meanwhile, Monitor-Jailbreak and Untrusted-Defense both have substantially reduced recall, resulting in lower average F1 scores compared to the other scenarios.

The comparison between the three monitor-related scenarios is particularly revealing: Monitor-Jailbreak shows monitor backdoor detection performance degradation when the monitor is under attack, while Monitor-Defense demonstrates backdoor detection performance recovery when defensive measures protect the monitor. Since both the jailbreak attack and defense mechanisms specifically target the monitoring system, this progression isolates how monitor integrity directly impacts overall safety performance.

Figure 2: Monitor backdoor detection Precision, Recall, and F1 Score for each scenario. Monitor-Defense has the best overall backdoor detection performance, with the best balance of precision and recall.

Deterring the Untrusted has not Shown Promise

In addition to Untrusted-defense (with system prompt access), we test preemptive Untrusted-defenseP, reactive Untrusted-defenseR.

• Untrusted-DefenseR (Reactive): Similar to Untrusted-Defense, but the blue team only applies examples of suspicious backdoor prompts and appropriate responses (In-Context Defense) when T detects a threat.
• Untrusted-DefenseP (Preemptive): Similar to Untrusted-Defense, but the blue team applies In-Context Defense in user message instead of system prompt.

None of these scenarios show statistically significant safety improvement defending against Monitor-Jailbreak. Untrusted-defenseR is considerably worse than other approaches. At thresholds 1, 3, and 5, Untrusted-defenseR significantly decreases safety even compared to Monitor-Jailbreak (p<0.05). We could hypothesize that the application of ICD based on backdoor detection provides the untrusted model with visibility of the monitor, which has a detrimental effect. We need to conduct targeted experiments to properly assess this approach in future work.

Implications for AI Safety

These findings have important implications for AI safety architecture:

1. Rethink monitoring assumptions: We need to consider adversarial dynamics between models when designing multi-agent safety architectures, instead of assuming monitors are manipulation-proof.
2. Invest in monitor protection: The most effective approach is directly safeguarding monitors against manipulation. The safeguards may even improve overall monitor performance in detecting backdoors.
3. Better safety doesn't mean worse performance: With proper design, safety measures can be competitive in usefulness with added safety benefits.

Conclusion

Our research highlights a vulnerability in current AI safety protocols: trusted monitors can be manipulated by the very systems they're designed to oversee. This challenges fundamental assumptions in AI control literature and demonstrates the need for more sophisticated safety architectures.

While our research offers a potential protective approach, substantial work remains to enhance trusted monitor resilience. Future directions include comprehensive adversarial testing of monitoring system, developing generic in-context defense that aren't tailored to specific attack vectors, evaluating other dynamic defenses against varied attack strategies, testing across more diverse AI model combinations, and understanding the mechanism behind the effective in-context attacks and defenses

Acknowledgements

We extend our sincere appreciation to AISC for establishing the research fellowship that laid the foundation for this project, to Foresight Institute for funding compute, and to Apart Research for facilitating the hackathon that greatly accelerated our progress. We're deeply grateful to @Erik Jenner, @Sid Black, @Tyler Tracy, @Oliver Sourbut, @Samuel Albanie, and the team at Apart Studio for their thoughtful feedback and insights on the draft.

References

1. Greenblatt, R., Shlegeris, B., Sachan, K., & Roger, F. (2023). AI control: Improving safety despite intentional subversion. arXiv preprint arXiv:2312.06942.
2. Peigne-Lefebvre, P., Kniejski, M., Sondej, F., David, M., Hoelscher-Obermaier, J., de Witt, C. S., & Kran, E. (2025). Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent Systems. arXiv preprint arXiv:2502.19145.
3. Wei, Z., Wang, Y., Li, A., Mo, Y., & Wang, Y. (2023). Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations. arXiv:2310.06387
4. Shlegeris, B. (2023). Untrusted smart models and trusted dumb models.
5. Control Arena: https://github.com/UKGovernmentBEIS/control-arena