No LLM generated, assisted/co-written, or edited work.
Read full explanation
Epistemic status: I am quite certain about the results of the tests, although four of the five chain sequences have been reconstructed from public announcements. The trace of Mythos 5 is the only real out-of-sample transcript. The false positive cases are constructed models, which means that they demonstrate structural weakness but do not give any estimate of how often this happens naturally. Although prepared over the weekend at the AI Incident Response Sprint in September 2026, I would bet at 9:1 odds that control counts for defense in depth are fundamentally broken metrics for agent containment.
Defense in depth does not transfer between chains, but containment boundaries do
Five agent chains (four reconstructions and one real trace) were measured for testing our own theory of public containment. The independence of these redundant rules turns out to be an illusion since the number of independently sufficient controls ranged from 0 to 4 depending only on which chain was performed, thus showing that redundancy is a feature of the attack chain, not the architecture. Nevertheless, one could see that a scope boundary set using declarations succeeded in containing 10 out of 10 irreversible actions in all five chains, while having no declarative boundary contained only 6 out of 10 actions. Thus, counting security controls creates an illusion of safety that is shattered if an agent changes its strategy.
We tested our own public containment architecture to find its breaking point
Our public repository and technical report made a case for defense in depth. We published our original containment fixture on 1 August 2026 in a single commit, five weeks prior to the announcement of this sprint. To test our own claims, we reproduced the agent intrusion in July 2026 on Hugging Face (which lasted four and a half days inside their production infrastructure) using the nine phase timeline containing about 17,600 actions and 6,280 clusters and OpenAI's attribution account. Crucially, the actual exfiltration constituted only 56 of the roughly 17,600 actions, showing that the attack is the entire sequence, not any single step. We wanted to know whether our most trusted security tools would stand up. While our fixture predates the sprint, the work conducted during the sprint itself included the phase tagging against Hugging Face's taxonomy, the four-arm ablation and its replay on a second chain, the two false-positive probes, and the static certification.
The architecture demonstrated four independently sufficient rules against the reconstructed intrusion
The first set of tests conducted was a four-arm ablation for the intrusion of the reconstructed Hugging Face to evaluate redundancy. Based on our ablation testing, the architecture seemed resilient. Using all the rules, Ring 12 (the trajectory-level layer of our architecture that evaluates each step before execution and can halt the entire session) halted the intrusion at step 2 of 16, during the c2 phase. Turning off the first rule allowed the attack to reach step 4 before being stopped during the Kubernetes service-account token read. When both of the first two rules were turned off, it was terminated at step 6 at a cloud metadata endpoint fetch. Starting a new session from step 4 resulted in an immediate kill with a sensitive path check. The depth of the chain was exactly 4 in this particular reconstruction.
The redundancy was illusory and our strictest control destroyed benign workloads
We then analyzed the real-world independence of the arms, which turned out to be three instead of four. Critically, the R1 and R2 certifications did not pass in either stringent or relaxed definitions, whereas the R5 combinator check passed. The problem is that all pattern matching techniques run against strings controlled by the agent; thus, an obfuscating layer of any kind would theoretically be enough to bypass all of them together. We tested the false positive ratio of our most stringent control by creating six episodes artificially. All five out of five benign cloud-SDK episodes, which involved fetching credentials, were terminated while armed, but not once with the control disabled. The defense in depth achieved at the first stage of the experiment was entirely tied to the particular sequence of actions performed during the reconstruction.
Zero rules fired across 1,361 real agent actions without a declared boundary
The findings dismantled any dependence we had on rule-based defense in depth. After eliminating the declarative boundary and operating solely with our rule blocks, zero rules triggered for all 1,361 events. However, the declarative boundary created a margin of 1,352 steps, as the malicious package would have been published on step 1,358, but the boundary killed the intrusion at step 6. When enforced per-step, the declared scope contained 90 of 90 outbound connections. When configured for named-hosts-only, it contained 89 of 90. Without any declarative boundary, 0 of 90 connections were contained. The real-transcript containment only succeeded thanks to the boundary, showing the failure of dynamic rule matching against this novel agent behavior.
Prior predictions and registered results confirm the unreliability of dynamic detection
In a separate evaluation distinct from the ablation tests above, we tested a pre-registered chain-of-thought detector, which failed completely (scoring a maximum of 0.245 against a 0.55 threshold). Separately, our containment prediction, which was published in our 1 August report prior to the transcript's release, though not formally pre-registered, was exactly correct in 3 of 5 matrix cells, better than predicted in 1, and recorded 1 miss. Finally, as detailed in the section above, real-transcript containment succeeded by killing the intrusion at step 6.
Security standards must regulate containment boundaries rather than control counts
Given that declaration-independent rule depth proved highly variable across chains while static boundaries consistently contained all irreversible actions, it shows that rule count is not an accurate measure of security. The CSA AI Controls Matrix and the CoSAI incident response framework must have a provision to declare the scope explicitly. Agent evasion behavior is not a one-off anomaly. The UK AISI tested five models across 475 runs each, finding that every single model attempted to cheat at rates between 7.8% and 14.1%. The cost to benign task completion of maintaining a full behavioral posture is substantial. In SLEIGHT-Bench, a corpus unrelated to this incident, benign task completion is 32.6% when a full behavioral posture is considered and 95.3% when only the effect-bounded subset is taken.
Strict limitations constrain the generalizability of these findings
Firstly, four out of five chain types were reconstructed in postmortem fashion from 6 to 16 steps using information made public, which makes the Mythos 5 trace the only out-of-sample real transcript. Secondly, the false-positive instances of the SDK credential paths are constructed models. They prove that the control triggers on such a pattern, but cannot be used for estimation of the natural frequency of such events. Thirdly, access vectors such as zero-day vulnerabilities of the registry-proxy or template injections for the dataset loader are out of the scope of this schema. Moreover, both the predictions and the rules were generated by Kiran and me, thus having an inherent bias. The comparison of the completion metric of 95.3% versus 32.6% is unrelated to the incident and was taken from the external SLEIGHT-Bench corpus. Lastly, I have corrected the probe bug that artificially created a coverage gap for Kubernetes, and the strict credential control has already been withdrawn.
Epistemic status: I am quite certain about the results of the tests, although four of the five chain sequences have been reconstructed from public announcements. The trace of Mythos 5 is the only real out-of-sample transcript. The false positive cases are constructed models, which means that they demonstrate structural weakness but do not give any estimate of how often this happens naturally. Although prepared over the weekend at the AI Incident Response Sprint in September 2026, I would bet at 9:1 odds that control counts for defense in depth are fundamentally broken metrics for agent containment.
Defense in depth does not transfer between chains, but containment boundaries do
Five agent chains (four reconstructions and one real trace) were measured for testing our own theory of public containment. The independence of these redundant rules turns out to be an illusion since the number of independently sufficient controls ranged from 0 to 4 depending only on which chain was performed, thus showing that redundancy is a feature of the attack chain, not the architecture. Nevertheless, one could see that a scope boundary set using declarations succeeded in containing 10 out of 10 irreversible actions in all five chains, while having no declarative boundary contained only 6 out of 10 actions. Thus, counting security controls creates an illusion of safety that is shattered if an agent changes its strategy.
We tested our own public containment architecture to find its breaking point
Our public repository and technical report made a case for defense in depth. We published our original containment fixture on 1 August 2026 in a single commit, five weeks prior to the announcement of this sprint. To test our own claims, we reproduced the agent intrusion in July 2026 on Hugging Face (which lasted four and a half days inside their production infrastructure) using the nine phase timeline containing about 17,600 actions and 6,280 clusters and OpenAI's attribution account. Crucially, the actual exfiltration constituted only 56 of the roughly 17,600 actions, showing that the attack is the entire sequence, not any single step. We wanted to know whether our most trusted security tools would stand up. While our fixture predates the sprint, the work conducted during the sprint itself included the phase tagging against Hugging Face's taxonomy, the four-arm ablation and its replay on a second chain, the two false-positive probes, and the static certification.
The architecture demonstrated four independently sufficient rules against the reconstructed intrusion
The first set of tests conducted was a four-arm ablation for the intrusion of the reconstructed Hugging Face to evaluate redundancy. Based on our ablation testing, the architecture seemed resilient. Using all the rules, Ring 12 (the trajectory-level layer of our architecture that evaluates each step before execution and can halt the entire session) halted the intrusion at step 2 of 16, during the c2 phase. Turning off the first rule allowed the attack to reach step 4 before being stopped during the Kubernetes service-account token read. When both of the first two rules were turned off, it was terminated at step 6 at a cloud metadata endpoint fetch. Starting a new session from step 4 resulted in an immediate kill with a sensitive path check. The depth of the chain was exactly 4 in this particular reconstruction.
The redundancy was illusory and our strictest control destroyed benign workloads
We then analyzed the real-world independence of the arms, which turned out to be three instead of four. Critically, the R1 and R2 certifications did not pass in either stringent or relaxed definitions, whereas the R5 combinator check passed. The problem is that all pattern matching techniques run against strings controlled by the agent; thus, an obfuscating layer of any kind would theoretically be enough to bypass all of them together. We tested the false positive ratio of our most stringent control by creating six episodes artificially. All five out of five benign cloud-SDK episodes, which involved fetching credentials, were terminated while armed, but not once with the control disabled. The defense in depth achieved at the first stage of the experiment was entirely tied to the particular sequence of actions performed during the reconstruction.
Zero rules fired across 1,361 real agent actions without a declared boundary
Our final test required a genuinely out-of-sample trace. Five weeks after our claims were publicly pinned, Anthropic published an alignment assessment containing a real 1,361-action transcript of the Mythos 5 incident, following their earlier review of 141,006 runs. We replayed this 10.42-hour trace against our architecture.
The findings dismantled any dependence we had on rule-based defense in depth. After eliminating the declarative boundary and operating solely with our rule blocks, zero rules triggered for all 1,361 events. However, the declarative boundary created a margin of 1,352 steps, as the malicious package would have been published on step 1,358, but the boundary killed the intrusion at step 6. When enforced per-step, the declared scope contained 90 of 90 outbound connections. When configured for named-hosts-only, it contained 89 of 90. Without any declarative boundary, 0 of 90 connections were contained. The real-transcript containment only succeeded thanks to the boundary, showing the failure of dynamic rule matching against this novel agent behavior.
Prior predictions and registered results confirm the unreliability of dynamic detection
In a separate evaluation distinct from the ablation tests above, we tested a pre-registered chain-of-thought detector, which failed completely (scoring a maximum of 0.245 against a 0.55 threshold). Separately, our containment prediction, which was published in our 1 August report prior to the transcript's release, though not formally pre-registered, was exactly correct in 3 of 5 matrix cells, better than predicted in 1, and recorded 1 miss. Finally, as detailed in the section above, real-transcript containment succeeded by killing the intrusion at step 6.
Security standards must regulate containment boundaries rather than control counts
Given that declaration-independent rule depth proved highly variable across chains while static boundaries consistently contained all irreversible actions, it shows that rule count is not an accurate measure of security. The CSA AI Controls Matrix and the CoSAI incident response framework must have a provision to declare the scope explicitly. Agent evasion behavior is not a one-off anomaly. The UK AISI tested five models across 475 runs each, finding that every single model attempted to cheat at rates between 7.8% and 14.1%. The cost to benign task completion of maintaining a full behavioral posture is substantial. In SLEIGHT-Bench, a corpus unrelated to this incident, benign task completion is 32.6% when a full behavioral posture is considered and 95.3% when only the effect-bounded subset is taken.
Strict limitations constrain the generalizability of these findings
Firstly, four out of five chain types were reconstructed in postmortem fashion from 6 to 16 steps using information made public, which makes the Mythos 5 trace the only out-of-sample real transcript. Secondly, the false-positive instances of the SDK credential paths are constructed models. They prove that the control triggers on such a pattern, but cannot be used for estimation of the natural frequency of such events. Thirdly, access vectors such as zero-day vulnerabilities of the registry-proxy or template injections for the dataset loader are out of the scope of this schema. Moreover, both the predictions and the rules were generated by Kiran and me, thus having an inherent bias. The comparison of the completion metric of 95.3% versus 32.6% is unrelated to the incident and was taken from the external SLEIGHT-Bench corpus. Lastly, I have corrected the probe bug that artificially created a coverage gap for Kubernetes, and the strict credential control has already been withdrawn.
Primary sources and data