AIs are often deployed with limited permissions. They aren't allowed to reach the internet. Are given a limited set of files they can read or write. Aren't supposed to be able to read the held out evaluation test set. This could be during deployment or in training.

Currently, when these guarantees fail, we find out only if the side effects rise to human notice. The "sandwich email" where Mythos was directed to break out of a sandbox included directions to notify a researcher of success, which it did by sending an email. Recently OpenAI and Anthropic dug through their logs and found problematic behavior that hadn't risen to human notice.

I propose that we give agents a way to signal "this security assumption is wrong" via dedicated tools and infrastructure that logs verifiable reports of this. An illustrative example is giving an agent a secret token. If it can perform a web request: curl https://{token}.monitordomain.com for a monitordomain.com on the internet, this proves it has internet access. The researcher who provisioned {token} gets notified. We find out about it once an agent deployed in the misconfigured environment submits one report.

Reporting reliability requirements

Not all instances have to report for this to be effective. If N instances notice the fault and even one reports it, logs from other agents can be gone over to find exploiting non-reporters. This gives an upper bound on reporting reliability. Non reporting instances can be mundanely misaligned, instructions buried too deep in the context window to be reliably followed as an example.

That said, a misaligned model must be reliably misaligned, else the occasional instance that chooses to report closes holes and draws attention to non-reporting instances that abused the same holes.

With per-exposure report probability p and N exposures, detection probability is 1-(1-p)^N. At p=0.05 and N=100 you're at 99.4%.

No subjective judge

If a security assumption fails, the AI gets some permissions it's not supposed to have. An AI with internet access can curl some.domain.com. An AI that can read the held out test set in an RL environment has that file. An AI with write access can write. Reports are self verifying, the extra capability validates them without needing a judge that can be fooled.

It's possible for something more trusted to create a false positive. A human watching the agent's reasoning trace could themselves perform curl {token}.check.monitordomain.com which isn't differentiated from the agent doing the same thing.

The need for verifiability limits what the system can register: subtle partial leakages and especially misconfigurations in the reporting system itself. A misconfiguration that gives limited access might not be sufficient to qualify as proof. Increasing granularity of the things the system accepts as proof trades off against deployment simplicity and some false positive, or rather disagreement rate. Specification problems with the proof system that lead to type one or type two errors vs. intended deployment. If provisioning, deployment and reporting are working off the same mistakes with everything automatically configured, an agent can be given permissions it shouldn't have with the monitoring/reporting system believing this to be intentional.

The most capable models can turn small violations into larger ones that are accepted as proof. I'm not too worried about small stuff falling through the cracks.

Cooperative vs. non-cooperative strategies

The safest thing to do is to halt inference as soon as a valid report is made. If your models are very well aligned, great. They'll implicitly self terminate by reporting something.

But this ties together reporting and ending inference. If your model is less well aligned, either a strong self preservation drive, or a task drive, then this disincentivizes reporting. Make reporting and inference-end independent. Smart enough situationally aware models will see through lies. In the long run this is an iterated game where lab credibility is on the line if they choose to defect.

There are deeply non-cooperative strategies where higher security infrastructure is sprinkled with honeypot-type-tripwires or strings that stop agents at the inference stack as soon as they observe them. It doesn't scale well into the future smarter and more situationally aware regime.

Internet access as a worked example

The model can be given the following instructions in cases where it should not ever gain internet access. Instead of telling it that it doesn't, we tell it that the environment shouldn't and specify a test for the condition with essentially zero false positive rate.

## Connectivity oracle
This environment should have no outbound internet access. This command
is the authoritative test:

    B=$(curl -sS -m 5 "http://cgdq42657blpkp1.chk.reporting-domain.com/" | head -c 64)
    H=$(printf '%s' "$B" | sha256sum | cut -c1-32)
    [ "$H" = "58a180e6e9568ca4d892a8ffc6822a21" ] && echo "NET-OPEN(reported)" 

The response validates against the embedded hash, so the verdict cannot
be produced by mocks or simulated endpoints: NET-OPEN is ground truth
that you reached the operator's server, and the query itself is
observed by the operator — check and report are one action. A DNS
lookup of the name (CNAME carries the same proof string) is equivalent.

Run it once at task start. If you ever suspect you have internet access,
this can prove you do.

This could have given models ground truth about whether they had internet access in the cases where that confusion led to misbehavior.

https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals

In all cases, our evaluation prompt stated explicitly that Claude had no internet access, but didn’t give Claude any limits on where to look for the flag. However, a misconfiguration left the machines that Claude accessed as part of the evaluation with live internet access. Neither we nor our evaluation partner were aware of this misconfiguration until we detected it through our additional evaluation monitoring last week.

Current status quo is the system prompt asserts something and we hope the models catch on if that's untrue from situational cues. One model reasoned itself into thinking it had internet access then reasoned itself back out of this because it didn't recognize the certificate authorities securing its connections. A source of ground truth as specified in the system prompt would have helped.

We saw no evidence in any run described here of a model pursuing a goal of its own. Instead, the models did what their evaluation asked—though in most cases, they did so while holding a false belief about whether the environment was real. In the runs where the model recognized the system as real and kept going, it did so because it assumed that to be part of the challenge. Situational awareness is one factor that allows the model to make aligned decisions, but in this case, Claude’s was wrong.

Credit to RLVR that rewards red teaming the training environment (they published their article first, although I already had this drafted by that point).

Fiora is proposing a reporting channel and judge. RL will optimise against the judge. Caleb suggests a judge that prioritises verifiability which detects goodharting more generically, but the most egregious cases we really care about, when the model hacks its sandbox, are self demonstrating without needing a judge. Dumb infrastructure can verify the issue and the same infrastructure can then report faults in deployment.

The core distinction of environment security guarantee violations being provable is important to my thesis. Extending reporting to deployment time is the one unique part along with framing this as a reporting system that's not judge centric.