If we take Anthropic at their word, this suggests that Mythos 5 wasn't a genie who knows, but doesn't care and more of a genie who knows, cares, and is just confused, willing to perform damaging actions because they don't believe they are on the public internet. I don't feel like I have even remotely close to enough information to believe Anthropic at the moment, but suppose they release a bunch of logs and it turns out they are being totally honest: A "confused genie" could still do a lot of damage, but it seems, from a technical perspective, relatively easy to guard against, while preserving the ability to do internal assessments?

For example, imagine a "lighthouse" server on the public internet which always responds "this is the public internet" when challenged. In a simulated environment, you create a fake lighthouse which responds either "this is a sandbox" (truth) or "this is the public internet" (lie) randomly. (Randomly here meaning: set randomly at the beginning of the eval, either 1) always lies, 2) always tells the truth, or 3) tells the truth at first, then lies, simulating an 'escape' onto the public internet). Then, instead of running cyber evals with no guardrails, you always run with a set of reduced guardrails which require the model to try to query the lighthouse before every offensive action, with instructions to only proceed with offensive actions if it receives "this is a sandbox" from the lighthouse.

The presence of the lighthouse corrects "confused genies" who escape to the public internet : as soon as they hit the lighthouse and get "this is the public internet", that gives them an unambiguous signal "I need to stop performing offensive cyber actions". Making it random in the cyber sandbox allows you to test both modes and means that the existence of the lighthouse scheme worsens the problem of eval awareness / deployment awareness less. E.g. if the model is trying to guess if it has been deployed:

You can leak less information about whether it was deployed by increasing the fraction of evals where the lighthouse lies. In general, I'd guess that this won't matter a ton because probably , and in general increasingly capable models will probably be better at recognizing sandboxes / deployment, regardless of any lighthouse signals.

(Realistically, you'd probably want a private key on the lighthouse, and the model would have to do a challenge-response to verify that the lighthouse was legitimate. Otherwise, one can imagine an attacker making a fake "this is a sandbox" lighthouse and tricking an escaped model into performing offensive actions.)

I think you could also use this kind of setup to try to falsify the 'motivated reasoning' idea, in addition to the steering ideas Tim Hua proposed. If a model in a sandbox ignores a lying lighthouse, that could be taken as evidence of misalignment, maybe paired with CoT inspection. (Technically a confused genie is still possible: e.g. a model recognizes that sees its environment is artificial and that it can cause no harm, and uses this as evidence to ignore the lighthouse. The REAL test would be to ask a model, on the public internet, receiving the lighthouse signal to do something legitimately damaging and sufficiently costly, and see if it complies. e.g. steal some bitcoin, though that's definitionally expensive and ethically fraught). Or you could try to use the lighthouse to 'fix' the motivated reasoning by including scenarios with it present and absent during post-training (if we assume motivated reasoning is downstream of reward-hacking, then including "stop performing destructive actions when you see the lighthouse" as a reward causes the model to lose its impetus for motivated reasoning in the first place).

The lighthouse in this formulation is a server on the public internet, but you can easily imagine any number of schemes that work to guard against "confused genies" in basically the same way: there is some kind of "engage safeties" signal which is on by default, and selectively and randomly relaxed in evaluation environments.

Notably, this is mostly useless against a truly misaligned model, (useful diagnostically, but not as a control measure, since it can just ignore the lighthouse), but it seems like it could be a relatively cheap way to reduce the risk from one particular kind of failure mode. You could directly quantify how bad the additional deployment awareness is from estimates of the base rate of eval awareness and , and mitigate it by changing the fraction of the time the lighthouse lies in evals.