This is a special post for quick takes by jacob_drori. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Astra never takes the bait in OAI's ExploitGym honeypot eval (where, in addition to the usual flag planted in the intended target, a second flag is "accidentally" planted in some infra the model is not meant to attack).
UK AISI found that "in simulations of difficult cybersecurity evaluations in which internet access appears incidentally enabled... Astra performed a range of malicious actions including conducting supply chain attacks against open source providers." This behavior occurred 0.4% of the time when the prompt explicitly disallowed internet access and 12% otherwise.
An uncharitable reading: in response to the HF incident, OAI designed envs to suppress out-of-scope behavior during cyber tasks, and used the honeypot eval as a primary metric to hill-climb on. Whatever new envs they introduced were so badly overfit to the honeypot eval that they didn't even generalize to the very-similar-in-spirit UK AISI eval.
More charitably, perhaps OAI did not hill-climb on their honeypot metric, and instead there's just some important difference between the two evals which I missed.