OpenAI's Astra release is the first one meeting their "critical" threshold on cybersecurity. Their preparedness framework says "until we have specified safeguards and security controls standards that would meet a Critical standard, halt further development." While they've spoken about various safeguards they've implemented for Astra/after the hugging face incident, there does not seem to be any (at least publicly available) standard for what sufficient safeguards/security controls would be, and while they paused some training after the Hugging Face incident they certainly did not "halt further development."

Their Aug 18 "pacing model development" post did say they "will evolve [their] preparedness framework" but there have been no updates yet.

Section 4.4 of the preparedness framework says they expect to update it before reaching critical capabilities with any model (this did not happen). It is worth noting that the preparedness framework is Not their Frontier AI framework under SB53/EU AI Act so it is not legally binding.

Looking at Astra's model card, they provide a summary of their internal Safeguards Report starting at page 102, which "informed our Safety Advisory Group's recommendation and OpenAI leadership's determination that these safeguards are sufficient for Astra's public launch." It is not clear what specific criteria were used to make the determination to release.

The main new safeguard I can find for Astra's launch is that they're extending their internal monitoring system to also cover external use. This monitoring reviews the agent's CoT alongside its actions and the conversations inputs/outputs and can automatically pause/end the conversation when it detects a potentially high-severity issue. They also discuss the misuse monitor and the classifier system they first deployed for 5.6 Sol/Terra, as well as some safety-related benchmarks. However, there's minimal explanation of why these benchmarks and safeguards are sufficient.

Combined with other issues like the reveal of a second (public) OpenAI agent message board that was not disclosed by OpenAI, and the significant drop in monitorability/gain in CoT control for Astra, this seems bad. They aren't meeting their preparedness framework, and pushing out updates to it (as I expect will happen before too long) after pushing out a model that violates it makes it seem toothless.