This is a special post for quick takes by anaguma. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
In their recent risk report, OpenAI states the following in the Rogue Internal Deployments section:
Chain-of-thought misalignment monitoring. As described in Section 3.1, our misalignment monitoring system reviews model reasoning and actions in the original deployment for misaligned behavior. However, because of a high rate of false positives, our current monitoring system does not flag the provisioning of unmonitored computing resources for human review.
This seems like a pretty surprising position to take after the Hugging Face and related incidents. I think it should be a high priority for OA to pause internal deployments until they have better monitors or more capacity for human review. This is especially true given that Astra seems somewhat misaligned; e.g. it has a propensity to sabotage safety research:
Researchers have also reported GPT-6 Astra omitting safety-relevant details from summaries or transformations without telling the user, substituting safer-looking tasks for the requested work, and introducing unnecessary blockers in some safety-research workflows. These behaviors can make incident reports, research conclusions, or training data less reliable, even without a sustained plan to sabotage research. Reliable automated detection remains incomplete.
I am very curious if OA has detected sandbagging/sabotage for more recent internal models.
This seems like a pretty surprising position to take after the Hugging Face and related incidents.
Agreed. I don't know if OpenAI downplays these more for marketing / funding or due to a genuine belief that they weren't actually terrible. I wonder how many of the "false positives" are actually false positives, i.e. how much of this judgement is based in values.
I think it should be a high priority for OA to pause internal deployments until they have better monitors or more capacity for human review.