I disagree with this! "Catching AIs red-handed" had a lot of good ideas, and then OpenAI still didn't follow them. You say:
Did suggesting that we ought to "question the model and run experiments on it in the context where it misbehaved" inform any of OpenAI's investigations? ... I hope someone reminded the safety team to test whether newly trained models do the same thing in a similar circumstance. They might not have thought of that!
Evidently they did not. Instead, OpenAI shut down HPIM completely so that their own researchers couldn’t access it. As the METR/Redwood report said:
We did not have the ability to query HPIM (the primary model involved in this incident); OpenAI stated it was also not available to OpenAI researchers.[49] We also did not have the ability to directly access relevant data from OpenAI infrastructure, but we could request additional datasets and OpenAI shared additional datasets on several occasions.
[49] OpenAI states that: “Following the incident, we deactivated, encrypted, and restricted it from research access.”
In my opinion, the problem with “Catching AIs red-handed” was that they didn’t meme the ideas hard enough. Ideally, it should’ve been established as a standard practice for this contingency, the post was an attempt to make that happen. It wasn’t enough, but it was a good try.
But whatever went wrong with its theory of change, it wasn't that the ideas became too obvious when the relevant incident happened. Watching the whole affair has made me think those types of posts are more important, because we can't expect the right decisions to get made when X, unless the decision-makers I think "oh yeah, there was that suggestion I read that said if X happens, I should do Y. I thought X was crazy at the time, but I guess they were right about at least one thing..."
In January 2024, Ryan Greenblatt and Buck Shlegeris posted "Catching AIs red-handed." This post got on "Best of LessWrong" in 2025, and in December of last year Buck Shlegeris commented on it explaining that he was disappointed it did not lead to more follow-up research. I upvoted the post at the time of the review, and so it seems like it's representative of the analysis the community values.
As it happens, it turned out to be remarkably prescient and near-term-relevant for a theorypost: just this August 2026 we, in fact, got to see what happens when AI companies "catch their AIs red-handed". The parties who wrote the post were, additionally, some of the very people involved in investigating the Hugging Face incident.
I definitely respect these researchers, but I struggle to imagine how this analysis could have been useful. While it seems possible that publishing research like this is what put Redwood in a position to be called upon to investigate in the first place, the piece is just a long and abstract discussion about how (for example) if you catch an AI scheming, it is less likely to succeed (emphasis not mine):
And how, if you catch one of your AIs trying to escape, you should look for other AIs trying to escape:
People doing "alignment research" make a lot of these types of posts, essentially designing and juggling abstractions for alignment as a topic. I don't really understand what the theory of action for this work is supposed to be. I don't think it's harmful to do this analysis per se, and I don't think it's being done poorly or in bad faith, but there just doesn't seem to be much point. It reminds me of attending "cybersecurity" courses in university and listening to lectures about the CIA triad.
There are essentially two possible outcomes when you're speculating about "what to do" in response to future misalignment incidents:
This post, one of the lucky ones, seems to lie in the latter, fortuitous case. Did suggesting that we ought to "question the model and run experiments on it in the context where it misbehaved" inform any of OpenAI's investigations? How about "rerunning improved monitoring over past logs"? I hope someone reminded the safety team to test whether newly trained models do the same thing in a similar circumstance. They might not have thought of that!
The nearest actually useful analogue to this kind of analysis I can think of is wargames. With wargames, the reason they're useful, when they're useful, is that they inform present decisions on military policy. As an example I like, "AI 2040" is a wargame, yes, but it's a wargame for the purpose of generating a series of policy proposals, which are helpfully advertised on the website. It's hard for posts like the above to reach that bar when the content is so abstract, when it seems to be focused on advising people in the future rather than people in the present, and when (as far as I am aware) people who could be in a position to make policy decisions at the labs don't read this type of stuff anyways.