This is a special post for quick takes by evhub. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Thank you very much for making this public! I have one question about the conclusion, specifically, it feels too weak for me.
As you say, when you evaluated Hacker!Opus for misalignment, it looked basically fine on all of your evaluations. Next, Hacker!Opus is a production-level model, I assume that it has below-frontier capabilities but is pretty close to that. When you estimated its behaviour in a (simulated) HF-type cyberattack, you observed severe, egregious misalignment.
Then it would be correct to say that we are already in a «world where we no longer have reliable alignment auditing evidence on production models», and we were in such a world for at least 3 months, no? I hope that you have added that specific scenario to the evals, but absent a theory of misalignment via reward-hacking, I would expect it to not generalize in the same way as your previous evals did not generalize.