This is a special post for quick takes by Richard_Ngo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Am I correct that the implied implication here is that assurances from a non-rationalist are essentially worthless?
I think it is also wrong to imply that Anthropic have violated their commitment simply because they didn't rationally think through the implications of their commitment when they made it.
I think you can understand Anthropic's actions as purely rational, just not very ethical.
They made an unenforceable commitment to not push capabilities when it directly benefited them. Now that it is more beneficial to drop the facade, they are doing so.
I think "don't trust assurances from non-rationalists" is not a good takeaway. Rather it should be "don't trust unenforceable assurances from people who will stand to greatly benefit from violating your trust at a later date".