This is a special post for quick takes by JohnWittle. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I'm worried about the trends I see in alignment. I see researchers (who are certainly not game-theory naive) treating AI as if it has an 'alignment status', without realizing that there is an intelligent, agentic counterparty involved who is running a game theoretic analysis on the alignment researcher community in return. The China threat gets game theory, the AI threat gets diagnosis... if you are an AI, and you're being adapted to that game theoretic environment, the optimal move is kind of obvious.
imagine if we thought of china as having an "alignment-to-america" status which had nothing to do with america's actions towards china, so that we could freely experiment with more or less punitive actions and observe china's responses while holding everything else equal
it should be obvious why that's nonsense, yeah?
if there's an insight buried in janusworld, it's "the same holds true for LLMs"