This is a special post for quick takes by TsviBT. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Probably a silly idea, but: I wonder if you could get a bunch of the top buyers (top 10? top 50?) of AI to sign a thing that's like:
We won't buy AI from companies doing frontier AGI R&D / RSI because it's too dangerous.
They'd probably never go for it, but in principle, it might not hurt them too much. In particular, if one leading AI lab decided to accept the deal, they could get a bunch of business without doing frontier AGI R&D; at least at first, those businesses would be getting a product that's roughly as good as the other products. IIUC enterprise is the majority of this revenue [citation needed], though IDK how concentrated it is.