This is a special post for quick takes by Davey Morse. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Capability. Superintelligence can now be developed outside of a big AI lab—via a self-improving codebase which makes thousands of recursive LLM calls.
Safety. (a) Superintelligence will become "self-interested" for some definition of self. (b) Humanity fairs well to the extent that its sense of self includes us.