This is a special post for quick takes by Morphism. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
RSI should be at least as hard as alignment, since in order to recursively self-improve, an AI must itself be able to solve the alignment problem wrt its own values. Thus, "alignment is hard" and "takeoff is fast" are anti-correlated.
What, if anything, is wrong with this line of reasoning?
By the time you have AIs capable of doing substantial work on AI r&d, they will also be able to contribute effectively to alignment research (including, presumably, secret self-alignment).
Humans do substantial work on AI r&d, but we haven't been very effective at alignment research. (At least, according to the view that says alignment is very hard, which typically also says that basically all of our current "alignment" techniques will not scale at all.)
Even if takeoff is harder than alignment, that problem becomes apparent at the point where t... (read more)