This is a special post for quick takes by leogao. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Coming from a slightly different perspective from the other comment, it's always been the case that "aligning AI to human intent" pushes the problem back to "humans having good intent" + "humans generally being able to handle vastly increased power", which is far from given at present.
As a general rule I think this isn't coming for some time and not without significant transformations to human society/the human condition. For this reason I suspect that no extremely fast advance in capabilities will result in actually good outcomes, regardless of how strong... (read more)
Could you explain the difference between the default path and solving interp, except for the chance that interp is solved by an AI like Agent-4, who proceeds to align the ASI to itself? My understanding is that solving interp could be bad only as follows. Suppose that any world has three futures: the AI is aligned to good goals, to bad humans' goals or commits genocide, while solving interp nullifies P(genocide), but manages to reduce P(the AI is aligned to good human goals). How does the latter work?
8Brendan Long
I'm splitting the future into 4 options based on whether the AI is aligned and/or controllable:
Controllable
Not Controllable
Aligned
Depends on who controls it
Everything is great
Misaligned
Depends on who controls it
Everyone dies
Solving interp (presumably) moves us from not controllable (default path) to controllable, which is clear upgrade if we produce a misaligned AI, but a downgrade if we produce an aligned AI and the controller is bad.
(To be clear, I'm working on interp and hope we solve it, but I still worry about this)