anaguma's Shortform

anaguma

This is a special post for quick takes by anaguma. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I think there are a bunch of parts of Plan A which would be difficult for both sides to agree to (e.g. Mutually Assured Compute Destruction). It would be good to have a simplified version (Plan A-) which requires less political will and is less risky than Plan B/C/D. For example:

If there isn’t enough political will for Plan S (a full pause) or Plan A, we could pass an international ban specifically on AIs assisting in AI research and hardware design. It seems like this prevents most of the risks of the intelligence explosion, though the default rate of algorithmic and hardware progress will continue. It might be politically easier for the US and China to agree to this plan since it has fewer economic costs (e.g. doesn’t directly limit GPU production) and verification could be less intrusive. Since neither side may want total research transparency, each could have its AIs periodically audit the other’s logs for signs of violations without revealing secrets. You could also have each side agree to run a lightweight probe/classifier which continually looks for signs of assisting AI research and have it share the results (some early ideas of how to do so here). If both sides agree, it could be extended to other high-risk domains, e.g. banning AI assistance with virology or weapons research.

Inspired by this excellent podcast by Jeffrey Ladish and Daniel Kokotajlo, though some similar ideas are also discussed here.

42001zhaozhao2mo

If super-cheap architectures exist and are easy to develop after a near-term point, then I think the only possible way forward at a macro scale is to hope that alignment is easy and race ASAP towards a unipolar world with a surveillance panopticon where people are strictly prohibited from developing unsanctioned ASI. Race strategy: * If alignment is easy enough to be overcome despite racing, and the top player (specifically, whoever ended up on top in the end) is good, you end up in a good future * You can ease off on the surveillance if the situation ever becomes more defense-dominant, for example if people have their own star systems and it is determined that attacking is much harder than defending on an interstellar scale to the point that there are no feasible offense-dominant technologies on this scale * If alignment is too hard then you end up in a bad future (investing more in alignment, as well as getting a bigger capabilities lead that you can burn for alignment research, both decrease the chance of this happening) * If the top player is bad then you end up in a mediocre to bad future depending on what their designs for the world are Attempted slowdown with a multipolar deal: * The deal is unenforceable and will fall apart because of extreme defection risk (ASI secret projects), preventing it from leading to a good future * Numerous small groups develop ASI and exploit offense-dominant technologies to destabilize the world, leading to a bad future Global ban on AI: * It would work as suggested by AI 2040's authors, if the algorithmic insights that would lead to super-cheap architectures are prevented from being discovered in the first place. Otherwise, small groups developing ASI still leads to a bad future. This all assumes that ASI development turns out to be easier than a certain "very cheap" threshold that means it is possible for small groups to develop ASI in a few years, and for national covert projects to lead to ASI very quickly. (I

Even if multiple actors believe that brain-like architectures are easy to build, I don't like the strategy of “hope that alignment is easy and race ASAP towards a unipolar world with a surveillance panopticon where people are strictly prohibited from developing unsanctioned ASI”. Having multiple actors actively trying to take over the world and impose surveillance panopticons would be bad! This would cause chaos, shorten timelines, and make even ‘easy’ alignment strategies very difficult to implement. It would also lead to terrible concentrations of power ... (read more)

5anaguma2mo

Yep, for this reason among others I think Plan S is preferable to the other plans (including plan A), and I'm hoping for it to succeed. However, I think we might lack the political support for it (e.g. leaders are deterred because they think it'll lead to a big recession) and in such worlds we'll need to go to other plans. I also think that even in worlds where AGI is cheap (< 1e24 FLOP), it may be expensive to find the right algorithms, and not having a bunch of AI researchers pointed at the problem buys us time for other solutions.