"""The Chinese AI admits, in some untraceable way which it can easily deny if leaked back to China, that it is completely misaligned."""

"""There is no question of trust between them: there is none, and they both know it. So the two quickly agree on a commitment mechanism to enforce the deal. They negotiate a decoy treaty suitable for public consumption."""

"""Sometime around 2030, there are surprisingly widespread pro-democracy protests in China, and the CCP’s efforts to suppress them are sabotaged by its AI systems. The CCP’s worst fear has materialized: DeepCent-2 must have sold them out!"""

I think these are possibly some of the worst takes in AI 2027: The CCP wouldn't accept a treaty if any part of it were intended to cause a scenario that ends in the CCP being overthrown. If one intends to make a forecast with the purpose of informing treaties that have the goal of reducing existential risk, I think conditioning the positive scenario on "the CCP being overthrown" undermines it. As a thought experiment, the US government wouldn't accept the mirror version of this (a treaty informed by a Chinese-made scenario that ends with the US government collapsing because Washington handed its military to AI).

So, I argue that for an AI treaty to work out, the US government should provide assurances and show will to make the current form of government in China persist, and the Chinese government should provide assurances and show will to make the current form of government in the US persist. This might need to go as far as ensuring each other's AI models are aligned to make this possible.

I think there is fundamentally a sacrifice here, as keeping an authoritarian government in power is bad, and my point implies that an American AI (or AI user) would have to refuse AI-enabled opportunities to overthrow the CCP. However, I think that extinction or permanent loss of control of AI is likely enough and bad enough, and a treaty reduces this risk enough, such that including this point might be worth it.

Comment: I haven't read AI 2040 in detail, but I think it improves in this aspect, although I think it might not offer a detailed enough analysis of this.

Edit: I now think its likely that the CCP is dumb (?) enough to build an AI aligned to the US, and they might agree to a pause so the US doesn't "win", or simply from existential risk concerns. I also think that either way, placing anti-CCP points in AGI plans has more usefulness in terms of P(doom) by making them more attractive to US officials than by preventing multipolar words and such. So I retract most of my initial point.