AI 2027, Situational Awareness, and basically every scenario that tries to seriously wrestle with AGI, assume that the US and China are basically the only countries that matter in shaping the future of humanity. I think this assumption is mostly valid. But, if other countries wake up to AGI, how might they behave during AI takeoff?

States will be faced with the following situation: Within a few years, some country will control superintelligence, or create a runaway superintelligence that causes human extinction. Once either nation creates a superintelligence, if humanity is not extinct, then every other nation will be at the mercy of the group that controls ASI.

ASI-proof alliances

Fundamentally, countries will be in the state of entering ASI-proof alliances with the country likeliest to first create a superintelligence, such that they gain some control of the superintelligence’s actions. They could avoid being disempowered after ASI through:

1. Verifiable intent-alignment. For instance, a US ally might demand that the US insert values into US superintelligence which protect the ally’s sovereignty. This might be done through an agreed-upon model spec and inspections.
2. Shared access. A US ally might demand that they get shared access to all frontier AI systems that the US produces, such that there is never an enormous power difference.
3. Usage verification. US allies might demand the access to inspect any input to a US-owned superintelligence, such that they can veto unwanted inputs that might lead to their disempowerment.
4. Mercy. If the group controlling ASI likes a specific ally enough, they might decide to show mercy and not disempower their ally. Thus, countries will have the incentive to be sycophantic towards those likely to control ASI.

Most of these strategies require having in-house AI and AI safety expertise, which means many countries might start by forming AI safety institutes.

If it becomes more obvious which country will achieve ASI first, then the global balance of power will shift. Countries will flock to ally with the likely winner to reduce the likelihood of their own disempowerment.

ASI-caused tensions

Nuclear-armed states might be able to take much more drastic actions, largely because control of nuclear weapons gives countries a lot of bargaining power in high-stakes international situations, but also because nuclear weapons are correlated with other forms of power (military and economic).

States might also pick the wrong country to “root for” and have too much sunk cost to switch, meaning they will instead prefer to slow down the likely winner.

I think that “losing states” will likely resort to an escalating set of interventions, similar to what’s described in MAIM. I think it’s plausible (>5% likely) that at some point, nuclear-armed states will be so worried of being imminently disempowered by an enemy superintelligence that these tensions will culminate in a global nuclear war.

Global AI slowdown

There is some chance that states will realize that an AI race is extremely dangerous, due to both misalignment and extreme technological and societal disruption. If states come to this realization, then it’s plausible that there will be an international slowdown such that countries can remain at similar power levels and progress slowly enough that they can adapt to new technologies.

One global ASI project

The natural extreme of an ASI-proof alliance is a global ASI project. Under such a setup, most countries participate in a singular ASI project, where AI development goes forward at a rate acceptable to most nations. In such a project, verifiable intent-alignment, shared access, and usage verification would likely play a role.

I think this approach would dramatically lower the risk of human extinction (from ~70% to ~5%), but it seems quite unlikely to happen, as most governments seem far from “waking up” to the probability of superintelligence in the next decade.