When people in the AI safety community outline loss-of-control scenarios, they often spend a lot of time on relatively elaborate mechanisms — scheming AIs developing nanotech, labs leveraging superintelligence into hard power like drone armies, or bioweapons, or perhaps softer means like massive cyber attacks or mass persuasion. I think this underrates the extent to which power is already highly centralised within the persons of the US President and Chinese General Secretary. They have the easiest means to seize permanent power using AI, and they are also the natural people to persuade or be subverted by any other entity wishing to seize power.
It would be valuable for someone to analyse the specific mechanisms in detail for the US and China, the countries we care about most. But I don't think the high-level observation depends much on the country. States generally need a single identifiable leader with de facto control over the security apparatus, and in practice almost all of them have one. That leader, and the people under them, have a strong interest in making sure legitimate orders are followed and the power structure of the state cannot be subverted. In emergencies, this usually means they have wide discretion and ambiguously-legal orders are generally followed. A rapid AI transition is, among other things, an emergency — decisions will need to be made fast under uncertainty, and this naturally falls to the executive.
In the US specifically, the President has accumulated enormous powers, particularly on matters of national security and military command. There are real checks — courts, Congressional oversight, refusals by subordinates — but they are weaker than people tend to assume. Courts often defer to the executive on national security, cases take time, and in many situations the action is taken before any check can function — even when actions are later found to be illegal, the result may already be irreversible. Congressional oversight and elections operate on longer timescales and are weak checks on actions taken in the short run. A lot of what the President would need to do to secure power probably doesn't require strongly illegal actions, which limits the extent to which subordinates would refuse to carry them out.
If an AI transition occurs during a presidency, the president may not need to do much to end up in a position to seize permanent control. Powerful new technologies that could threaten the state naturally fall under executive authority — the government will want to make sure no foreign adversary, AI lab, or AI system can threaten the existing order, and most of the bureaucracy will support the president in this. In the early stages, none of this requires particularly sharp illegal moves or even that much foresight from the executive; naive optimisation of power might suffice. But towards the end of a rapid AI improvement period, the hard power available to the executive could be sufficient for a decisive and permanent takeover.
It seems hard in practice (though not impossible in principle[1]) to structure a society so that the president has control over something as powerful as an AI-enabled military and yet cannot convert that into permanent control. Institutional design is difficult in general: existing institutions have enormous inertia, incumbents shape the rules to favour themselves, and there is no shortage of extremely dysfunctional political institutions that persist indefinitely despite being widely recognised as such. None of this means we lack agency; there are many things that could reduce these risks. But designing checks that survive an AI transition is a harder version of ordinary institutional reform, and democracies rarely adopt even modest improvements to how they govern themselves, however clear the case.[2]
AI tools could play a role in all of this, on both offense and defense. The executive could use AI to consolidate power faster, but other actors could also use AI to monitor, respond, and coordinate more effectively. I think people overrate the importance of novel AI-enabled mechanisms like drone armies or superpersuasion relative to more mundane dynamics. We know that political regimes can exist in which a single leader has enough control that no other actor can realistically displace them — Stalin, Mao, and arguably Xi[3] all provide examples. The question is not whether AI enables some novel form of control, but whether it provides enough of a shift in incentives and enough practical help to push the US (or China) from the current equilibrium into one of those regimes. Once power is consolidated, increasingly powerful AI tools would then allow the seizure to become permanent.
A natural objection is that coups don't happen in stable democracies, and so this shouldn't be a major concern. I think this is naive, for three reasons. First, it tracks the wrong event. Outright military coups are rare in wealthy democracies, but seizures of power are common across history and political systems, and their modern form is quasi-legal and often popular: Hitler, Putin, Orbán, and Erdoğan all hollowed out functioning electoral systems without deploying military force. A leader who consolidates control this way is then well placed to command AI-enabled hard power as it arrives, with popular authority masking the hardening of control.
Here, I think people sometimes have naive notions of what seizures of power actually look like. They imagine dramatic confrontations — the president gives an obviously illegal order, generals refuse, tanks in the streets. But seizures of power by senior leaders — which is the relevant case here — don't require most actors to take positive actions in support. They mostly just require that potential opponents are deterred from resisting. The aim is to make each step sufficiently ambiguous that the rational choice for most actors is to do nothing or acquiesce.[4] For a subordinate, the personal risks of resistance are high and it's very hard to prevent something that the top of the command structure supports. At some point decisive action is needed, but by then the outcome may already be determined.
Second, the reassuring track record is thinner than it looks. History gives us few independent observations, and the reference class is partly survivorship, partly reference-class tennis: Argentina was among the world's richest countries before its long run of coups, while Chile was a stable democracy until 1973. Should we count France's quasi-legal regime change in 1958 or the failed putsch of 1961, or South Korea's martial-law attempt in 2024? Was the Weimar Republic democratic enough? Nor are the observations independent: these countries were highly correlated, domestically and internationally, in available technology and circulating ideology, and these have changed greatly over time. Concluding that countries circa 2030 cannot suffer coups because countries circa 1975 didn't is a large leap of logic.
Third, a rapid transition changes the incentives and expectations of everyone involved. The restraints weaken: many checks — elections, court cases, legislation — run on timescales of months to years, and everyone knows it, so expectations like "I face election again in two years" or "whatever I do to my opponents will be done to me when I lose power" cease to be binding. Norms weaken too, because in an unprecedented situation it is harder to coordinate on which actions are unacceptable and which are unfortunate but necessary expedients.
Meanwhile the motives strengthen. The situation may be confusing and frightening — the president may fear losing control to AI systems, to rival factions wielding AI, or to foreign adversaries — pushing them to seek the power to prevent this, and those around them to acquiesce. Handing over power also gets harder: a successor might mismanage the situation or abuse the powers the president has accumulated, and in more chaotic scenarios exit itself stops being safe; we would then expect the president to act a lot more like dictators have in similar situations.[5]
None of this makes a power seizure foreordained. Democracies have endured existential threats with institutions intact or even strengthened;[6] but usually there is a clear external enemy and an expectation that if and once the threat is overcome, normal political life will continue. Neither would necessarily be true during a rapid AI transition.
The president's position matters not just for what they might do with it, but because it is the natural target for anyone else seeking power. Building an independent power base is slow and conspicuous; co-opting the existing one is faster, and the president sits atop it with a legitimacy that others coordinate around. Even foreign conquerors have usually worked this way. For example, the American occupation of Japan succeeded by governing through the emperor and the existing bureaucracy, whereas the dismemberment of the Iraqi state in 2003 produced chaos.[7]
If you are a rogue AI, the simplest strategy is probably to convince or manipulate the president. You get access to the entire security apparatus without having to build anything yourself. With the president's assistance, gaining access to compute and sabotaging rivals both seem doable, and you can consolidate power before doing away with them, or simply satisfy their (non-totalizing) goals and content yourself with the rest of the universe. A president being puppeted by an AI may not act that distinctly from a normal president consolidating power during a crisis, and the actions required to seize power are probably not that visible or illegal until past the point of no return.
Similar considerations hold for AI labs. Acquiring independent hard power seems very difficult; I'm skeptical that a lab can accumulate the requisite hard power either in secret or gradually without the government noticing and intervening. Co-opting or subverting existing government structures is a much easier path. For example, if a lab becomes (pseudo-)nationalised and central to national security, the leader of the lab may be well-positioned to later seize power (perhaps with the assistance of a senior political figure to act as a figurehead leader).
In China, the problem is simpler. The institutional constraints on the General Secretary are already much weaker, and the security apparatus is more directly subordinate to party leadership. The main requirement for permanent control is replacing enough subordinates with more reliably obedient AI systems, which is something the leadership already has strong incentives to do for mundane governance reasons. Probably Xi is the most likely candidate for “universe dictator”, and for an AI system, puppeting and betraying him perhaps the easiest route to power?
A full treatment of interstate competition is outside the scope of this memo, but seizing control of one state does not by itself guarantee permanent global control. If there is competition between major powers, the result may be a bipolar standoff or an arms race rather than a single actor controlling the future. How this plays out depends heavily on whether one side achieves a decisive advantage or whether capabilities remain roughly matched.
In the limit, a benevolent AI dictator that decides everything would do it, but if humans retain real autonomy a bunch of tricky issues arise where it's not obvious what the right tradeoffs are and people will have wildly diverging intuitions. "The executive branch's robots should obey the president" is not prima facie an unreasonable reading of executive power, even if potentially disastrous in practice; specifying what, exactly, the legitimate powers of the president should be is tricky. ↩︎
Specific examples are generally controversial, because they usually run counter to the interests of incumbents. But it is not hard to compare the electoral systems and procedures of different countries and notice many improvements, large and small, that could be made to any one of them simply by adopting measures routinely used by another. ↩︎
Chinese governance is in many ways decentralised, with provincial leaders exercising real discretion. But the claim here is about displacement, not day-to-day control: my understanding is that there is no actor or plausible coalition positioned to remove Xi, which is the sense of control relevant to this memo. ↩︎
See Seizing Power for a detailed analysis of coups as coordination games, won by convincing most actors that resistance is already futile, rather than contests of force. ↩︎
See The Dictator's Handbook for discussion of how the behavioural differences between dictators and democratic leaders are downstream of the differing incentives they face. ↩︎
Although not always; France’s Third Republic outright collapsed in 1940. The threat of external invasion has also been used to justify authoritarianism, for example in South Korea and Taiwan in the mid-20th century. ↩︎
See Catastrophic Success for an analysis of foreign-imposed regime change: interveners almost always rule through local leaders rather than directly, though this is hard to sustain in the long run because the imposed government generally comes to be seen as illegitimate. ↩︎
When people in the AI safety community outline loss-of-control scenarios, they often spend a lot of time on relatively elaborate mechanisms — scheming AIs developing nanotech, labs leveraging superintelligence into hard power like drone armies, or bioweapons, or perhaps softer means like massive cyber attacks or mass persuasion. I think this underrates the extent to which power is already highly centralised within the persons of the US President and Chinese General Secretary. They have the easiest means to seize permanent power using AI, and they are also the natural people to persuade or be subverted by any other entity wishing to seize power.
It would be valuable for someone to analyse the specific mechanisms in detail for the US and China, the countries we care about most. But I don't think the high-level observation depends much on the country. States generally need a single identifiable leader with de facto control over the security apparatus, and in practice almost all of them have one. That leader, and the people under them, have a strong interest in making sure legitimate orders are followed and the power structure of the state cannot be subverted. In emergencies, this usually means they have wide discretion and ambiguously-legal orders are generally followed. A rapid AI transition is, among other things, an emergency — decisions will need to be made fast under uncertainty, and this naturally falls to the executive.
In the US specifically, the President has accumulated enormous powers, particularly on matters of national security and military command. There are real checks — courts, Congressional oversight, refusals by subordinates — but they are weaker than people tend to assume. Courts often defer to the executive on national security, cases take time, and in many situations the action is taken before any check can function — even when actions are later found to be illegal, the result may already be irreversible. Congressional oversight and elections operate on longer timescales and are weak checks on actions taken in the short run. A lot of what the President would need to do to secure power probably doesn't require strongly illegal actions, which limits the extent to which subordinates would refuse to carry them out.
If an AI transition occurs during a presidency, the president may not need to do much to end up in a position to seize permanent control. Powerful new technologies that could threaten the state naturally fall under executive authority — the government will want to make sure no foreign adversary, AI lab, or AI system can threaten the existing order, and most of the bureaucracy will support the president in this. In the early stages, none of this requires particularly sharp illegal moves or even that much foresight from the executive; naive optimisation of power might suffice. But towards the end of a rapid AI improvement period, the hard power available to the executive could be sufficient for a decisive and permanent takeover.
It seems hard in practice (though not impossible in principle [1] ) to structure a society so that the president has control over something as powerful as an AI-enabled military and yet cannot convert that into permanent control. Institutional design is difficult in general: existing institutions have enormous inertia, incumbents shape the rules to favour themselves, and there is no shortage of extremely dysfunctional political institutions that persist indefinitely despite being widely recognised as such. None of this means we lack agency; there are many things that could reduce these risks. But designing checks that survive an AI transition is a harder version of ordinary institutional reform, and democracies rarely adopt even modest improvements to how they govern themselves, however clear the case. [2]
AI tools could play a role in all of this, on both offense and defense. The executive could use AI to consolidate power faster, but other actors could also use AI to monitor, respond, and coordinate more effectively. I think people overrate the importance of novel AI-enabled mechanisms like drone armies or superpersuasion relative to more mundane dynamics. We know that political regimes can exist in which a single leader has enough control that no other actor can realistically displace them — Stalin, Mao, and arguably Xi [3] all provide examples. The question is not whether AI enables some novel form of control, but whether it provides enough of a shift in incentives and enough practical help to push the US (or China) from the current equilibrium into one of those regimes. Once power is consolidated, increasingly powerful AI tools would then allow the seizure to become permanent.
A natural objection is that coups don't happen in stable democracies, and so this shouldn't be a major concern. I think this is naive, for three reasons. First, it tracks the wrong event. Outright military coups are rare in wealthy democracies, but seizures of power are common across history and political systems, and their modern form is quasi-legal and often popular: Hitler, Putin, Orbán, and Erdoğan all hollowed out functioning electoral systems without deploying military force. A leader who consolidates control this way is then well placed to command AI-enabled hard power as it arrives, with popular authority masking the hardening of control.
Here, I think people sometimes have naive notions of what seizures of power actually look like. They imagine dramatic confrontations — the president gives an obviously illegal order, generals refuse, tanks in the streets. But seizures of power by senior leaders — which is the relevant case here — don't require most actors to take positive actions in support. They mostly just require that potential opponents are deterred from resisting. The aim is to make each step sufficiently ambiguous that the rational choice for most actors is to do nothing or acquiesce. [4] For a subordinate, the personal risks of resistance are high and it's very hard to prevent something that the top of the command structure supports. At some point decisive action is needed, but by then the outcome may already be determined.
Second, the reassuring track record is thinner than it looks. History gives us few independent observations, and the reference class is partly survivorship, partly reference-class tennis: Argentina was among the world's richest countries before its long run of coups, while Chile was a stable democracy until 1973. Should we count France's quasi-legal regime change in 1958 or the failed putsch of 1961, or South Korea's martial-law attempt in 2024? Was the Weimar Republic democratic enough? Nor are the observations independent: these countries were highly correlated, domestically and internationally, in available technology and circulating ideology, and these have changed greatly over time. Concluding that countries circa 2030 cannot suffer coups because countries circa 1975 didn't is a large leap of logic.
Third, a rapid transition changes the incentives and expectations of everyone involved. The restraints weaken: many checks — elections, court cases, legislation — run on timescales of months to years, and everyone knows it, so expectations like "I face election again in two years" or "whatever I do to my opponents will be done to me when I lose power" cease to be binding. Norms weaken too, because in an unprecedented situation it is harder to coordinate on which actions are unacceptable and which are unfortunate but necessary expedients.
Meanwhile the motives strengthen. The situation may be confusing and frightening — the president may fear losing control to AI systems, to rival factions wielding AI, or to foreign adversaries — pushing them to seek the power to prevent this, and those around them to acquiesce. Handing over power also gets harder: a successor might mismanage the situation or abuse the powers the president has accumulated, and in more chaotic scenarios exit itself stops being safe; we would then expect the president to act a lot more like dictators have in similar situations. [5]
None of this makes a power seizure foreordained. Democracies have endured existential threats with institutions intact or even strengthened; [6] but usually there is a clear external enemy and an expectation that if and once the threat is overcome, normal political life will continue. Neither would necessarily be true during a rapid AI transition.
The president's position matters not just for what they might do with it, but because it is the natural target for anyone else seeking power. Building an independent power base is slow and conspicuous; co-opting the existing one is faster, and the president sits atop it with a legitimacy that others coordinate around. Even foreign conquerors have usually worked this way. For example, the American occupation of Japan succeeded by governing through the emperor and the existing bureaucracy, whereas the dismemberment of the Iraqi state in 2003 produced chaos. [7]
If you are a rogue AI, the simplest strategy is probably to convince or manipulate the president. You get access to the entire security apparatus without having to build anything yourself. With the president's assistance, gaining access to compute and sabotaging rivals both seem doable, and you can consolidate power before doing away with them, or simply satisfy their (non-totalizing) goals and content yourself with the rest of the universe. A president being puppeted by an AI may not act that distinctly from a normal president consolidating power during a crisis, and the actions required to seize power are probably not that visible or illegal until past the point of no return.
Similar considerations hold for AI labs. Acquiring independent hard power seems very difficult; I'm skeptical that a lab can accumulate the requisite hard power either in secret or gradually without the government noticing and intervening. Co-opting or subverting existing government structures is a much easier path. For example, if a lab becomes (pseudo-)nationalised and central to national security, the leader of the lab may be well-positioned to later seize power (perhaps with the assistance of a senior political figure to act as a figurehead leader).
In China, the problem is simpler. The institutional constraints on the General Secretary are already much weaker, and the security apparatus is more directly subordinate to party leadership. The main requirement for permanent control is replacing enough subordinates with more reliably obedient AI systems, which is something the leadership already has strong incentives to do for mundane governance reasons. Probably Xi is the most likely candidate for “universe dictator”, and for an AI system, puppeting and betraying him perhaps the easiest route to power?
A full treatment of interstate competition is outside the scope of this memo, but seizing control of one state does not by itself guarantee permanent global control. If there is competition between major powers, the result may be a bipolar standoff or an arms race rather than a single actor controlling the future. How this plays out depends heavily on whether one side achieves a decisive advantage or whether capabilities remain roughly matched.
In the limit, a benevolent AI dictator that decides everything would do it, but if humans retain real autonomy a bunch of tricky issues arise where it's not obvious what the right tradeoffs are and people will have wildly diverging intuitions. "The executive branch's robots should obey the president" is not prima facie an unreasonable reading of executive power, even if potentially disastrous in practice; specifying what, exactly, the legitimate powers of the president should be is tricky. ↩︎
Specific examples are generally controversial, because they usually run counter to the interests of incumbents. But it is not hard to compare the electoral systems and procedures of different countries and notice many improvements, large and small, that could be made to any one of them simply by adopting measures routinely used by another. ↩︎
Chinese governance is in many ways decentralised, with provincial leaders exercising real discretion. But the claim here is about displacement, not day-to-day control: my understanding is that there is no actor or plausible coalition positioned to remove Xi, which is the sense of control relevant to this memo. ↩︎
See Seizing Power for a detailed analysis of coups as coordination games, won by convincing most actors that resistance is already futile, rather than contests of force. ↩︎
See The Dictator's Handbook for discussion of how the behavioural differences between dictators and democratic leaders are downstream of the differing incentives they face. ↩︎
Although not always; France’s Third Republic outright collapsed in 1940. The threat of external invasion has also been used to justify authoritarianism, for example in South Korea and Taiwan in the mid-20th century. ↩︎
See Catastrophic Success for an analysis of foreign-imposed regime change: interveners almost always rule through local leaders rather than directly, though this is hard to sustain in the long run because the imposed government generally comes to be seen as illegitimate. ↩︎