Summary: Right now, we have no idea how practical it is for countries to sabotage each others’ AI projects. This makes it hard to forecast what countries are going to do once the reality of superintelligence sets in: among other things, it’s unclear what military options middle powers have to slow the U.S. and China down, whether blockading Taiwan is a death blow to U.S. AI or a setback, and whether it’s realistic to expect AI deterrence to last for decades like MAD.
To help get better intuitions about this, I ran a quantitative model of how much destroying AI compute would set back the U.S. and China’s progress towards superintelligence. The long and short of it is that even extensive attacks (on compute alone) will only delay AI development by a handful of years, and can be significantly offset by nationalizing surviving compute. Rather than think of kinetic sabotage as a way to permanently stop a state from acquiring advanced AI, you should think of it as a ballpark 1-5 year delay. If this is realistic, then relying on kinetic attacks to maintain an indefinite stalemate on AI development would be incredibly risky and counterproductive. Instead, we should a) do more research into how deterrence at lower levels of escalation would work, and b) implement the ones that don’t trade off too much on our ability to later negotiate over AI development.
Views preceed and do not represent those of CAIS. Special thanks to Oscar Delaney for mentorship and feedback throughout this project.
If you blow something up, it stops working. As far as heuristics go, this is a pretty good one. It works for dams, bridges, Russian tanks, other missiles, and dictators fortunate enough to have to ruled Iran for 30 years yet unfortunate enough not to be named Fidel Castro.
In Superintelligence Strategy, Hendrycks and co apply this same heuristic to AI development. The descriptive argument is straightforward:
Superintelligence is an existential national security threat. A country that gets a large enough lead in AI will get a corresponding lead in military technology, dominating their rivals the same way the Spaniards dominated the Aztecs.[1]
AI development is harder to defend than it is to attack, since its infrastructure is expensive, exposed, and takes a long time to replace.
Therefore, states will sabotage rival AI development.
The normative argument is that we should encourage this dynamic, because it enforces a strategic equilibrium. If it’s impossible for any country to unilaterally develop superintelligence without getting their wings clipped, then AI development would need to proceed under (at least tacitly) mutually agreed terms.[2] Compared to the alternative of a no-holds-barred race, development would be both forcibly slowed down, leaving more breathing room to work on technical alignment, and more pluralistic, preventing a single country or company from dominating their competitors. Ergo, since sabotage is both inevitable and stabilizing, we should push the government to do it well: building more institutional capacity for sabotage, communicating a clear escalation ladder, and drafting up contingency plans for an attack.
Mutually Assured Destruction (MAD) vs. Mutually assured AI malfunction (MAIM).
Whether this is a good plan (or even a viable one) depends on how credible a deterrent sabotage is. After all, if the countries building superintelligence don’t expect their AI projects to get sabotaged, then there’s not much reason to slow down development. Most criticism of MAIM focuses on this problem from the perspective of political credibility, arguing that MAIMing attacks risk massive retaliation, lack of catalysts for action, and induce high risk-tolerance, making them poor deterrents.[3] Given these problems, the thinking goes, states can’t credibly threaten to damage each other’s AI infrastructure severely enough to actually shut AI development (and therefore deter it).
The problem with this debate is that we have no idea how hard it is to sabotage AI development. Sure, observability and communication are unavoidable problems. But they’d be simple to manage if shutting down AI development was as easy as drone striking a datacenter like you were hitting the Death Star’s exhaust port. Likewise, if the only way to destroy a datacenter was a direct land invasion, Xi could mail Congress his personal plans for world domination before they’d vote to march troops into Shanghai. Will to strike is downstream of capability to strike.
To get better intuitions about the viability of AI deterrence then, we need a model of its operational requirements: something that tells us how much AI infrastructure would need to be destroyed, how much destroying it would slow down development, and which offensive and defensive targeting strategies states could use. To that end, this post discusses the results of my model of kinetic strikes, as well as its main implications for AI deterrence. For the full model methodology and limitations, you can read the appendix of the original paper—otherwise, here’s a brief overview:
First, we set an effective FLOP target for the model we want to deter (e.g. corresponding to the AIFP’s SAR and ASI benchmarks).[4] The goal is to prevent the defender from successfully training a model this large for as long as possible.
Second, we plot how much compute the U.S. and China will have access to. As a baseline, we use Romeo Dean’s compute forecast to set global compute growth, and then use the combined Epoch Cluster & Frontier datasets to estimate each country’s relative share and internal distribution.[5]
Third, we use the AIFP’s model of software progress to measure how compute growth drives algorithmic improvements.[6]
Finally, we model kinetic strikes by using them to generate an alternate compute timeline, which we compare against the model’s baseline predictions to measure delay to different effective compute milestones. We vary the percentage of national compute destroyed, whether the AI supply chain is simultaneously attacked, whether strikes are persistent or one-time, and whether compute is centralized into the leading project after the strikes.
Motivations for Sabotage
Attacks on AI infrastructure are rightly seen as an extreme escalation over AI development, which can make them feel like unrealistic threats. To that end, it helps think about the mindset a state would have to have before it would consider kinetic strikes. I use the example of China below for concreteness, but the same dynamics apply to any state that isn’t the leader in AI development. For those only interested in the results of the model, feel free to skip to the following section.
By most measures, China is set to lose the AI race. Because the U.S. has access to so much more compute than its rival, and because the U.S. can so cheaply deny its rival’s growth by squeezing chip exports, the U.S. is much better positioned to both train powerful models earlier and to deploy them more extensively in automated AI R&D, and thus to acquire the military capabilities of superintelligence first. At the moment, this fact has mostly escaped the attention of either government. Insofar as there is a race to AI, it’s a race to integrate it into the economy, not to total military domination.
Still, the idea of superintelligence is no doubt stumbling through the bureaucracy on both sides of the Pacific (particularly in the U.S.), signal boosted by a handful of technocratic advisors and the outreach efforts of the AI companies. Eventually, in the course of pursuing general intelligence, those same companies are going to stumble onto AIs with extremely strategically relevant capabilities: superhuman hacking skills, amateur bioweapons uplift, autonomous loitering munitions, or any other powerful dual-use skills. By achieving these capabilities, the labs will fall firmly under the gaze of the state: not just as a woke thorn in their side, but as threats to its own monopoly on violence and geopolitical competitiveness.
Having drawn the eye of Sauron, the AI companies will have to bend the knee to Mordor, working with the government to both secure their models and develop new military applications of their technology. The securitization and success of these new American military projects will make China’s current position—that embodied AI and diffusion into the industrial base are the true path to competitiveness—harder to maintain. To stay competitive, its policy will need to shift towards facilitating the same software-based intelligence explosion the American frontier companies have been aiming at. But even with mass centralization of its own compute and outpouring of investment into domestic chip production, China can’t hope to catch up physically with the U.S.’s output, least of all when the U.S. has been investing so aggressively in its own compute buildout.
If China cannot outrun the U.S., then it will need to slow the U.S. down. The first and cheapest tools are the ones China already uses: espionage and cyber operations. Chinese intelligence services are no strangers to attacking U.S. tech and defense companies, and are (allegedly) already using AI tools in order to do so. It could attempt cyber operations directly against American datacenters, aiming to corrupt training runs, exfiltrate checkpoints, or disable cooling and power systems in Stuxnet-esque attacks. Some of these would succeed, at least initially. But cyberattacks on critical infrastructure tend to be one-shot weapons: each zero-day you expose through your attacks is one you burn to a future patch. After a few high-profile incidents, the major hyperscalers might (with government assistance) air-gap their most sensitive training clusters, silo their AI developers, and start properly screening employees for human intelligence.
At this point, China's options for closing the gap nonviolently start to thin out. It can continue to strengthen its own industrial policy: centralizing compute under state control, pouring money into SMIC and domestic HBM production, and trying to squeeze more algorithmic progress out of the compute it has. But these are slow-burn strategies that take years to pay off, and China might not have years. It could try to leverage its dominance in rare earth minerals and critical material processing to disrupt American supply chains, but these dependencies are less acute and more substitutable than the semiconductor chokepoint the U.S. holds over China.[7]
Under this kind of pressure, the Chinese government could crack in any number of unpredictable ways. The government might reach out and try to make a deal with the U.S. to slow down capabilities growth, or at least get some mutual verification of what capabilities they really have. They might regress into denial, defaulting to advisors who predict that AI will be important but not existentially dangerous: closer to hypersonic missiles than to nuclear weapons. Or maybe a spy reports back that the U.S. has a secret DARPA project aiming to use ASI for superweapons R&D and all hell breaks loose. Whichever path they take depends on how seriously they take the possibility of superintelligence and how much restraint the American government has shown in deploying its technological edge internationally.
In a scenario in which kinetic strikes are politically viable, you’d expect the following. First, it would have to be obvious to the Chinese government (such as through espionage) that the U.S. is actually on the cusp of superintelligence. Second, it needs to believe that superintelligence would allow the U.S. to disempower it (such as by overcoming nuclear deterrence or enabling a decapitation strike), and that the U.S. would aggressively use these powers to do so. Finally, it needs to assess that the costs of U.S. retaliation are worth accepting (i.e. that the chance of WW3 or outright nuclear war are sufficiently low). Such a strike would therefore be much more plausible if it were small in scope. If China could derail progress towards superhuman AI for several years by taking out just a handful of datacenters, it would be both able to accomplish this attack using less provocative means (ex: human insiders) and at a less provocative scale. A small, decisive strike could be framed as a limited action, paired with negotiations, making it harder for the U.S. to justify full-scale retaliation.
In practice, however, China (and the U.S.) would need to hit dozens upon dozens of targets in order to inflict a multi-year slowdown. In the model, we can see the practical effort required for these strikes directly.
Strike Scenarios
Using this model, we can quantitatively measure both the value of destroying rival compute and the number of clusters that would need to be targeted for a given level of delay. In the paper proper, we divide these into three strike scenarios, in ascending order of scope and aggression: minimal strikes, which target the supply chain or largest datacenters alone, medium strikes, which target them together, and maximal strikes, which involve persistent destruction of compute. If you want to check through the model yourself or run some tests, feel free to visit the repo here and play with the interactive model to test different scenarios.
Minimal Strikes
For the past few years, the datacenter buildout has been dominated by a hyperscale strategy: putting the majority of each country’s chips into increasingly massive clusters. Assuming that the communication, power, and regulatory incentives that are causing this concentration don’t change, most future compute is going to stay conveniently grouped into a handful of geographic targets.[8]
Extrapolated compute distributions, based on the Epoch cluster and frontier datasets. The bottom 1% of compute added each year is hidden for visual clarity.
Although attacks on AI infrastructure are going to be extremely escalatory no matter what, this level of consolidation could still encourage states to hold back a bit. If the majority of a rival state’s compute can be taken out by destroying (roughly) a few dozen sites, there might not be a need to risk a disproportionate counter-attack by targeting anything aside from the largest datacenters. To measure the effectiveness of this strategy, we can plot the delay from targeting 50-90% of present-day compute.
Despite the ostensible scale of the attacks, this has very little impact on on the overall development timeline. Fundamentally, the problem is that too much new compute is being brought online. On current trends, each year is going to keep introducing more compute than the last, both from the raw physical scaling of the chip supply chain and the constant improvement in the per-unit performance of those chips. Even if an attacker destroys upwards of 90% of the defender’s compute, most of that damage will be quickly washed out—assuming that each country decides to race through the sabotage rather than slow down their planned compute buildout.[9]
In order to avoid this washout effect and shrink the target count, states could instead opt for the alternative strategy of just directly attacking the AI supply chain. You can count the number of frontier semiconductor and CoWoS fabs on two hands, and destroying them would probably knock out more than 90% of future compute for years, even assuming ASML, the HBM manufacturers, and the many other suppliers escape unscathed. Even if we just limit the destruction to TSMC and SMIC to constrain the geopolitical scope to the U.S., Taiwan, and China, semiconductors and packaging would bottleneck the entire supply.
Although TSMC primarily supplies the U.S., it would still have a significant impact on Chinese AI development by reducing the availability of chips to smuggle.
Indeed, the peak delay of this strategy is nearly an order of magnitude higher. If most compute is going to exist in the future, then you can effectively “destroy” much more of it by shrinking production. The obvious problem with this approach is that it’s most useful early in AI development while development is immature, and then starts flatlining around the same time you get empirical evidence of powerful models. And at that point, most of the compute the defender needs will already be installed—forcing the attacker back to direct strikes if they need an emergency delay.
Medium Strikes
Since these two targeting strategies have complementary strengths and weaknesses, the next step up the kinetic escalation ladder is to combine them. After all, if you’ve already committed to mainland strikes on AI compute, the marginal risk of escalation you take on from increasing the scope of attacks to the supply chain is going to be pretty low. So assuming that states commit to a giant preventative strike on enemy compute and its suppliers, what happens?
Here, the main benefit is that the value of strikes don’t fall off as quickly—rather than getting much more peak delay, your strikes just stay effective for longer. Normally, supply chain attacks peter out in effectiveness because the amount of compute they prevent from coming online becomes irrelevantly small by the strike date. But by destroying so much compute upfront, the attacker increases dependence on future compute to offset the strikes, making the marginal reduction in future compute from supply chain strikes more meaningful.
Maximal Strikes
The last option states have for increasing the impact of their attacks (at least while still only targeting compute) would be to keep destroying new datacenters as they come online. Although this would be even more operationally demanding and make it much harder to later sue for peace, it should throttle the influx of new compute enough to meaningfully slow down AI development. Below, we plot out the effects of sustaining this kind of compute denial indefinitely.[10]
Unsurprisingly, this proves much more effective. Adding compute has diminishing returns to research progress; conversely, removing compute becomes more impactful the closer you get to destroying 100% of it. In this scenario, most chips that would be produced under the status quo are not produced due to destroyed fabs. Of the few chips that are produced, most are destroyed when they are placed in data centers, creating a double filter.
Defensive Offsets
Still, AI sabotage isn’t a one-turn game. If states are having their leading projects sabotaged, especially to the point it threatens the global economy or runs the risk of all out war, they’re going to adapt.
One of the simplest adaptations would be to nationalize compute and give it to the leading lab. After all, if you’ve just had your datacenters shut down through strikes on the homeland, commandeering your country’s militarily-irrelevant compute isn’t exactly beyond the pale. In the model, we treat centralization as a 90% sweep: nine-tenths of the country’s surviving compute (and of every buildout afterwards) is handed to the leading company, which then reallocates it entirely to AI R&D.[11]
Delay offset from centralizing compute the same month as a medium strike attack.
Despite the scale of the original medium strike scenario, for example, compute centralization nearly neutralizes the original strike! And in the case of the lighter minimum strike scenarios, the results can be even worse for the attacker:
Delay offsets from minimal attacks on the AI supply chain alone.
In these cases, the strikes are weak enough that they become involuntary industrial policy. It’s entirely plausible that China could try to slow down U.S. AI development by blockading Taiwan, for example, only for them to miscalculate and spook the USG into leaving the leading lab with more compute than they would have had otherwise.
Implications for AI Deterrence
So what do these results mean for MAIM, or at least this particular rung of the MAIM escalation ladder? From our modeling, destroying compute alone isn’t enough to force a stalemate over AI development. If a state is motivated enough to continue racing towards superintelligence despite the sabotage, they will eventually succeed.
The main problem is that it’s too easy for the defender to replace lost compute. However massive the initial strike, it can be partially compensated for by a) direct replacement with new clusters, b) internal concentration of surviving compute, and c) algorithmic efficiency gains reducing the need for compute in the first place. The key constraint on strike effectiveness is less whether the attacker can destroy enough compute, and more whether they can keep it from being replaced—and on that dimension, kinetic strikes are only partially effective. They can’t halt new datacenter construction without persistent attacks, can’t stop experiments on surviving hardware, and can’t undo algorithmic progress that has already happened.
This problem has difficult implications for kinetic AI deterrence. Namely:
Delay, not denial. Across all of our strike scenarios, we could not achieve a delay of more than three years for an attack in the last year of ASI development. So long as an intelligence explosion dynamic holds, algorithmic efficiency will grow too fast for states to deny superintelligence indefinitely. If states try to maintain the MAIM equilibrium in spite of this fact, they will be forced to either up the ante to non-compute targets or invest in the robustness of a lower rung of the escalation ladder.
Strategic constraints. Still, some might reason that threatening kinetic strikes to secure a few years of delay is worth it, given the even greater risks of catastrophic loss of control or strategic irrelevance.[12] Unfortunately, even this limited delay would be subject to inconvenient constraints, including:
Striking early is optimal but inconvenient: Early strikes minimize target count and maximize damage to the AI supply chain. The earlier strikes are conducted, however, the less empirical evidence there will be to actually motivate something as severe as a kinetic attack on AI infrastructure. Given the already huge uncertainty about the timelines of AI development, it’s difficult to imagine an intervention as severe as kinetic strikes being employed while they are still maximally useful.
The target count is huge. Conservatively, we expect strikes on AI compute to involve several dozen targets across mainland clusters and the AI supply chain.[13] Succeeding at destroying these targets against a peer nuclear power (especially while containing escalation risk) would be enormously difficult.
Strikes encourage nationalization. Since the motivating factors for nationalization are the same as those for strikes (situational awareness, fears of competitiveness), and since strikes themselves could act as a trigger for nationalization by creating demand for compute, it seems extremely likely that countries would centralize their surviving capacity in response to a major attack.
U.S.-China Asymmetry. As difficult as it would be for the U.S. to launch a MAIMing strike, a preemptive Chinese strike would be almost infeasible. It would need to destroy significantly more targets, using a military far less able to project power overseas. And even if it succeeded, it would not escape unscathed. China depends so extensively on (il)legal imports of Nvidia GPUs that it would still severely damage its own chip buildout in the process of targeting the American supply chain. And in the likely event that its attack provokes a proportional counterstrike from the U.S., China will only be left further behind than it began.
Strike delays are low quality. Ultimately, the point of attacking a rival AI project is to buy time—time to avoid strategic disempowerment, or time to work on technical alignment. Securing this time with kinetic strikes, however, would make it fundamentally less useful for these purposes. If you care about national security, your strikes will trade away whatever chance you had that the winner would handle ASI development magnanimously by demonstrating you are an imminent security threat. The next few years would either be spent trying to win a race you will likely lose regardless, or to establish a monitoring regime at gunpoint, creating intense pressure to subvert it. Likewise, an attack in the name of buying time to work on misalignment would only encourage the defending state to disperse their frontier AI projects into hardened facilities, placing them under direct military or intelligence oversight, and start prioritizing the development of capabilities powerful enough to defeat this new security risk. Kinetic delays can only be so effective when they validate the national security paranoia that caused the racing to begin with.
Closing thoughts
Overall, it seems like states would have a difficult time denying rival AI projects by targeting compute alone. As long as the defending states is willing to race through the sabotage and pour their resources into reconstituting the leading project, the attacker probably can’t enforce a permanent state of deterrence by denial. And since the potential gains from dominating in AI are so massive, a situationally aware state would probably be willing to absorb massive amounts of damage (economic or otherwise) in order to be able to secure victory---all the moreso if escalation over AI has made them paranoid and risk tolerant.
Still, I don’t think these results should be read as a condemnation of AI deterrence in general. Even if direct strikes on AI compute are unacceptably inefficient and hostile, deterrence could still work if the lower rungs of the escalation ladder were threatening enough. It could be the case that cyberattacks on AI projects alone, for example, are sufficient to prevent states from deploying their AIs for automated R&D, or that economic sanctions and the threat of economic collapse are threatening enough on their own to dissuade national leaders. As with kinetic strikes at the start of this piece, however, we simply have no idea how efficiently these escalations would deter rival AI development—and until then, governments can have no faith that they will suffice as threats.
Acknowledgements
We would like to thank James Nicholas-Bryant, Matthew Gentzel, David Abecassis, Liam Patel, Adam Khoja, Bill Anderson-Samways, Will Anderson, Rhea Kanuparthi, Jason Hausenloy, and Sophie Kim for their feedback and support.
Concretely, we’re talking about the use of superintelligence to subvert MAD. This could happen through the design of exotic new technologies (superweapons that give a massive first strike advantage, or super-defenses that prevent retaliation), or through brute force industrial expansion (e.g. churning out a thousand interceptors for every ICBM).
Depending on the speed of takeoff and the ceiling of technology, the gap might be so large as to provide an “Overwhelming Strategic Advantage”, where victory is not only assured but basically costless for the attacker, the same way that the North Sentinelese would have no chance of slowing down a submarine. If superintelligent AIs are especially persuasive, for example, it could be possible to simply convince other governments to avoid retaliating in the first place.
For example, China might use its kinetic leverage over the U.S. in order to force concessions on mutual verification, making it harder for the U.S. to undercut MAIM in the future by building secret blacksites or taking advantage of secret algorithmic efficiency improvements. This would naturally subject China to demands for the same adversarial scrutiny from the U.S., reinforcing the MAIM equilibrium.
They might argue, for example, that similar incentives for an international agreement to prevent the development of ICBM defenses exist—and that the U.S. repeatedly violated those same agreements without any real fear of repercussion.
In this article, for example, I describe how scaling the equivalent of GPT-4 by ten orders of magnitude would allow for an AI that could trivially match the performance of an expert human virologist. Of course, we don’t have another 10 OOMs of GPUs lying around. In fact, we probably won’t get more than another 3 OOMs from hardware scaleups by 2030, even accounting for hardware efficiency improvements.
But while power constraints and chip production will eventually throttle the hardware buildout, physical compute is only half the story. The rest of the scaleup will come from algorithmic efficiency improvements: algorithmic insights make it computationally cheaper to achieve the same level of capability. If half as many FLOPs are needed to match the performance of say, GPT-4, then the model could be trained with just half the GPUs---conversely, your GPUs would be capable of yielding twice the “effective” FLOPs. In other words, the amount of compute that physically exists is distinct from how computationally useful that compute is, which is what we want to use the notion of effective compute to capture.
But while power constraints and chip production will eventually throttle the hardware buildout, physical compute is only half the story. The rest of the scaleup will come from algorithmic efficiency improvements: algorithmic insights make it computationally cheaper to achieve the same level of capability. If half as many FLOPs are needed to match the performance of say, GPT-4, then the model could be trained with just half the GPUs—conversely, your GPUs would be capable of yielding twice the “effective” FLOPs. In other words, the amount of compute that physically exists is distinct from how computationally useful that compute is, which is what we want to use the notion of effective compute to capture.
Since information on China’s compute shares is relatively sparse in the Epoch dataset, we instead based its share on the supply and demand side estimates from Zakaria and Corvino, which give China a ~14% share of global compute when compared to Epoch’s Chip Sales data. In our case, we also erode this relative share to 9% by the strike date, on the basis that the legal import, smuggling, and remote access pathways Zakaria pointed out will have been partially suppressed by then.
An algorithm that achieved the same performance on some benchmark with half the required training compared to the previous SOTA, for example, has “effectively” doubled the power of your compute (for the purposes of training to that benchmark). If we assume that these improvements are scale-independent (i.e., that they generalize to all scales of computation), they should similarly reduce the computational requirements of future AI models as well.
The actual source of these improvements varies considerably. Sometimes the improvements come from using an entirely new architecture, like the replacement of LSTMs with transformers (the scaling efficiency of which finally made LLMs viable to train). Other times, the improvements are as small as training the models on higher quality data. Once you zoom out though, most of these improvements are basically correlated with compute, which lets researchers run experiments and, increasingly, run AI labor to design and test experiments directly.
As a sidenote, every delay chart shown is plotted in terms of percentage of national compute destroyed. In other words, they show what would happen if you managed to destroy a set amount of each country’s compute, regardless of how many targets it took to reach that point. The compute buildout graph below is essentially just an illustrative example of what targets each country will have at different points in time.
In practice, we would expect any kind of kinetic attack to have a chilling effect on the funding for private datacenter construction. Most investors—and the companies insuring them—are pretty averse to building billion dollar test targets. That leaves the question of whether the government could or would pick up the financial slack. If AI infrastructure is seen as critical enough to be worth striking, the government might well reason that it’s worth funding.
With that caveat in mind, you can consider these results in the context of the defending government intentionally racing through the kinetic sabotage, committing as many financial resources as they need to stick to the original development schedule.
Specifically, assuming that states continually destroy 90% of the new compute being brought online each month, after destroying 90% of the defender’s compute in the initial strike.
Specifically, into running AI research assistants internally, training models, and running experiments. Most importantly, it excludes customer inference, which was previously assumed to take up 37% of the leading lab’s frontier compute (shrinking to a ~9% relative share), since it doesn’t contribute to AI development directly.
The leader of a nuclear third power without any of its own AI infrastructure, for example, might reason that it has no choice but to threaten to strike the U.S. and China’s AI projects, expanding the window where it can negotiate from a position of strength.
And that’s just the target count: you still have to actually take out the site itself, which might require independently targeting and destroying multiple data halls, fabs, or grid connections per target.
Summary: Right now, we have no idea how practical it is for countries to sabotage each others’ AI projects. This makes it hard to forecast what countries are going to do once the reality of superintelligence sets in: among other things, it’s unclear what military options middle powers have to slow the U.S. and China down, whether blockading Taiwan is a death blow to U.S. AI or a setback, and whether it’s realistic to expect AI deterrence to last for decades like MAD.
To help get better intuitions about this, I ran a quantitative model of how much destroying AI compute would set back the U.S. and China’s progress towards superintelligence. The long and short of it is that even extensive attacks (on compute alone) will only delay AI development by a handful of years, and can be significantly offset by nationalizing surviving compute. Rather than think of kinetic sabotage as a way to permanently stop a state from acquiring advanced AI, you should think of it as a ballpark 1-5 year delay. If this is realistic, then relying on kinetic attacks to maintain an indefinite stalemate on AI development would be incredibly risky and counterproductive. Instead, we should a) do more research into how deterrence at lower levels of escalation would work, and b) implement the ones that don’t trade off too much on our ability to later negotiate over AI development.
Views preceed and do not represent those of CAIS. Special thanks to Oscar Delaney for mentorship and feedback throughout this project.
If you blow something up, it stops working. As far as heuristics go, this is a pretty good one. It works for dams, bridges, Russian tanks, other missiles, and dictators fortunate enough to have to ruled Iran for 30 years yet unfortunate enough not to be named Fidel Castro.
In Superintelligence Strategy, Hendrycks and co apply this same heuristic to AI development. The descriptive argument is straightforward:
The normative argument is that we should encourage this dynamic, because it enforces a strategic equilibrium. If it’s impossible for any country to unilaterally develop superintelligence without getting their wings clipped, then AI development would need to proceed under (at least tacitly) mutually agreed terms.[2] Compared to the alternative of a no-holds-barred race, development would be both forcibly slowed down, leaving more breathing room to work on technical alignment, and more pluralistic, preventing a single country or company from dominating their competitors. Ergo, since sabotage is both inevitable and stabilizing, we should push the government to do it well: building more institutional capacity for sabotage, communicating a clear escalation ladder, and drafting up contingency plans for an attack.
Mutually Assured Destruction (MAD) vs. Mutually assured AI malfunction (MAIM).
Whether this is a good plan (or even a viable one) depends on how credible a deterrent sabotage is. After all, if the countries building superintelligence don’t expect their AI projects to get sabotaged, then there’s not much reason to slow down development. Most criticism of MAIM focuses on this problem from the perspective of political credibility, arguing that MAIMing attacks risk massive retaliation, lack of catalysts for action, and induce high risk-tolerance, making them poor deterrents.[3] Given these problems, the thinking goes, states can’t credibly threaten to damage each other’s AI infrastructure severely enough to actually shut AI development (and therefore deter it).
The problem with this debate is that we have no idea how hard it is to sabotage AI development. Sure, observability and communication are unavoidable problems. But they’d be simple to manage if shutting down AI development was as easy as drone striking a datacenter like you were hitting the Death Star’s exhaust port. Likewise, if the only way to destroy a datacenter was a direct land invasion, Xi could mail Congress his personal plans for world domination before they’d vote to march troops into Shanghai. Will to strike is downstream of capability to strike.
To get better intuitions about the viability of AI deterrence then, we need a model of its operational requirements: something that tells us how much AI infrastructure would need to be destroyed, how much destroying it would slow down development, and which offensive and defensive targeting strategies states could use. To that end, this post discusses the results of my model of kinetic strikes, as well as its main implications for AI deterrence. For the full model methodology and limitations, you can read the appendix of the original paper—otherwise, here’s a brief overview:
Motivations for Sabotage
Attacks on AI infrastructure are rightly seen as an extreme escalation over AI development, which can make them feel like unrealistic threats. To that end, it helps think about the mindset a state would have to have before it would consider kinetic strikes. I use the example of China below for concreteness, but the same dynamics apply to any state that isn’t the leader in AI development. For those only interested in the results of the model, feel free to skip to the following section.
By most measures, China is set to lose the AI race. Because the U.S. has access to so much more compute than its rival, and because the U.S. can so cheaply deny its rival’s growth by squeezing chip exports, the U.S. is much better positioned to both train powerful models earlier and to deploy them more extensively in automated AI R&D, and thus to acquire the military capabilities of superintelligence first. At the moment, this fact has mostly escaped the attention of either government. Insofar as there is a race to AI, it’s a race to integrate it into the economy, not to total military domination.
Still, the idea of superintelligence is no doubt stumbling through the bureaucracy on both sides of the Pacific (particularly in the U.S.), signal boosted by a handful of technocratic advisors and the outreach efforts of the AI companies. Eventually, in the course of pursuing general intelligence, those same companies are going to stumble onto AIs with extremely strategically relevant capabilities: superhuman hacking skills, amateur bioweapons uplift, autonomous loitering munitions, or any other powerful dual-use skills. By achieving these capabilities, the labs will fall firmly under the gaze of the state: not just as a woke thorn in their side, but as threats to its own monopoly on violence and geopolitical competitiveness.
Having drawn the eye of Sauron, the AI companies will have to bend the knee to Mordor, working with the government to both secure their models and develop new military applications of their technology. The securitization and success of these new American military projects will make China’s current position—that embodied AI and diffusion into the industrial base are the true path to competitiveness—harder to maintain. To stay competitive, its policy will need to shift towards facilitating the same software-based intelligence explosion the American frontier companies have been aiming at. But even with mass centralization of its own compute and outpouring of investment into domestic chip production, China can’t hope to catch up physically with the U.S.’s output, least of all when the U.S. has been investing so aggressively in its own compute buildout.
If China cannot outrun the U.S., then it will need to slow the U.S. down. The first and cheapest tools are the ones China already uses: espionage and cyber operations. Chinese intelligence services are no strangers to attacking U.S. tech and defense companies, and are (allegedly) already using AI tools in order to do so. It could attempt cyber operations directly against American datacenters, aiming to corrupt training runs, exfiltrate checkpoints, or disable cooling and power systems in Stuxnet-esque attacks. Some of these would succeed, at least initially. But cyberattacks on critical infrastructure tend to be one-shot weapons: each zero-day you expose through your attacks is one you burn to a future patch. After a few high-profile incidents, the major hyperscalers might (with government assistance) air-gap their most sensitive training clusters, silo their AI developers, and start properly screening employees for human intelligence.
At this point, China's options for closing the gap nonviolently start to thin out. It can continue to strengthen its own industrial policy: centralizing compute under state control, pouring money into SMIC and domestic HBM production, and trying to squeeze more algorithmic progress out of the compute it has. But these are slow-burn strategies that take years to pay off, and China might not have years. It could try to leverage its dominance in rare earth minerals and critical material processing to disrupt American supply chains, but these dependencies are less acute and more substitutable than the semiconductor chokepoint the U.S. holds over China.[7]
Under this kind of pressure, the Chinese government could crack in any number of unpredictable ways. The government might reach out and try to make a deal with the U.S. to slow down capabilities growth, or at least get some mutual verification of what capabilities they really have. They might regress into denial, defaulting to advisors who predict that AI will be important but not existentially dangerous: closer to hypersonic missiles than to nuclear weapons. Or maybe a spy reports back that the U.S. has a secret DARPA project aiming to use ASI for superweapons R&D and all hell breaks loose. Whichever path they take depends on how seriously they take the possibility of superintelligence and how much restraint the American government has shown in deploying its technological edge internationally.
In a scenario in which kinetic strikes are politically viable, you’d expect the following. First, it would have to be obvious to the Chinese government (such as through espionage) that the U.S. is actually on the cusp of superintelligence. Second, it needs to believe that superintelligence would allow the U.S. to disempower it (such as by overcoming nuclear deterrence or enabling a decapitation strike), and that the U.S. would aggressively use these powers to do so. Finally, it needs to assess that the costs of U.S. retaliation are worth accepting (i.e. that the chance of WW3 or outright nuclear war are sufficiently low). Such a strike would therefore be much more plausible if it were small in scope. If China could derail progress towards superhuman AI for several years by taking out just a handful of datacenters, it would be both able to accomplish this attack using less provocative means (ex: human insiders) and at a less provocative scale. A small, decisive strike could be framed as a limited action, paired with negotiations, making it harder for the U.S. to justify full-scale retaliation.
In practice, however, China (and the U.S.) would need to hit dozens upon dozens of targets in order to inflict a multi-year slowdown. In the model, we can see the practical effort required for these strikes directly.
Strike Scenarios
Using this model, we can quantitatively measure both the value of destroying rival compute and the number of clusters that would need to be targeted for a given level of delay. In the paper proper, we divide these into three strike scenarios, in ascending order of scope and aggression: minimal strikes, which target the supply chain or largest datacenters alone, medium strikes, which target them together, and maximal strikes, which involve persistent destruction of compute. If you want to check through the model yourself or run some tests, feel free to visit the repo here and play with the interactive model to test different scenarios.
Minimal Strikes
For the past few years, the datacenter buildout has been dominated by a hyperscale strategy: putting the majority of each country’s chips into increasingly massive clusters. Assuming that the communication, power, and regulatory incentives that are causing this concentration don’t change, most future compute is going to stay conveniently grouped into a handful of geographic targets.[8]
Extrapolated compute distributions, based on the Epoch cluster and frontier datasets. The bottom 1% of compute added each year is hidden for visual clarity.
Although attacks on AI infrastructure are going to be extremely escalatory no matter what, this level of consolidation could still encourage states to hold back a bit. If the majority of a rival state’s compute can be taken out by destroying (roughly) a few dozen sites, there might not be a need to risk a disproportionate counter-attack by targeting anything aside from the largest datacenters. To measure the effectiveness of this strategy, we can plot the delay from targeting 50-90% of present-day compute.
Despite the ostensible scale of the attacks, this has very little impact on on the overall development timeline. Fundamentally, the problem is that too much new compute is being brought online. On current trends, each year is going to keep introducing more compute than the last, both from the raw physical scaling of the chip supply chain and the constant improvement in the per-unit performance of those chips. Even if an attacker destroys upwards of 90% of the defender’s compute, most of that damage will be quickly washed out—assuming that each country decides to race through the sabotage rather than slow down their planned compute buildout.[9]
In order to avoid this washout effect and shrink the target count, states could instead opt for the alternative strategy of just directly attacking the AI supply chain. You can count the number of frontier semiconductor and CoWoS fabs on two hands, and destroying them would probably knock out more than 90% of future compute for years, even assuming ASML, the HBM manufacturers, and the many other suppliers escape unscathed. Even if we just limit the destruction to TSMC and SMIC to constrain the geopolitical scope to the U.S., Taiwan, and China, semiconductors and packaging would bottleneck the entire supply.
Although TSMC primarily supplies the U.S., it would still have a significant impact on Chinese AI development by reducing the availability of chips to smuggle.
Indeed, the peak delay of this strategy is nearly an order of magnitude higher. If most compute is going to exist in the future, then you can effectively “destroy” much more of it by shrinking production. The obvious problem with this approach is that it’s most useful early in AI development while development is immature, and then starts flatlining around the same time you get empirical evidence of powerful models. And at that point, most of the compute the defender needs will already be installed—forcing the attacker back to direct strikes if they need an emergency delay.
Medium Strikes
Since these two targeting strategies have complementary strengths and weaknesses, the next step up the kinetic escalation ladder is to combine them. After all, if you’ve already committed to mainland strikes on AI compute, the marginal risk of escalation you take on from increasing the scope of attacks to the supply chain is going to be pretty low. So assuming that states commit to a giant preventative strike on enemy compute and its suppliers, what happens?
Here, the main benefit is that the value of strikes don’t fall off as quickly—rather than getting much more peak delay, your strikes just stay effective for longer. Normally, supply chain attacks peter out in effectiveness because the amount of compute they prevent from coming online becomes irrelevantly small by the strike date. But by destroying so much compute upfront, the attacker increases dependence on future compute to offset the strikes, making the marginal reduction in future compute from supply chain strikes more meaningful.
Maximal Strikes
The last option states have for increasing the impact of their attacks (at least while still only targeting compute) would be to keep destroying new datacenters as they come online. Although this would be even more operationally demanding and make it much harder to later sue for peace, it should throttle the influx of new compute enough to meaningfully slow down AI development. Below, we plot out the effects of sustaining this kind of compute denial indefinitely.[10]
Unsurprisingly, this proves much more effective. Adding compute has diminishing returns to research progress; conversely, removing compute becomes more impactful the closer you get to destroying 100% of it. In this scenario, most chips that would be produced under the status quo are not produced due to destroyed fabs. Of the few chips that are produced, most are destroyed when they are placed in data centers, creating a double filter.
Defensive Offsets
Still, AI sabotage isn’t a one-turn game. If states are having their leading projects sabotaged, especially to the point it threatens the global economy or runs the risk of all out war, they’re going to adapt.
One of the simplest adaptations would be to nationalize compute and give it to the leading lab. After all, if you’ve just had your datacenters shut down through strikes on the homeland, commandeering your country’s militarily-irrelevant compute isn’t exactly beyond the pale. In the model, we treat centralization as a 90% sweep: nine-tenths of the country’s surviving compute (and of every buildout afterwards) is handed to the leading company, which then reallocates it entirely to AI R&D.[11]
Delay offset from centralizing compute the same month as a medium strike attack.
Despite the scale of the original medium strike scenario, for example, compute centralization nearly neutralizes the original strike! And in the case of the lighter minimum strike scenarios, the results can be even worse for the attacker:
Delay offsets from minimal attacks on the AI supply chain alone.
In these cases, the strikes are weak enough that they become involuntary industrial policy. It’s entirely plausible that China could try to slow down U.S. AI development by blockading Taiwan, for example, only for them to miscalculate and spook the USG into leaving the leading lab with more compute than they would have had otherwise.
Implications for AI Deterrence
So what do these results mean for MAIM, or at least this particular rung of the MAIM escalation ladder? From our modeling, destroying compute alone isn’t enough to force a stalemate over AI development. If a state is motivated enough to continue racing towards superintelligence despite the sabotage, they will eventually succeed.
The main problem is that it’s too easy for the defender to replace lost compute. However massive the initial strike, it can be partially compensated for by a) direct replacement with new clusters, b) internal concentration of surviving compute, and c) algorithmic efficiency gains reducing the need for compute in the first place. The key constraint on strike effectiveness is less whether the attacker can destroy enough compute, and more whether they can keep it from being replaced—and on that dimension, kinetic strikes are only partially effective. They can’t halt new datacenter construction without persistent attacks, can’t stop experiments on surviving hardware, and can’t undo algorithmic progress that has already happened.
This problem has difficult implications for kinetic AI deterrence. Namely:
Closing thoughts
Overall, it seems like states would have a difficult time denying rival AI projects by targeting compute alone. As long as the defending states is willing to race through the sabotage and pour their resources into reconstituting the leading project, the attacker probably can’t enforce a permanent state of deterrence by denial. And since the potential gains from dominating in AI are so massive, a situationally aware state would probably be willing to absorb massive amounts of damage (economic or otherwise) in order to be able to secure victory---all the moreso if escalation over AI has made them paranoid and risk tolerant.
Still, I don’t think these results should be read as a condemnation of AI deterrence in general. Even if direct strikes on AI compute are unacceptably inefficient and hostile, deterrence could still work if the lower rungs of the escalation ladder were threatening enough. It could be the case that cyberattacks on AI projects alone, for example, are sufficient to prevent states from deploying their AIs for automated R&D, or that economic sanctions and the threat of economic collapse are threatening enough on their own to dissuade national leaders. As with kinetic strikes at the start of this piece, however, we simply have no idea how efficiently these escalations would deter rival AI development—and until then, governments can have no faith that they will suffice as threats.
Acknowledgements
We would like to thank James Nicholas-Bryant, Matthew Gentzel, David Abecassis, Liam Patel, Adam Khoja, Bill Anderson-Samways, Will Anderson, Rhea Kanuparthi, Jason Hausenloy, and Sophie Kim for their feedback and support.
Concretely, we’re talking about the use of superintelligence to subvert MAD. This could happen through the design of exotic new technologies (superweapons that give a massive first strike advantage, or super-defenses that prevent retaliation), or through brute force industrial expansion (e.g. churning out a thousand interceptors for every ICBM).
Depending on the speed of takeoff and the ceiling of technology, the gap might be so large as to provide an “Overwhelming Strategic Advantage”, where victory is not only assured but basically costless for the attacker, the same way that the North Sentinelese would have no chance of slowing down a submarine. If superintelligent AIs are especially persuasive, for example, it could be possible to simply convince other governments to avoid retaliating in the first place.
For example, China might use its kinetic leverage over the U.S. in order to force concessions on mutual verification, making it harder for the U.S. to undercut MAIM in the future by building secret blacksites or taking advantage of secret algorithmic efficiency improvements. This would naturally subject China to demands for the same adversarial scrutiny from the U.S., reinforcing the MAIM equilibrium.
They might argue, for example, that similar incentives for an international agreement to prevent the development of ICBM defenses exist—and that the U.S. repeatedly violated those same agreements without any real fear of repercussion.
In this article, for example, I describe how scaling the equivalent of GPT-4 by ten orders of magnitude would allow for an AI that could trivially match the performance of an expert human virologist. Of course, we don’t have another 10 OOMs of GPUs lying around. In fact, we probably won’t get more than another 3 OOMs from hardware scaleups by 2030, even accounting for hardware efficiency improvements.
But while power constraints and chip production will eventually throttle the hardware buildout, physical compute is only half the story. The rest of the scaleup will come from algorithmic efficiency improvements: algorithmic insights make it computationally cheaper to achieve the same level of capability. If half as many FLOPs are needed to match the performance of say, GPT-4, then the model could be trained with just half the GPUs---conversely, your GPUs would be capable of yielding twice the “effective” FLOPs. In other words, the amount of compute that physically exists is distinct from how computationally useful that compute is, which is what we want to use the notion of effective compute to capture.
But while power constraints and chip production will eventually throttle the hardware buildout, physical compute is only half the story. The rest of the scaleup will come from algorithmic efficiency improvements: algorithmic insights make it computationally cheaper to achieve the same level of capability. If half as many FLOPs are needed to match the performance of say, GPT-4, then the model could be trained with just half the GPUs—conversely, your GPUs would be capable of yielding twice the “effective” FLOPs. In other words, the amount of compute that physically exists is distinct from how computationally useful that compute is, which is what we want to use the notion of effective compute to capture.
Since information on China’s compute shares is relatively sparse in the Epoch dataset, we instead based its share on the supply and demand side estimates from Zakaria and Corvino, which give China a ~14% share of global compute when compared to Epoch’s Chip Sales data. In our case, we also erode this relative share to 9% by the strike date, on the basis that the legal import, smuggling, and remote access pathways Zakaria pointed out will have been partially suppressed by then.
An algorithm that achieved the same performance on some benchmark with half the required training compared to the previous SOTA, for example, has “effectively” doubled the power of your compute (for the purposes of training to that benchmark). If we assume that these improvements are scale-independent (i.e., that they generalize to all scales of computation), they should similarly reduce the computational requirements of future AI models as well.
The actual source of these improvements varies considerably. Sometimes the improvements come from using an entirely new architecture, like the replacement of LSTMs with transformers (the scaling efficiency of which finally made LLMs viable to train). Other times, the improvements are as small as training the models on higher quality data. Once you zoom out though, most of these improvements are basically correlated with compute, which lets researchers run experiments and, increasingly, run AI labor to design and test experiments directly.
Mostly because the U.S. could secure rare earth imports from Australia and Canada, as well as internally by loosening its environmental standards.
As a sidenote, every delay chart shown is plotted in terms of percentage of national compute destroyed. In other words, they show what would happen if you managed to destroy a set amount of each country’s compute, regardless of how many targets it took to reach that point. The compute buildout graph below is essentially just an illustrative example of what targets each country will have at different points in time.
In practice, we would expect any kind of kinetic attack to have a chilling effect on the funding for private datacenter construction. Most investors—and the companies insuring them—are pretty averse to building billion dollar test targets. That leaves the question of whether the government could or would pick up the financial slack. If AI infrastructure is seen as critical enough to be worth striking, the government might well reason that it’s worth funding.
With that caveat in mind, you can consider these results in the context of the defending government intentionally racing through the kinetic sabotage, committing as many financial resources as they need to stick to the original development schedule.
Specifically, assuming that states continually destroy 90% of the new compute being brought online each month, after destroying 90% of the defender’s compute in the initial strike.
Specifically, into running AI research assistants internally, training models, and running experiments. Most importantly, it excludes customer inference, which was previously assumed to take up 37% of the leading lab’s frontier compute (shrinking to a ~9% relative share), since it doesn’t contribute to AI development directly.
The leader of a nuclear third power without any of its own AI infrastructure, for example, might reason that it has no choice but to threaten to strike the U.S. and China’s AI projects, expanding the window where it can negotiate from a position of strength.
And that’s just the target count: you still have to actually take out the site itself, which might require independently targeting and destroying multiple data halls, fabs, or grid connections per target.