The old Iran deal aimed to keep Iran at least one year away from having a nuclear bomb. Similar controls can be used to design an enforceable AI pause.
[Note: this is probably a bit underspecified in a few places. I deliberately accelerated the pace at which I wrote this piece, so as to get it out to the public as far ahead as possible of the Trump-Xi summit in late September 2026, and get people thinking about the core idea if it's useful]
To pause frontier AI development, you need to define the AI capabilities threshold that must not be passed. There are a number of ways to do so. You could freeze development at the currently-existing frontier, whether freezing each company at its current level of capabilities or allowing laggards to catch up to the leader’s level while going no further. You could define a dangerous-capability threshold and prohibit further progress beyond that point. You could set a maximum allowable amount of GPUs any one entity can hold, and/or the maximal allowed amount of compute that can go into any one training run. I argue that these approaches are suboptimal, and that the threshold should be defined so as to ensure it would take a skilled team with capacious resources at least some duration of time to progress from already-available (and permitted) capabilities to gravely dangerous levels: those capabilities levels at which an AI system could either take over control from humans on its own, or be used by individual humans to enable mass-casualty attacks, pandemics, and biological or chemical warfare. Additionally, the threshold should be adaptive rather than fixed, able to be adjusted on the fly (including downward reductions).
The key observation here is that any one AI model doesn’t exist in a vacuum. We are now at a stage where already-existing AI models significantly speed up the development of their successors, by empowering researchers to fine-tune systems, improve training pipelines, and investigate new avenues of research faster and more skillfully than the same researchers would be able to accomplish without this AI assistance. We are not yet at full recursive self-improvement, but every new frontier model (including models that are only deployed internally) closes a bit more of the loop, and speeds up the rate at which its own successor will be developed. Getting from 80% to 100% of the way to dangerous capabilities will be faster than getting from 60% to 80%, et cetera. Therefore, the higher you set the pause threshold, the less time it will take for rogue actors to achieve the same amount of capabilities advances above that threshold. It could very well be that 80% of the way is not in itself dangerous, but would allow a rogue actor to reach 100% of the way fast enough that we would be running the risk of not detecting them in time.
The solution to this problem is to set the pause threshold not based on the level of capabilities that is itself non-dangerous, but instead based on the level of capabilities from which it would take sufficient time for a rogue actor to advance to dangerous capabilities, that we can confidently assume we will be able to catch them well in advance of reaching that threshold. There is precedent for this type of institutional design, in the Joint Comprehensive Plan of Action (JCPOA), better known as the former Iran deal. While this deal broke down due to both geopolitical and domestic political pressures, this occurred for reasons largely exogenous to this specific design mechanism - and this deal was centrally between the US and Iran, a much more difficult and untrustworthy adversary than China (the primary counterparty of any feasible AI pause treaty), with whom our relations have historically been vastly more strained.
Nuclear Weapons and Powerful AI: Similarities and Differences
To understand why this approach is feasible for AI, let’s take a look at the commonalities between the supply chains for nuclear weapons and for frontier AI.
The research paper “Computing Power and the Governance of Artificial Intelligence” (Sastry, Heim, et al 2024) contains a useful discussion of the similarities between the raw materials needed to produce nuclear weapons, and those needed for powerful AI.[1] Both require a very specific material (enriched uranium and advanced GPUs, respectively). In both cases, the process to produce this material from its raw inputs requires cutting-edge technology whose supply chains can be restricted, monitored, and tracked. There are, of course, differences between the two processes: some which make the problem of supply-chain monitoring more tractable for advanced AI than for nuclear weapons; some which pose additional challenges for our problem and thus require additional considerations.
Centrifuges, the machines used to extract highly-enriched uranium from the raw yellowcake produced by uranium ore processing, require cutting-edge materials engineering, because they must spin at 50,000-100,000 RPM in order to separate the isotopes in uranium, taking advantage of the 1% weight difference between highly fissile U-235 and relatively inert U-238. To sustain this incredible rate of rotation, advanced control systems are required, as well as specialized metallurgy so that the extreme forces at play do not destroy the centrifuge itself. Strict supply-chain controls already exist for these components, and there are very limited numbers of suppliers, strictly controlled by state oversight. However, there aren’t true global monopolies. For everything from maraging steel to centrifuge rotors manufactured from aerospace-grade carbon fiber, there is generally at least one Western supplier, one Chinese supplier, and one Russian supplier, sometimes more. Parallel supply chains already exist.
AI chips, on the other hand, have a number of true monopolies, where only one company makes an essential component. And these essential components aren’t knickknacks, odds and ends, but are among the most complex pieces of machinery ever built. ASML, a Dutch company that builds the EUV lithography machines used to manufacture ultra-high-density computer chips, has no competition whatsoever. They are the only company in the world making these devices. They only produce about 60 of these machines per year, which individually cost $200 million to $400 million. Some of ASML’s suppliers are also monopolies: Carl Zeiss, a German company which produces the optics used within the EUV lithography machines, is also a true monopoly with no competitors also capable of producing optics to the same required level of precision. When only one company can produce a key component, the whole supply chain can be immediately locked down by imposing strict oversight - or even a total production halt - on that one company. This is made even more robust when the component is exceptionally complex and expensive, as EUV lithography machines are. It’s much easier to trace $400 million transactions than $40,000 transactions, and if an entity wanted to begin building its own such machines to route around the restrictions, they would first have to develop cutting-edge optics, cutting-edge lasers and much more, each of which has its own complex supply chain and talent pipelines which can also be monitored. [Of course, an effective AI pause would also require controls and monitoring on the already-existing supply of AI chips. The more chips an entity has, the faster they can progress from one stage of capabilities to the next, and the larger the models they can train.]
On the other hand, there are a few ways in which AI progress is more difficult to monitor than progress toward nuclear weapons. The first is that, as mentioned above, the rate of progress is nonlinear. A pound of enriched uranium is a pound of enriched uranium. Having 25 pounds of enriched uranium doesn’t in itself accelerate the rate at which you enrich the additional 25 pounds required to build a basic nuclear bomb. The more progress on AI you achieve, though, the faster you get to the next stages of AI. Secondly, the Iran deal was intended to bind a country that had not yet achieved any nuclear bombs. A large stock of AI chips already exists and is not yet well-monitored.[2] And, like the previous point, whereas a nuclear bomb is not a productive instrument for building more nuclear bombs, an AI chip is.
Conversely, though, the already-existing frontier AI models are much more centralized in terms of that which makes them powerful - their model weights. People all over the world use the latest and greatest models from OpenAI and Anthropic, but the weights themselves are not available to the public. You’re just calling an API, which queues a request on a server controlled by OpenAI/Anthropic, then forwards you the resulting output. This means that if you destroy the weights of the frontier models, you can push everyone’s starting point further back (though no further back than the most powerful currently-existing open-weights models). This is true not just of OpenAI and Anthropic themselves, but of everyone else too, because distillation of frontier models is a powerful way for competitors to quickly and inexpensively get to a capability level one or two model generations behind the frontier. Chinese AI companies, in particular, have been heavily dependent on distillation.
Setting the threshold
So, all this considered, where should the threshold be set? What should be the target for breakout time?
I advocate that, at bare minimum, the targeted breakout time should be two to three years, with dangerous AI capabilities being defined as full autonomous RSI and/or the ability to carry out bioweapon attacks targeting humans, for example by facilitating the creation and release of novel deadly viruses. This is a challenge, because I personally believe that at current rates of progress we are only 1-2 years from full RSI (and also because existing open-weights models are only 6-8 months behind the leading closed-weights models from OpenAI and Anthropic). Much AI pause rhetoric has focused on the idea that we only need to control future AIs, not roll back existing ones. However, in September 2026 I believe the situation on the ground has changed, and, for instance, GPT-6 is sufficiently capable so as to be concerning in its ability to accelerate AI research itself. I don’t think GPT-6 is itself capable of true RSI, but I do think it will drastically accelerate getting there. This means that GPT-6 being permitted under a pause would not be entirely farcical, but it would make things more difficult, closer to the knife-edge, with less room for error. My gut instinct is that we should destroy the weights of, at least, GPT-6 and Mythos/Fable 5.1.
Of course, there are multiple variables that can be controlled to manipulate the breakout time. For instance, if we drastically constrain access to AI chips, we can slow the rate of progress forward from a given starting point, compared to if we didn’t constrain chips as tightly. For instance, a pause in which GPT-6 is the most capable model and the max allowable accumulation of chips is N may have approximately the same breakout time as a pause in which Fable 5.0 is the most capable model and the max allowable accumulation of chips is 2N.
Why 2 to 3 years? First off, due to the nonlinearity of AI progress, it’s harder to predict than nuclear weapons development, where every pound of enriched uranium takes the same amount of work to produce, and having half the enriched uranium you need for a bomb doesn’t in itself speed up the process to produce the second half. The error bars are wide, and I want to be confident that the true breakout time will be at least one year, which allows time for somewhat sluggish human institutions to react. Yes, that’s somewhat arbitrary, but any threshold is arbitrary, and many organizations have their top-level meetings once a year, for instance. All in all, I’d much rather set up a pause with a breakout time that we think is 3 years, and risk it actually being 1 year, versus setting it up with a breakout time of 1 year, that turns out to actually be 4 months. If that means destroying the weights of some models, so be it.
Flexible adaptation
The threshold should be a starting point, not an unchangeable contract. A robust AI pause will necessarily include prohibitions on further research on AI algorithmic efficiency improvements, and this will presumably effectively stop most such research. However, over a sufficient period of time it’s likely that some amount of this type of research would be illicitly conducted, and then released for the whole world to see. In such an event, facilitators of the pause must be able to respond to this change in circumstances. For instance, imagine the allowable threshold is set at Model 8, which is 20% more powerful than Model 7, and that the most capable existing open-weights models are only as powerful as Model 5. Say someone leaks a method that improves algorithmic efficiency by 20%. In such a case, the authority overseeing the pause ought to strongly consider ordering destruction of the weights of Model 8, and pushing back the permitted capabilities threshold to Model 7. Note that I’m aware that capabilities and chip efficiency are somewhat orthogonal; this is meant only to be a toy example [though improved chip efficiency still can improve capabilities by freeing up compute for more tokens to be thrown at any given query, enabling inference providers to add a new tier of thinking depth].
Flexibility should also be applied not just to the permitted AI capabilities threshold, but to other aspects of control as well. For instance, if a rogue actor releases a powerful open-weights model that exceeds the permitted capabilities threshold, the maximum allowable compute held by any one entity should be reduced correspondingly.[3] That would effectively recognize that the permitted capabilities threshold has been forcibly advanced, and reduce entities’ potential rate of advance correspondingly.
A robust pause will necessarily include scope for kinetic measures against particularly harmful rogue actors and their resources (e.g. data centers), but that’s beyond the scope of this essay.
The old Iran deal aimed to keep Iran at least one year away from having a nuclear bomb. Similar controls can be used to design an enforceable AI pause.
[Note: this is probably a bit underspecified in a few places. I deliberately accelerated the pace at which I wrote this piece, so as to get it out to the public as far ahead as possible of the Trump-Xi summit in late September 2026, and get people thinking about the core idea if it's useful]
To pause frontier AI development, you need to define the AI capabilities threshold that must not be passed. There are a number of ways to do so. You could freeze development at the currently-existing frontier, whether freezing each company at its current level of capabilities or allowing laggards to catch up to the leader’s level while going no further. You could define a dangerous-capability threshold and prohibit further progress beyond that point. You could set a maximum allowable amount of GPUs any one entity can hold, and/or the maximal allowed amount of compute that can go into any one training run. I argue that these approaches are suboptimal, and that the threshold should be defined so as to ensure it would take a skilled team with capacious resources at least some duration of time to progress from already-available (and permitted) capabilities to gravely dangerous levels: those capabilities levels at which an AI system could either take over control from humans on its own, or be used by individual humans to enable mass-casualty attacks, pandemics, and biological or chemical warfare. Additionally, the threshold should be adaptive rather than fixed, able to be adjusted on the fly (including downward reductions).
The key observation here is that any one AI model doesn’t exist in a vacuum. We are now at a stage where already-existing AI models significantly speed up the development of their successors, by empowering researchers to fine-tune systems, improve training pipelines, and investigate new avenues of research faster and more skillfully than the same researchers would be able to accomplish without this AI assistance. We are not yet at full recursive self-improvement, but every new frontier model (including models that are only deployed internally) closes a bit more of the loop, and speeds up the rate at which its own successor will be developed. Getting from 80% to 100% of the way to dangerous capabilities will be faster than getting from 60% to 80%, et cetera. Therefore, the higher you set the pause threshold, the less time it will take for rogue actors to achieve the same amount of capabilities advances above that threshold. It could very well be that 80% of the way is not in itself dangerous, but would allow a rogue actor to reach 100% of the way fast enough that we would be running the risk of not detecting them in time.
The solution to this problem is to set the pause threshold not based on the level of capabilities that is itself non-dangerous, but instead based on the level of capabilities from which it would take sufficient time for a rogue actor to advance to dangerous capabilities, that we can confidently assume we will be able to catch them well in advance of reaching that threshold. There is precedent for this type of institutional design, in the Joint Comprehensive Plan of Action (JCPOA), better known as the former Iran deal. While this deal broke down due to both geopolitical and domestic political pressures, this occurred for reasons largely exogenous to this specific design mechanism - and this deal was centrally between the US and Iran, a much more difficult and untrustworthy adversary than China (the primary counterparty of any feasible AI pause treaty), with whom our relations have historically been vastly more strained.
Nuclear Weapons and Powerful AI: Similarities and Differences
To understand why this approach is feasible for AI, let’s take a look at the commonalities between the supply chains for nuclear weapons and for frontier AI.
The research paper “Computing Power and the Governance of Artificial Intelligence” (Sastry, Heim, et al 2024) contains a useful discussion of the similarities between the raw materials needed to produce nuclear weapons, and those needed for powerful AI.[1] Both require a very specific material (enriched uranium and advanced GPUs, respectively). In both cases, the process to produce this material from its raw inputs requires cutting-edge technology whose supply chains can be restricted, monitored, and tracked. There are, of course, differences between the two processes: some which make the problem of supply-chain monitoring more tractable for advanced AI than for nuclear weapons; some which pose additional challenges for our problem and thus require additional considerations.
Centrifuges, the machines used to extract highly-enriched uranium from the raw yellowcake produced by uranium ore processing, require cutting-edge materials engineering, because they must spin at 50,000-100,000 RPM in order to separate the isotopes in uranium, taking advantage of the 1% weight difference between highly fissile U-235 and relatively inert U-238. To sustain this incredible rate of rotation, advanced control systems are required, as well as specialized metallurgy so that the extreme forces at play do not destroy the centrifuge itself. Strict supply-chain controls already exist for these components, and there are very limited numbers of suppliers, strictly controlled by state oversight. However, there aren’t true global monopolies. For everything from maraging steel to centrifuge rotors manufactured from aerospace-grade carbon fiber, there is generally at least one Western supplier, one Chinese supplier, and one Russian supplier, sometimes more. Parallel supply chains already exist.
AI chips, on the other hand, have a number of true monopolies, where only one company makes an essential component. And these essential components aren’t knickknacks, odds and ends, but are among the most complex pieces of machinery ever built. ASML, a Dutch company that builds the EUV lithography machines used to manufacture ultra-high-density computer chips, has no competition whatsoever. They are the only company in the world making these devices. They only produce about 60 of these machines per year, which individually cost $200 million to $400 million. Some of ASML’s suppliers are also monopolies: Carl Zeiss, a German company which produces the optics used within the EUV lithography machines, is also a true monopoly with no competitors also capable of producing optics to the same required level of precision. When only one company can produce a key component, the whole supply chain can be immediately locked down by imposing strict oversight - or even a total production halt - on that one company. This is made even more robust when the component is exceptionally complex and expensive, as EUV lithography machines are. It’s much easier to trace $400 million transactions than $40,000 transactions, and if an entity wanted to begin building its own such machines to route around the restrictions, they would first have to develop cutting-edge optics, cutting-edge lasers and much more, each of which has its own complex supply chain and talent pipelines which can also be monitored. [Of course, an effective AI pause would also require controls and monitoring on the already-existing supply of AI chips. The more chips an entity has, the faster they can progress from one stage of capabilities to the next, and the larger the models they can train.]
On the other hand, there are a few ways in which AI progress is more difficult to monitor than progress toward nuclear weapons. The first is that, as mentioned above, the rate of progress is nonlinear. A pound of enriched uranium is a pound of enriched uranium. Having 25 pounds of enriched uranium doesn’t in itself accelerate the rate at which you enrich the additional 25 pounds required to build a basic nuclear bomb. The more progress on AI you achieve, though, the faster you get to the next stages of AI. Secondly, the Iran deal was intended to bind a country that had not yet achieved any nuclear bombs. A large stock of AI chips already exists and is not yet well-monitored.[2] And, like the previous point, whereas a nuclear bomb is not a productive instrument for building more nuclear bombs, an AI chip is.
Conversely, though, the already-existing frontier AI models are much more centralized in terms of that which makes them powerful - their model weights. People all over the world use the latest and greatest models from OpenAI and Anthropic, but the weights themselves are not available to the public. You’re just calling an API, which queues a request on a server controlled by OpenAI/Anthropic, then forwards you the resulting output. This means that if you destroy the weights of the frontier models, you can push everyone’s starting point further back (though no further back than the most powerful currently-existing open-weights models). This is true not just of OpenAI and Anthropic themselves, but of everyone else too, because distillation of frontier models is a powerful way for competitors to quickly and inexpensively get to a capability level one or two model generations behind the frontier. Chinese AI companies, in particular, have been heavily dependent on distillation.
Setting the threshold
So, all this considered, where should the threshold be set? What should be the target for breakout time?
I advocate that, at bare minimum, the targeted breakout time should be two to three years, with dangerous AI capabilities being defined as full autonomous RSI and/or the ability to carry out bioweapon attacks targeting humans, for example by facilitating the creation and release of novel deadly viruses. This is a challenge, because I personally believe that at current rates of progress we are only 1-2 years from full RSI (and also because existing open-weights models are only 6-8 months behind the leading closed-weights models from OpenAI and Anthropic). Much AI pause rhetoric has focused on the idea that we only need to control future AIs, not roll back existing ones. However, in September 2026 I believe the situation on the ground has changed, and, for instance, GPT-6 is sufficiently capable so as to be concerning in its ability to accelerate AI research itself. I don’t think GPT-6 is itself capable of true RSI, but I do think it will drastically accelerate getting there. This means that GPT-6 being permitted under a pause would not be entirely farcical, but it would make things more difficult, closer to the knife-edge, with less room for error. My gut instinct is that we should destroy the weights of, at least, GPT-6 and Mythos/Fable 5.1.
Of course, there are multiple variables that can be controlled to manipulate the breakout time. For instance, if we drastically constrain access to AI chips, we can slow the rate of progress forward from a given starting point, compared to if we didn’t constrain chips as tightly. For instance, a pause in which GPT-6 is the most capable model and the max allowable accumulation of chips is N may have approximately the same breakout time as a pause in which Fable 5.0 is the most capable model and the max allowable accumulation of chips is 2N.
Why 2 to 3 years? First off, due to the nonlinearity of AI progress, it’s harder to predict than nuclear weapons development, where every pound of enriched uranium takes the same amount of work to produce, and having half the enriched uranium you need for a bomb doesn’t in itself speed up the process to produce the second half. The error bars are wide, and I want to be confident that the true breakout time will be at least one year, which allows time for somewhat sluggish human institutions to react. Yes, that’s somewhat arbitrary, but any threshold is arbitrary, and many organizations have their top-level meetings once a year, for instance. All in all, I’d much rather set up a pause with a breakout time that we think is 3 years, and risk it actually being 1 year, versus setting it up with a breakout time of 1 year, that turns out to actually be 4 months. If that means destroying the weights of some models, so be it.
Flexible adaptation
The threshold should be a starting point, not an unchangeable contract. A robust AI pause will necessarily include prohibitions on further research on AI algorithmic efficiency improvements, and this will presumably effectively stop most such research. However, over a sufficient period of time it’s likely that some amount of this type of research would be illicitly conducted, and then released for the whole world to see. In such an event, facilitators of the pause must be able to respond to this change in circumstances. For instance, imagine the allowable threshold is set at Model 8, which is 20% more powerful than Model 7, and that the most capable existing open-weights models are only as powerful as Model 5. Say someone leaks a method that improves algorithmic efficiency by 20%. In such a case, the authority overseeing the pause ought to strongly consider ordering destruction of the weights of Model 8, and pushing back the permitted capabilities threshold to Model 7. Note that I’m aware that capabilities and chip efficiency are somewhat orthogonal; this is meant only to be a toy example [though improved chip efficiency still can improve capabilities by freeing up compute for more tokens to be thrown at any given query, enabling inference providers to add a new tier of thinking depth].
Flexibility should also be applied not just to the permitted AI capabilities threshold, but to other aspects of control as well. For instance, if a rogue actor releases a powerful open-weights model that exceeds the permitted capabilities threshold, the maximum allowable compute held by any one entity should be reduced correspondingly.[3] That would effectively recognize that the permitted capabilities threshold has been forcibly advanced, and reduce entities’ potential rate of advance correspondingly.
https://arxiv.org/pdf/2402.08797 § The Compute-Uranium Analogy (page 75 of PDF)
Much has already been written about how to achieve chip controls, so I’m leaving that outside the scope of this essay.
A robust pause will necessarily include scope for kinetic measures against particularly harmful rogue actors and their resources (e.g. data centers), but that’s beyond the scope of this essay.