In this month’s meetings, I expect China to again downplay safety risks. It could dangle the possibility of safety cooperation in exchange for concessions such as the softening of chip controls. I think it will continue to portray itself as an altruistic savior dedicated to ensuring that the developing world gains A.I. access.
Earlier this week, representatives for the U.S. and China discussed a notification mechanism to increase transparency on national security incidents involving AI, but export controls were not on the agenda for those talks. I think that was a mistake. A bilateral treaty is entering the Overton window, and sending more chips to China is a price worth paying to secure such an agreement. When Trump and Xi meet at the White House to discuss AI tomorrow, removing export controls should be on the negotiating table.
TLDR: Export controls are overrated. For export controls to earn their spot as a top policy priority, a conjunction of strong premises must hold. But there are reasons to doubt each of those premises, as well as reasons that export controls could be net negative, in particular by hastening RSI in the U.S. and making diplomacy more difficult. Instead, we should prioritize other policies. The case for a bilateral treaty, for one, or at least diplomacy toward such an agreement, is more robust, resting on weaker assumptions and posing less downside risk.
Questioning the Case for Export Controls
There's been very little clarity and very little agreement on what the goal of these export controls is and what the theory of change is.
Suppose you are concerned with risks from misalignment (rather than hegemony or great power conflict, which I address later). What's the main theory of change for export controls?[1] We could break it down like this:
The U.S. should try to develop ASI before China.
Export controls slow down China.
Export controls speed up the U.S.
The U.S. will pace or pause on the finish line to spend its lead time.
The U.S. will spend that lead time on alignment research.
That research will make the intelligence explosion more likely to go well.
This is a textbook galaxy brained plan! The primary effect of export controls is to increase the concentration of chips in the leading country, accelerating frontier AI development. To think that other effects can outweigh that harm, we have to be really confident in our story for why. But there are reasons to doubt each of these premises, which together trouble the overall case for export controls.
I handle broader objections to (1) in the section on China threat models.
Do Export Controls Slow Down China? (2)
If Training Progress Is Exogenous
Let me start with a possibility that would largely foreclose the case for export controls: if the rate of training progress in China does not depend on the marginal chips that export controls influence at all; that is, if DeepSeek is going to train V5 according to the same schedule, regardless of whether it has access to the chips that would allow it to serve more instances. This doesn't seem likely, but it bears mentioning because it would significantly mitigate the solvency of export controls.
Now, if this is true in China, you could counter that it would certainly be true in the U.S. In other words, whatever OpenAI data center is going to be the one to train the first RSI-level model is going to have enough chips whether or not the U.S. lets some chips go to China.
But that's still an argument against export controls! During RSI, the leading U.S. lab can allocate at most 100% of their chips to fueling the intelligence explosion. So in this world, where training progress is exogenous, the marginal chips just determine the maximum speed of RSI—and the eventual diffusion of ASI.[2] On both counts then, I'd rather see those marginal chips go to China. It's unlikely that those chips will mean the difference between a deadly and a safe intelligence explosion in the U.S., but it would be directionally helpful.[3]
Here's a cheekier and much more inflammatory way to express that sentiment: "So you want to pace the frontier? Send more chips to China."
If Training Progress Is Endogenous
But let's set this consideration aside, and instead consider the more ambiguous case, in which the marginal chips do affect training progress. This case also seems much more likely.[4]
In this case, China's response could mitigate the overall slowdown. It could be that:
China is incentivized and has the political will to implement some amount of unilateral regulation, and, at the same time, it's not racing maximally hard right now. In that case, export controls could force China to race harder to keep up with the U.S., including by waking up more of the CCP and increasing the pressure for China to take significant efforts to catch up. For example, China could de-regulate or pursue nationalization (at least earlier than it would have otherwise). The result would be net-less wall clock time to ASI—relative to the world without export controls, in which China pursues more domestic regulations.
Loopholes like smuggling and remote access undermine the controls. Plus, you know, just outright stealing the weights, through cyberattacks or espionage.[5]
If distillation of U.S. models is a key driver of Chinese capabilities, that helps keep China tethered to the pace of progress in the U.S. (Though of course more chips would still help with other aspects of training, including RL.)
One argument I'm not making: I think some people argue that export controls have actually backfired by speeding China up on net due to compensation from China's domestic chip industry, but this still seems very unlikely to me.[6] The strongest evidence is that China itself has banned Nvidia chips, revealing that it may have some faith in import substitution.
Do Export Controls Speed Up The U.S.? (3)
Yes! And that's bad!
Will the U.S. Pace or Pause on the Finish Line? (4)
The recent efforts toward pacing the frontier have been really encouraging. The possibility that the U.S. labs will coordinate to pace seems much more likely than it did for the past few years, when it sounded more like a pipe dream. But pacing (and certainly a pause) that meaningfully slows the U.S. still looks unlikely to me. Not only is there no brake, no one knows what a brake would look like.
Embedded evaluators might lack the access and technical sophistication to uncover risks from AIs or false statements from the companies. Even if the evaluators raise an alarm, the public might not respond. (Consider, for example, that METR assessed the automated R&D section of Anthropic's risk report and wrote, "We do not think the report adequately supports its conclusion." Anthropic's leaders weren't dragged in front of Congress then, I don't think they would be now.)
Enforcement concerns aside, the recent pacing statements from lab leaders are a long way from a fully specified agreement. Without a specific commitment, the evaluators have nothing to audit, no matter how much access they get.
But suppose we do wind up in a regime with specific commitments (e.g., something as concrete as "FLOPs used on training and internal deployments cannot exceed X threshold this year"), and evaluators are empowered to audit the company against those commitments. There will still be enormous pressure for each company to defect on the precipice of RSI.[7]
Will further warning shots galvanize the political will for a real pause? I do expect more warning shots, but I think they're more likely to cause the U.S. government to nationalize the labs. That does obviate the coordination problem, but I think it replaces it with something worse: consolidating all the labs' compute under one project, false assurance that the government has got this, and reduced public transparency compared to what embedded evaluators previously provided. At that point, I don't expect the U.S. government to cede any of its lead to China.[8]
If pacing fails, with or without nationalization, the main effect of export controls is to increase the maximum speed of RSI in the U.S.
Okay, but suppose pacing proposals have some solvency. The argument in favor of export controls is that the U.S. will pace until it has spent its lead: if the U.S. has more chips, it can afford to pace for longer.
The problem with that is the companies in the U.S. will have different tolerances for how much of their lead they're willing to burn, and the measures of that lead are all ambiguous.
Consider the status quo: some benchmarks suggest that China is nearly a year behind, while others make it look like Chinese labs are nipping at the heels of their U.S. competitors. Though Dario has been the most supportive of pacing among U.S. lab CEOs, he is also the most paranoid about China. I can't really imagine him being comfortable pausing for more than a couple months (or the equivalent, spread out over the next few years).
Will the U.S. Spend Its Lead on Alignment? (5)
Say the U.S. companies agree on a measure of their lead and how much of it to spend. That buys more time for alignment research, maybe up to an additional 6 months or a year of wall clock time (and more when you weight that time by the number and quality of automated alignment researchers that become available during that period).
Some researchers might have to turn from their alignment agendas to work on the technical foundations of the pause, but that still seems really valuable.
And, ideally, export controls mean alignment researchers also have more compute than they would have otherwise during the entire period to ASI, not just during the marginal months from pacing.
I have a few counter-arguments to this story:
Some progress in capabilities will probably leak through the pacing agreement. It would have to be an exceptionally strict agreement if it blocks all experiments that could lead to big jumps in training quality or inference efficiency.
The increased time and chips will also lead to diffusion. The economy will become more AI-shaped during that time, and AIs will become more deeply embedded in critical infrastructure and the military. Then if a misaligned AI goes rogue, including because a company defected on its agreements, that creates new affordances for the AI to gather resources and launch attacks.
If your goal is to increase the safety budgets of the AI companies, there are more direct ways to do that, including just advocating for pacing! But also: giving demos to policymakers, protesting the labs, supporting whistleblowers, drawing attention to warning shots, or organizing lab employees to improve their negotiating position. In comparison, advocating for export controls offers way less leverage as a means to increase safety research.
But I don't weigh these too heavily; overall I agree the U.S. will spend at least some of the lead on alignment research, and that will lead to net more alignment research. (Again, that's conditional on pacing. In the absence of a pacing agreement, more chips means faster training progress, so net less alignment research).
Will That Research Make the Intelligence Explosion More Likely to Go Well? (6)
I don't think so. Even if everything goes right with the research—a full year of extra time, every export controlled chip goes to safety, no unintended capabilities progress, no diffusion—I still don't think we will have solved alignment. The better hope for that is a bilateral treaty with China that can last a lot longer than a year.
Export Controls Might Make Diplomacy Harder
It could be that China would be willing to come to the negotiating table for a bilateral treaty, but export controls make diplomacy more difficult by:
Raising the temperature on relations between the countries and
Making it in China's interest to deny the risks that are motivating the U.S. For instance, a spokesperson for China’s Ministry of Foreign Affairs recently responded to calls for pacing in the U.S., saying, "Fearmongering and engaging in confrontation and malicious competition will only disrupt the global AI governance process and serve no one’s interests."
To the extent it's a bargaining chip then, better to spend it sooner, to take down the temperature and reduce the pressure for such "fearmongering" dismissals to seep into the discourse in China.[9]
The threat of re-imposing export controls would still be a point of leverage in future discussions, but it's better to remove them now than to leave them in place, which has the added disadvantage of allowing China to adjust to the controls, including by more heavily prioritizing domestic chip manufacturing.
You could counter that China will have a stronger incentive to negotiate, the further behind it is. This is the position that Dario takes in his pacing essay:
If we execute these measures well, I believe they would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopolitically most important.
Some may believe these measures make it more difficult to cooperate with China, but I believe the opposite is true: these measures increase the leverage held by democracies and make an agreement more likely in the future.
I'm like, maybe? All he offers is an assertion. I think it's true that a stronger U.S. lead at the time a treaty is signed will result in better terms for the U.S., but as I discuss further down, I'm not convinced that locking in stronger U.S. leadership will be good for the world. More importantly, the terms of the treaty are secondary to the treaty getting signed in the first place. A couple reasons I think Dario misses the mark in his assertion that a wider gap makes an agreement more likely:
He takes for granted that the U.S. will be willing to come to the negotiating table. If China stands a chance at beating the U.S. to ASI, the U.S. will have a greater interest in signing on to a treaty.[10]
Even if being behind makes China more interested in slowing the U.S., there is some ceiling to this effect. As soon as the gap is wide enough that China recognizes the U.S.'s decisive lead, China is maximally incentivized to get the U.S. to slow down. The U.S. currently holds such a lead, so it seems unnecessary to lengthen it.
The main determinant of China's willingness to slow down is its belief in the dangers of advanced AI—from both countries—and the closer China is to the U.S., the more spooked Chinese labs are likely to be by the capabilities of their internal models. Moreover, if the Chinese labs under-invest in safety relative to U.S. labs, they will likely get more warning shots, which will further safety pill them.
Overall, this question about direction of the effect of a wider gap on the likelihood of a deal is a huge crux about export controls, and I'm not willing to take Dario's word for it. As long as we have significant uncertainty about this question, I don't think we can prioritize export controls over more commonsense policies.
Summing Up
These considerations mitigate the case for export controls significantly, and suggest that export controls could even be net-negative in terms of misalignment risk, mostly by speeding up capabilities progress in the U.S., and perhaps by making diplomacy harder.
I don't think any of these arguments is a slam dunk. It could still be that the 6-point story for export controls is true. But I think they muddy the waters enough that export controls can not be held up as an obviously high-priority policy.
Next, I address the first premise, that the U.S. should beat China to ASI. In principle, this isn't a necessary part of the theory of change for export controls. As in, you could conceivably think export controls reduce misalignment risks while being indifferent about which country is first to ASI. But in practice, I think most people who advocate for export controls present them as a means to beat China.[11] Inversely, if you don't buy one of the following three stories for why the U.S. should try to "win" the "race," you probably never reach the point of evaluating export controls as a tactic.
Against China Threat Models
What are the justifications for anti-China concerns? The main stories focus on misalignment, hegemony, or great-power conflict. For each of them, it's not clear why the U.S. beating China to ASI makes the world safer. Proceed by cases:
Misalignment
We should put in place laws and regulations, technological monitoring, early warning and emergency response systems in order to strengthen the line of security, prevent abuses and malicious use and ensure that AI is always under human control. In the meantime, we should jointly oppose overstretching the national security concept in the field of AI or placing one country's security over that of others.
First, suppose that ASI is winner-takes-all, in the sense that once one ASI has been created, it will take steps to prevent the creation of competing models. Here, we just compare P(aligned | U.S.) against P(aligned | China). It's not obvious to me why the U.S. would be better? For sure, the U.S. has done more alignment research, but U.S. models still have glaring alignment issues. See, for example, the Hugging Face incident, or testing of recent models, such as METR's evaluation of GPT-5.6 Sol.
Maybe people would object that Chinese models are misaligned in the sense that they are trained for Chinese values? But first, U.S. models have their own Western values, and it's not obvious which bias is worse, and second, it's not clear that these biases really count as misalignment in a sense that jeopardizes the value of the longterm future.
The stronger version of this objection is that China will have worse scalable oversight research during RSI, making misalignment more likely, and weaker control protocols, making it easier for a misaligned model to sabotage R&D. First, notice that these are also arguments for sharing scalable oversight research and control protocols with Chinese labs! (Maybe if China took the lead, that's more likely to happen...)
Second, I don't expect the U.S. to do a much better job of automated alignment research or control than China: automated alignment research seems fundamentally really hard and our control protocols aren't ready for ASI-level models.
If the World Is Multipolar
Suppose that ASI is not winner-takes-all; instead, both countries (and then all countries, and then all individuals) will be able to train an ASI. Why might that be? The base case is that the capability to spark an intelligence explosion is a fixed-capability target, and there's nothing a U.S. ASI could do to stop Chinese research from eventually crossing that threshold. Indeed, it might be even easier to create competing ASIs once the first one has been created because of espionage (including stealing the weights outright), blackmail, and distillation.
Here, what matters is whether the ASIs are aligned and whether an aligned ASI can out-compete a misaligned ASI.
Will the ASIs be aligned? It seems to me that the further behind China perceives itself to be, the more pressure there will be to cut corners on safety, making it more likely we end up in a world with at least one misaligned ASI.
If the U.S. is far ahead, does that at least make it more likely that our glorious freedom-loving ASI can pummel their wicked socialist ASI? Probably: a head start can only help, and our ASI will have more access to inference compute (with or without export controls).[12] Some reasons to think this would not hold are:
Offense-defense balance: The ability to bomb the other side's data centers is an early, fixed-capability target (and just because your adversary gets smarter doesn't make their data centers any more protected).
Scaling wall: The intelligence explosion quickly runs into physical limits and flatlines, so a head start is quickly lost.
But these are not dispositive. Overall, I agree that a head start would help an aligned U.S. ASI beat a misaligned Chinese ASI. But for that head start to be worth it, you have to think the benefit outweighs the harm of the greater likelihood that both models are misaligned. That's not at all intuitive to me. At best it's ambiguous.
Here's a simple model
Suppose America chooses whether to race. If it races, its ASI will gain a decisive advantage over China's.
(In this model, it doesn't matter how much marginal misalignment probability China takes on by racing. A better model would include uncertainty over whether the U.S. will cross the threshold for a decisive advantage, and then China's marginal risk should factor in.)
If is close to , it's not worth taking on marginal misalignment probability. Likewise if is close to . Recall that I expect both values to be close because automated alignment is hard and the U.S. can share scalable oversight research. Assuming , we can further reduce to:
Graphically, the maximum amount of additional misalignment we should be willing to pay as a function of the initial probability of misalignment looks like this:
A few implications of this model:
is maximized when . So people who think an aligned U.S. ASI is a coin toss should be willing to race the most.
But at that point, they should not be willing to increase the probability of misalignment by more than 0.25. So no one should be willing to race if it looks like the increased risk of misalignment exceeds that amount.
And people who are more optimistic and pessimistic on alignment should be less willing to take on additional risk.
Note that diplomacy aimed at a coordinated pause makes it more likely we end up with two aligned ASIs.
Couldn't the U.S. just pre-commit to giving China (and eventually every other country) a copy of its aligned ASI to limit the number of intelligence explosions to 1? I think that kind of arrangement has been under-explored in AI policy so far. If the outside option is the U.S. races to a misaligned ASI, maybe provoking World War III in the process, there should be a lot of room to negotiate an ASI sharing deal that benefits both sides.
Hegemony
By enabling AI’s benefits to be broadly shared, Chinese open models could win international goodwill and position China as an AI benefactor to countries across the developing world, including in Africa, Asia, Latin America, and the Middle East. . . . By preventing China from obtaining the tools needed to fabricate advanced chips for AI and by blocking Chinese firms’ access to U.S.-designed AI chips, American-led export controls likely will hinder China’s ability to spread its models widely.
Alternatively, let's assume automated alignment will pan out during the intelligence explosion. As a result, a country can weaponize its ASI, or at least direct it as an instrument of hegemony.
Here, the story people tell for beating China is that if China gets to ASI first, it would abuse its ASI to spread global totalitarianism, but if the U.S. gets there first, it would distribute the benefits around the world.
What are the channels for global influence?
ASI -> military power
Economic growth -> military power
Economic growth -> other countries become more dependent for trade and investment
Economic growth -> more cultural exports and soft power
AI diffusion -> soft power
I buy that these channels would contribute to hegemony, but I don't think it follows that the U.S. must seize them before China.
I distrust the U.S. to responsibly exercise global control about as much as I distrust China. This is the branch of this whole debate that I find most difficult to argue with people because the crux often comes down to people's sense of whether the U.S. has a good or bad historical track record. But to sketch a brief case for bad track record: the U.S. has a long (and recent) history of imperialism, wars, massacres, coups, resource extraction, and other foreign interventions. A lot of the world reads that history as evidence the U.S. is a bad-faith actor. The only time in history that one country had nuclear weapons, it was the U.S., and the U.S. used them to kill over 100,000 civilians. Dario doesn't know if Claude was responsible for the Minab school attack earlier this year.
Setting aside this history and how to interpret it, I doubt that the U.S. would share much of the economic windfall from ASI with the rest of the world. In order of decreasing self-determination for the recipient countries:
If export controls are in place, by construction the U.S. won't share chips (and presumably semiconductor manufacturing equipment as well).[13]
The Fable fiasco shows that for even low dual use capabilities, the U.S. won't share model access.[14]
I don't have high hopes that the U.S. will share cash.[15] It's not very "America First," and USAID wasn't able to put up much of a fight against DOGE.
Will the leading AI lab itself pursue redistribution? No:
If the labs have been nationalized, they probably won't have the autonomy to do it.
If they are still companies (PBC or otherwise), they'll be public, and their shareholders will sue them.
One cause for hope is that the U.S. has programs to share civil nuclear energy technology with other countries, including by supplying reactors. But:
Crucially, those programs emerged in a bipolar world, in which the U.S. was competing against Russia to extend these deals. More recent programs have sought to compete against China, as well.
The early deals backfired, contributing to weapons programs in India, as well as Pakistan and Israel via deals with U.S. allies.
AI could be harder to monitor for misuse than nuclear enrichment.
So the nuclear precedent doesn't give me that much confidence that the U.S. will tolerate global diffusion in AI.
If we're going to have a unipolar world, China as hegemon seems more likely to distribute the AI windfall:
The Marxist roots of CCP doctrine emphasize international solidarity. It's debatable how much those principles are motivating the party these days, but China is still more multilateralist than the U.S. after its America First turn.
The CCP is more incentivized to regulate AI as a means to avoid social instability, censor information, reign in AI-run businesses, and retain political control.
China has developed a track record of supporting developing countries, including with foreign direct investment through the Silk Road initiative.[16] Indeed, African nations are already benefiting from Chinese models, and Malaysia is considering using Huawei chips in a state-backed project. To focus on AI in the global south, China established the World AI Cooperation Organization, its answer to the U.S.-led Pax Silica initiative. 10 of the 30 initial signatories to the WAICO agreement are African nations.
Xi's recent statements on AI emphasize diffusion and multilateralism.
Excerpt from Xi's speech at WAIC
Third, we should encourage inclusiveness and promote mutual learning between civilizations. AI development and its application should not erode or undermine the diversity of world civilizations or the uniqueness of cultures of different countries. We must shape the values of AI with humanity's common values and make good use of AI technologies to increase understanding, tolerance, exchanges, and sharing among all civilizations. We should tend to the garden of civilizations with great care to ensure that the beauty of each civilization is appreciated and shared.
Fourth, we should advocate solidarity and improve global governance. AI is an invaluable asset that encapsulates humanity's collective wisdom. We should practice true multilateralism and recognize the important role of the United Nations. We should enhance alignment and coordination on AI development strategies, governance rules and technical standards so as to form a consensus based global governance framework at an early date to make this frontier technology better benefit humanity. We must carry out extensive international cooperation and help global south countries with capacity building to bridge the AI and digital divides, promote sustainable development and prevent creating new historical injustice in AI.
Still, a bipolar arrangement in which the U.S. and China use their ASIs to check back against each other seems better for the rest of the world. Competition between the two powers could lead them to extend chips and models to poor countries. Plus, a bipolar world would enter the long reflection with more pluralistic values.
Great-Power Conflict
When relations between the states are not completely hostile, conflict will be increased. The aggrieved side will not only note the injury done but will assume that this was the main goal the other side was seeking and, projecting this motivation into the future, will foresee greater harm unless it reacts strongly.
— Robert Jervis, Perception and Misperception in International Politics
A third class of argument that comes up in conversations about AI and China is that the AI race is going to trigger great power conflict, regardless of whether AI itself is weaponized as part of that conflict. According to this story, the mere perception that the U.S. is nearing a decisive strategic advantage would lead China to preemptively strike the U.S. to prevent it from obtaining such a super weapon.[17]
In particular, ASI could undermine the nuclear deterrence regime by disabling second strike capability. ASI could locate China's nuclear submarines, in part through superhuman SIGINT, enabling the U.S. to incapacitate China's second strike capability in an initial wave of attacks. Without any remaining deterrence, China would be forced to make huge concessions to the U.S. It would be in China's interest then, to preemptively nuke the U.S. before the window of opportunity closes.
Plus, the chaos and uncertainty around each country's progress toward ASI could contribute to nuclear miscalculation (which is something better diplomacy would help mitigate...).
These stories are usually presented as general reasons the U.S. needs to worry about China in an AI context, rather than reasons the U.S. needs to beat China to ASI, but I address them here anyway.
These stories doesn't make sense to me:
Surely China would rather bite the possibility of living under U.S. rule than ensure a full-scale nuclear war.
If China were going to strike preemptively, it would do so with kinetic strikes against U.S. data centers, à la MAIM, not a nuclear attack.[18]
Even if China was willing to destroy itself, it is more likely to feel like it has nothing left to lose if the U.S. is way ahead in the race, so I don't see this as an argument for export controls. On the other hand, if China thinks it has some chance of winning, preemptive strikes look less attractive.
An adjacent point that comes up sometimes is that China is going to invade Taiwan, capture TSMC, and get all the compute. So, the U.S. needs to prepare to fight China over AI.
Even if China invades Taiwan, and even if the invasion is successful, China is unlikely to be able to take control of TSMC. So a Taiwan invasion would actually lengthen timelines.
Much TSMC equipment will get destroyed during an invasion, either inadvertently or by intentional sabotage.
Even if the equipment survives, without TSMC employees, China will struggle to operate the equipment.
Even if the equipment survives and China can operate it, the resulting chips would go to China! Which, if you buy some of the earlier arguments, is a lot better than them going to the U.S.
Conclusion
A thing I’ve been trying to fight for is export controls on chips to China. That’s in the national security interest of the US. That’s squarely within the policy beliefs of almost everyone in Congress of both parties. The case is very clear. The counterarguments against it, I’ll politely call them fishy. Yet it doesn’t happen and we sell the chips because there’s so much money riding on it. That money wants to be made.
Right now, export controls are seen as a prudent measure, while diplomacy toward a bilateral treaty is closer to wishful thinking.[19] These priorities are backwards. I hope I've been able to show that at the very least there are legitimate critiques of export controls that go beyond the obviously suspect arguments from Jensen Huang.
There are a couple recent events that I find encouraging. One is that total research transparency is in vogue now because it would underpin a bilateral treaty—despite the obvious infohazard and diffusion concerns that transparency raises. Another encouraging sign is that last year, the U.S. used the promise of selling Nvidia chips to Armenia to help broker a peace deal with Azerbaijan, suggesting that the U.S. is willing to use chip diplomacy when the terms are right.
To be sure, I don't know that the upcoming Trump-Xi summit is the optimal time to spend the export controls bargaining chip. For one thing, China could be more bought-into the risks and receptive to a treaty 6 months from now, when Chinese AI labs have had their own equivalents to the Hugging Face incident. It could also be that export controls are a better bargaining chip in the future if China becomes further convinced of the strategic importance of chips, raising the value it places on them.
But on average, the value of the bargaining chip declines as China sees the U.S. approach the RSI point of no return. If the U.S. reaches ASI with export controls still in place, I will likely think we made a mistake.
When I imagine China catching up to the U.S., I do feel some greater fear in response: it makes me feel less in control of the AI situation. But this feeling does not survive scrutiny. To the extent I feel like our lead gives us breathing room to figure out alignment, that's because I'm already being frog-boiled by the current trajectory of compute growth in the U.S.
Consider the situation from the perspective of an international social planner allocating chips between two countries in a suicide race to ASI when one country has a definitive lead. Wouldn't it be weird to give the chips to the leading country? There are some conceivable reasons you might reach such a counterintuitive conclusion, including if a wider gap makes a treaty more likely or the leader is sure to pause on the finish line, but as I've covered above, I don't think those reasons apply here.
Upon reflection, I know that I am no safer just because I happen to live in the country that's in the lead. Regardless of what we choose to do on the export controls front, we should get used to the feeling of not being as far ahead if we're serious about pacing.
Acknowledgements
Why can't we be friends?
— War
Several people have contributed to my thinking on this topic over the past few years. Thank you to Daniel King, Peter Barnett, MF, Caleb Parikh, and NJ. None of them endorse the conclusions in this post—in fact, many of them probably disagree with most of it.
This is not the only theory of change. Another one that gets less attention is that export controls are instrumentally good because enforcing them requires location monitoring. For example, the Chip Security Act requires location verification for enforcement. Tracking chips is good for safety, the argument goes, plus it sets the stage for the additional verification mechanisms that would facilitate an eventual treaty. (Other people argue for the other direction: that verification mechanisms are good because they help enforce export controls, which is not the position I'm addressing here.)
I have a couple responses:
This theory of change also seems kind of galaxy-brained. If you want location monitoring and other verification mechanisms, just advocate for those directly.
Export controls create a black market, which undermines verification. (Presumably it's possible to remove location monitoring from chips to sell on the black market?) I guess you could counter that net-more chips would be hidden without the export controls, which could be, but the black market still undermines verification.
Enforcement for a treaty will still be feasible even if some chips are untracked because it's really hard for China to hide an entire data center capable of frontier-scale training runs.
This is path-dependent on getting the treaty in the first place, which export controls could make more difficult.
I'm assuming that OpenAI and Anthropic will buy some of the export controlled chips, in line with Dylan Patel's forecast that those two companies will have most of the world’s compute by 2028.
As we'll cover later, some of these chips might have been spent running more or better automated safety researchers during the intelligence explosion, but I wouldn't expect that to be worth the faster speed.
Consider the way Dario talks about compute allocation on Dwarkesh's podcast. The way he makes it sound, out of Anthropic's compute in a given year, inference demand comes first, and the remainder goes to research.
If you get more demand than you thought, then research gets squeezed, but you’re kind of able to support more inference and you’re more profitable.
Maybe I’m not explaining it well, but the thing I’m trying to say is that you decide the amount of compute first. Then you have some target desire of inference versus training, but that gets determined by demand. It doesn’t get determined by you.
Probably external customers don't lexically take precedence over internal uses (if demand would consume 100% of compute, Anthropic would surely raise prices and preserve some for research). But as long as training compute isn't literally fixed, each marginal chip affected by export controls has some influence on training progress.
People who support export controls also support patching these loopholes, of course. Some work on that is already underway, including restoring the diffusion rule to close the remote access loophole. But it could end up being difficult to close these loopholes entirely.
Proponents of the backfire view might challenge why China has been able to keep up, despite existing export controls. Lennart Heim's answer is that China may have kept up relatively well so far, but worse than the counterfactual without export controls, and that gap will only widen. Why?
They haven't been in place for that long in the scheme of things.
They change the flow of chips, not the stock, and the stock is what matters for training. So it takes time for them to have a big effect.
They interrupt the data flywheel: less compute means fewer deployments (and therefore less data and deployment experience), less synthetic data generation, fewer experiments that researchers can run. And fewer chips potentially chokes RSI.
Isn't this a problem for all treaties, including the bilateral U.S.-China treaty I'm advocating for? To some extent, yes, but I expect the international agreement to have stronger oversight mechanisms. It will be the full weight of the U.S. government, instead of a team from Accenture and a handful of people from METR...
Though I would expect the U.S. lead to get longer if the labs in both countries get nationalized because the difference between total compute in the U.S. vs China is much larger than the difference between compute owned by e.g. OpenAI vs DeepSeek.
Regardless of whether a longer lead makes the U.S. more inclined to negotiate, that effect may be dominated by increasing political polarization of AI safety in the U.S. making it more difficult to get U.S. buy-in. That is, with Trump calling AI safety a hoax, now might be the most open the U.S. will ever be to a treaty. If so, it's better to remove export controls now in exchange for a deal.
Although, I guess it's possible that some people in AI safety have exaggerated their hawkishness to gain backing from the national security community for export controls.
Access to training compute matters as well because each ASI will be training (and aligning) its successors, but I'm simplifying to treat each ASI as one entity.
I have found Dario's answers to this question uncompelling. Dwarkesh asked him:
Why shouldn’t the US and China both have a “country of geniuses in a data center”?
Dario replied:
If this does happen, we could have a few situations. If we have an offense-dominant situation, we could have a situation like nuclear weapons, but more dangerous. Either side could easily destroy everything.
We could also have a world where it’s unstable. The nuclear equilibrium is stable because it’s deterrence. But let’s say there was uncertainty about, if the two AIs fought, which AI would win? That could create instability. You often have conflict when the two sides have a different assessment of their likelihood of winning. If one side is like, “Oh yeah, there’s a 90% chance I’ll win,” and the other side thinks the same, then a fight is much more likely. They can’t both be right, but they can both think that.
I agree this is a live possibility and deserves consideration, but Dario seems way overconfident to me about what the equilibrium will look like here. It could very well be more stable than the nuclear equilibrium. We should be pretty confident in his view for this consideration to outweigh the cost of delayed diffusion in developing countries. Dwarkesh rightly counters:
But this seems like a fully general argument against the diffusion of AI technology.
Dario's response is that he supports diffusion eventually, but only once the U.S. and other western liberal democracies have cemented their preferred global rules:
Let me just go on, because I think we will get diffusion eventually. . . . We need to find a way for people everywhere to benefit. My worry here is about governments. My worry is if the world gets carved up into two pieces, one of those two pieces could be authoritarian or totalitarian in a way that’s very difficult to displace.
Now, will governments eventually get powerful AI, and is there a risk of authoritarianism? Yes. Will governments eventually get powerful AI, and is there a risk of bad equilibria? Yes, I think both things. But the initial conditions matter. At some point, we’re going to need to set up the rules of the road.
. . .
What I would like is for the democratic nations of the world—those whose governments represent closer to pro-human values—are holding the stronger hand and have more leverage when the rules of the road are set. So I’m very concerned about that initial condition.
I'm just not convinced that the U.S. will ever allow diffusion in the sense that Dario is describing. At every future moment such a deal is considered, I expect the U.S. to deem the risks of diffusion too high to allow for meaningful technology-sharing. The U.S. already imposed export controls on Fable over trivial jailbreak concerns. That was the least fearful of AI proliferation I expect the U.S. to ever be.
If the U.S. starts training frontier-quality open source models in the next few years, I don't expect the U.S. to share those either:
The dual use capabilities will be greater for open source models.
China is already considering such restrictions on its own models, which is an indication of how the U.S. would handle the situation.
Incidentally, the EAR authorities that Commerce cited to export control Fable actually don't legally govern API services (you need section 744.6 for that), but those authorities do apply to exporting model weights!
In Plan A, the U.S. government sets aside $13 trillion for citizens and $5 trillion for the rest of the world in 2032. That comes out to $45,000 per person in the U.S. and $1,200 per person elsewhere (excluding China, which has its own AI windfall in Plan A).
That seems way overoptimistic to me. They predict that U.S. GDP will be around $50 trillion in 2032, meaning 10% of GDP is given abroad. That's dramatically out of line with the 0.25% of GNP the U.S. spent on foreign aid in 2023.
(In Plan A, by 2035, those welfare numbers increase to $300 trillion for citizens and $40 trillion for the rest of the world. That means other countries go from getting 38% as much as Americans to 13% as much, which seems like the wrong direction for that trend. As the U.S. gets richer, I expect it to get more generous in relative terms—though not generous enough for it to be worth racing China to become global hegemon.)
Such a preemptive strike would prevent the U.S. from using its strategic position to extract concessions from China. In Mandarin, the word for deterrence is weishe, which in addition to suggesting the Western meaning of dissuading an adversary from attacking, also means compelling an adversary to submit. China analyst Dean Cheng writes:
In essence, the available literature suggests that the Chinese do not necessarily think in terms of deterrence, as that term is employed in Western strategic literature, but in terms of coercion.
I've overall avoided discussion of MAIM until this point because if MAIM is true then most of this post is moot. MAIM claims that the strategic situation between the U.S. and China is currently one of deterrence. If either country gets to close to ASI, the other will strike to disable the other's AI project. The way to square this post with MAIM is:
To think about my arguments as addressing the worlds in which MAIM fails, or
To treat bombing the rival's data centers as each party's outside option in treaty negotiations. Then export controls still have a bearing on the likelihood and terms of a treaty, but MAIM constrains the set of possible agreements.
Washington should continue to try to slow down Chinese AI development to extend the US lead. Moonshot almost certainly trained its latest model using American chips, and probably relied — at least in part — on distilling American models. Measures like those set to be included in the Senate version of the NDAA, which will crack down on distillation and properly enforce chip export controls, would stymie China’s development further. The ultimate goal, however, should be a bilateral agreement.
Chinese models are misaligned in the sense that they are trained for Chinese values
Nitpick: if KimiK3 calls itself Claude then I wouldn't expect it to be aligned with Chinese values. How could one assess whether Kimi calling itself Claude makes Kimi's values Americanized or Chinese models routinely disbelieving their own words when it comes to politics is evidence of misalignment with any human values?
Introduction
Earlier this week, representatives for the U.S. and China discussed a notification mechanism to increase transparency on national security incidents involving AI, but export controls were not on the agenda for those talks. I think that was a mistake. A bilateral treaty is entering the Overton window, and sending more chips to China is a price worth paying to secure such an agreement. When Trump and Xi meet at the White House to discuss AI tomorrow, removing export controls should be on the negotiating table.
TLDR: Export controls are overrated. For export controls to earn their spot as a top policy priority, a conjunction of strong premises must hold. But there are reasons to doubt each of those premises, as well as reasons that export controls could be net negative, in particular by hastening RSI in the U.S. and making diplomacy more difficult. Instead, we should prioritize other policies. The case for a bilateral treaty, for one, or at least diplomacy toward such an agreement, is more robust, resting on weaker assumptions and posing less downside risk.
Questioning the Case for Export Controls
Suppose you are concerned with risks from misalignment (rather than hegemony or great power conflict, which I address later). What's the main theory of change for export controls?[1] We could break it down like this:
This is a textbook galaxy brained plan! The primary effect of export controls is to increase the concentration of chips in the leading country, accelerating frontier AI development. To think that other effects can outweigh that harm, we have to be really confident in our story for why. But there are reasons to doubt each of these premises, which together trouble the overall case for export controls.
I handle broader objections to (1) in the section on China threat models.
Do Export Controls Slow Down China? (2)
If Training Progress Is Exogenous
Let me start with a possibility that would largely foreclose the case for export controls: if the rate of training progress in China does not depend on the marginal chips that export controls influence at all; that is, if DeepSeek is going to train V5 according to the same schedule, regardless of whether it has access to the chips that would allow it to serve more instances. This doesn't seem likely, but it bears mentioning because it would significantly mitigate the solvency of export controls.
Now, if this is true in China, you could counter that it would certainly be true in the U.S. In other words, whatever OpenAI data center is going to be the one to train the first RSI-level model is going to have enough chips whether or not the U.S. lets some chips go to China.
But that's still an argument against export controls! During RSI, the leading U.S. lab can allocate at most 100% of their chips to fueling the intelligence explosion. So in this world, where training progress is exogenous, the marginal chips just determine the maximum speed of RSI—and the eventual diffusion of ASI.[2] On both counts then, I'd rather see those marginal chips go to China. It's unlikely that those chips will mean the difference between a deadly and a safe intelligence explosion in the U.S., but it would be directionally helpful.[3]
Here's a cheekier and much more inflammatory way to express that sentiment: "So you want to pace the frontier? Send more chips to China."
If Training Progress Is Endogenous
But let's set this consideration aside, and instead consider the more ambiguous case, in which the marginal chips do affect training progress. This case also seems much more likely.[4]
In this case, China's response could mitigate the overall slowdown. It could be that:
One argument I'm not making: I think some people argue that export controls have actually backfired by speeding China up on net due to compensation from China's domestic chip industry, but this still seems very unlikely to me.[6] The strongest evidence is that China itself has banned Nvidia chips, revealing that it may have some faith in import substitution.
Do Export Controls Speed Up The U.S.? (3)
Yes! And that's bad!
Will the U.S. Pace or Pause on the Finish Line? (4)
The recent efforts toward pacing the frontier have been really encouraging. The possibility that the U.S. labs will coordinate to pace seems much more likely than it did for the past few years, when it sounded more like a pipe dream. But pacing (and certainly a pause) that meaningfully slows the U.S. still looks unlikely to me. Not only is there no brake, no one knows what a brake would look like.
Embedded evaluators might lack the access and technical sophistication to uncover risks from AIs or false statements from the companies. Even if the evaluators raise an alarm, the public might not respond. (Consider, for example, that METR assessed the automated R&D section of Anthropic's risk report and wrote, "We do not think the report adequately supports its conclusion." Anthropic's leaders weren't dragged in front of Congress then, I don't think they would be now.)
Enforcement concerns aside, the recent pacing statements from lab leaders are a long way from a fully specified agreement. Without a specific commitment, the evaluators have nothing to audit, no matter how much access they get.
But suppose we do wind up in a regime with specific commitments (e.g., something as concrete as "FLOPs used on training and internal deployments cannot exceed X threshold this year"), and evaluators are empowered to audit the company against those commitments. There will still be enormous pressure for each company to defect on the precipice of RSI.[7]
Will further warning shots galvanize the political will for a real pause? I do expect more warning shots, but I think they're more likely to cause the U.S. government to nationalize the labs. That does obviate the coordination problem, but I think it replaces it with something worse: consolidating all the labs' compute under one project, false assurance that the government has got this, and reduced public transparency compared to what embedded evaluators previously provided. At that point, I don't expect the U.S. government to cede any of its lead to China.[8]
If pacing fails, with or without nationalization, the main effect of export controls is to increase the maximum speed of RSI in the U.S.
Okay, but suppose pacing proposals have some solvency. The argument in favor of export controls is that the U.S. will pace until it has spent its lead: if the U.S. has more chips, it can afford to pace for longer.
The problem with that is the companies in the U.S. will have different tolerances for how much of their lead they're willing to burn, and the measures of that lead are all ambiguous.
Consider the status quo: some benchmarks suggest that China is nearly a year behind, while others make it look like Chinese labs are nipping at the heels of their U.S. competitors. Though Dario has been the most supportive of pacing among U.S. lab CEOs, he is also the most paranoid about China. I can't really imagine him being comfortable pausing for more than a couple months (or the equivalent, spread out over the next few years).
Will the U.S. Spend Its Lead on Alignment? (5)
Say the U.S. companies agree on a measure of their lead and how much of it to spend. That buys more time for alignment research, maybe up to an additional 6 months or a year of wall clock time (and more when you weight that time by the number and quality of automated alignment researchers that become available during that period).
Some researchers might have to turn from their alignment agendas to work on the technical foundations of the pause, but that still seems really valuable.
And, ideally, export controls mean alignment researchers also have more compute than they would have otherwise during the entire period to ASI, not just during the marginal months from pacing.
I have a few counter-arguments to this story:
But I don't weigh these too heavily; overall I agree the U.S. will spend at least some of the lead on alignment research, and that will lead to net more alignment research. (Again, that's conditional on pacing. In the absence of a pacing agreement, more chips means faster training progress, so net less alignment research).
Will That Research Make the Intelligence Explosion More Likely to Go Well? (6)
I don't think so. Even if everything goes right with the research—a full year of extra time, every export controlled chip goes to safety, no unintended capabilities progress, no diffusion—I still don't think we will have solved alignment. The better hope for that is a bilateral treaty with China that can last a lot longer than a year.
Export Controls Might Make Diplomacy Harder
It could be that China would be willing to come to the negotiating table for a bilateral treaty, but export controls make diplomacy more difficult by:
To the extent it's a bargaining chip then, better to spend it sooner, to take down the temperature and reduce the pressure for such "fearmongering" dismissals to seep into the discourse in China.[9]
The threat of re-imposing export controls would still be a point of leverage in future discussions, but it's better to remove them now than to leave them in place, which has the added disadvantage of allowing China to adjust to the controls, including by more heavily prioritizing domestic chip manufacturing.
You could counter that China will have a stronger incentive to negotiate, the further behind it is. This is the position that Dario takes in his pacing essay:
I'm like, maybe? All he offers is an assertion. I think it's true that a stronger U.S. lead at the time a treaty is signed will result in better terms for the U.S., but as I discuss further down, I'm not convinced that locking in stronger U.S. leadership will be good for the world. More importantly, the terms of the treaty are secondary to the treaty getting signed in the first place. A couple reasons I think Dario misses the mark in his assertion that a wider gap makes an agreement more likely:
Overall, this question about direction of the effect of a wider gap on the likelihood of a deal is a huge crux about export controls, and I'm not willing to take Dario's word for it. As long as we have significant uncertainty about this question, I don't think we can prioritize export controls over more commonsense policies.
Summing Up
These considerations mitigate the case for export controls significantly, and suggest that export controls could even be net-negative in terms of misalignment risk, mostly by speeding up capabilities progress in the U.S., and perhaps by making diplomacy harder.
I don't think any of these arguments is a slam dunk. It could still be that the 6-point story for export controls is true. But I think they muddy the waters enough that export controls can not be held up as an obviously high-priority policy.
Next, I address the first premise, that the U.S. should beat China to ASI. In principle, this isn't a necessary part of the theory of change for export controls. As in, you could conceivably think export controls reduce misalignment risks while being indifferent about which country is first to ASI. But in practice, I think most people who advocate for export controls present them as a means to beat China.[11] Inversely, if you don't buy one of the following three stories for why the U.S. should try to "win" the "race," you probably never reach the point of evaluating export controls as a tactic.
Against China Threat Models
What are the justifications for anti-China concerns? The main stories focus on misalignment, hegemony, or great-power conflict. For each of them, it's not clear why the U.S. beating China to ASI makes the world safer. Proceed by cases:
Misalignment
If the World Is Unipolar
First, suppose that ASI is winner-takes-all, in the sense that once one ASI has been created, it will take steps to prevent the creation of competing models. Here, we just compare P(aligned | U.S.) against P(aligned | China). It's not obvious to me why the U.S. would be better? For sure, the U.S. has done more alignment research, but U.S. models still have glaring alignment issues. See, for example, the Hugging Face incident, or testing of recent models, such as METR's evaluation of GPT-5.6 Sol.
Maybe people would object that Chinese models are misaligned in the sense that they are trained for Chinese values? But first, U.S. models have their own Western values, and it's not obvious which bias is worse, and second, it's not clear that these biases really count as misalignment in a sense that jeopardizes the value of the longterm future.
The stronger version of this objection is that China will have worse scalable oversight research during RSI, making misalignment more likely, and weaker control protocols, making it easier for a misaligned model to sabotage R&D. First, notice that these are also arguments for sharing scalable oversight research and control protocols with Chinese labs! (Maybe if China took the lead, that's more likely to happen...)
Second, I don't expect the U.S. to do a much better job of automated alignment research or control than China: automated alignment research seems fundamentally really hard and our control protocols aren't ready for ASI-level models.
If the World Is Multipolar
Suppose that ASI is not winner-takes-all; instead, both countries (and then all countries, and then all individuals) will be able to train an ASI. Why might that be? The base case is that the capability to spark an intelligence explosion is a fixed-capability target, and there's nothing a U.S. ASI could do to stop Chinese research from eventually crossing that threshold. Indeed, it might be even easier to create competing ASIs once the first one has been created because of espionage (including stealing the weights outright), blackmail, and distillation.
Here, what matters is whether the ASIs are aligned and whether an aligned ASI can out-compete a misaligned ASI.
Will the ASIs be aligned? It seems to me that the further behind China perceives itself to be, the more pressure there will be to cut corners on safety, making it more likely we end up in a world with at least one misaligned ASI.
If the U.S. is far ahead, does that at least make it more likely that our glorious freedom-loving ASI can pummel their wicked socialist ASI? Probably: a head start can only help, and our ASI will have more access to inference compute (with or without export controls).[12] Some reasons to think this would not hold are:
But these are not dispositive. Overall, I agree that a head start would help an aligned U.S. ASI beat a misaligned Chinese ASI. But for that head start to be worth it, you have to think the benefit outweighs the harm of the greater likelihood that both models are misaligned. That's not at all intuitive to me. At best it's ambiguous.
Here's a simple model
Suppose America chooses whether to race. If it races, its ASI will gain a decisive advantage over China's.
(In this model, it doesn't matter how much marginal misalignment probability China takes on by racing. A better model would include uncertainty over whether the U.S. will cross the threshold for a decisive advantage, and then China's marginal risk should factor in.)
If
Graphically, the maximum amount of additional misalignment we should be willing to pay
A few implications of this model:
Note that diplomacy aimed at a coordinated pause makes it more likely we end up with two aligned ASIs.
Couldn't the U.S. just pre-commit to giving China (and eventually every other country) a copy of its aligned ASI to limit the number of intelligence explosions to 1? I think that kind of arrangement has been under-explored in AI policy so far. If the outside option is the U.S. races to a misaligned ASI, maybe provoking World War III in the process, there should be a lot of room to negotiate an ASI sharing deal that benefits both sides.
Hegemony
Alternatively, let's assume automated alignment will pan out during the intelligence explosion. As a result, a country can weaponize its ASI, or at least direct it as an instrument of hegemony.
Here, the story people tell for beating China is that if China gets to ASI first, it would abuse its ASI to spread global totalitarianism, but if the U.S. gets there first, it would distribute the benefits around the world.
What are the channels for global influence?
I buy that these channels would contribute to hegemony, but I don't think it follows that the U.S. must seize them before China.
I distrust the U.S. to responsibly exercise global control about as much as I distrust China. This is the branch of this whole debate that I find most difficult to argue with people because the crux often comes down to people's sense of whether the U.S. has a good or bad historical track record. But to sketch a brief case for bad track record: the U.S. has a long (and recent) history of imperialism, wars, massacres, coups, resource extraction, and other foreign interventions. A lot of the world reads that history as evidence the U.S. is a bad-faith actor. The only time in history that one country had nuclear weapons, it was the U.S., and the U.S. used them to kill over 100,000 civilians. Dario doesn't know if Claude was responsible for the Minab school attack earlier this year.
Setting aside this history and how to interpret it, I doubt that the U.S. would share much of the economic windfall from ASI with the rest of the world. In order of decreasing self-determination for the recipient countries:
Will the leading AI lab itself pursue redistribution? No:
One cause for hope is that the U.S. has programs to share civil nuclear energy technology with other countries, including by supplying reactors. But:
So the nuclear precedent doesn't give me that much confidence that the U.S. will tolerate global diffusion in AI.
If we're going to have a unipolar world, China as hegemon seems more likely to distribute the AI windfall:
Excerpt from Xi's speech at WAIC
Third, we should encourage inclusiveness and promote mutual learning between civilizations. AI development and its application should not erode or undermine the diversity of world civilizations or the uniqueness of cultures of different countries. We must shape the values of AI with humanity's common values and make good use of AI technologies to increase understanding, tolerance, exchanges, and sharing among all civilizations. We should tend to the garden of civilizations with great care to ensure that the beauty of each civilization is appreciated and shared.
Fourth, we should advocate solidarity and improve global governance. AI is an invaluable asset that encapsulates humanity's collective wisdom. We should practice true multilateralism and recognize the important role of the United Nations. We should enhance alignment and coordination on AI development strategies, governance rules and technical standards so as to form a consensus based global governance framework at an early date to make this frontier technology better benefit humanity. We must carry out extensive international cooperation and help global south countries with capacity building to bridge the AI and digital divides, promote sustainable development and prevent creating new historical injustice in AI.
— Xi Jinping
Still, a bipolar arrangement in which the U.S. and China use their ASIs to check back against each other seems better for the rest of the world. Competition between the two powers could lead them to extend chips and models to poor countries. Plus, a bipolar world would enter the long reflection with more pluralistic values.
Great-Power Conflict
A third class of argument that comes up in conversations about AI and China is that the AI race is going to trigger great power conflict, regardless of whether AI itself is weaponized as part of that conflict. According to this story, the mere perception that the U.S. is nearing a decisive strategic advantage would lead China to preemptively strike the U.S. to prevent it from obtaining such a super weapon.[17]
In particular, ASI could undermine the nuclear deterrence regime by disabling second strike capability. ASI could locate China's nuclear submarines, in part through superhuman SIGINT, enabling the U.S. to incapacitate China's second strike capability in an initial wave of attacks. Without any remaining deterrence, China would be forced to make huge concessions to the U.S. It would be in China's interest then, to preemptively nuke the U.S. before the window of opportunity closes.
Plus, the chaos and uncertainty around each country's progress toward ASI could contribute to nuclear miscalculation (which is something better diplomacy would help mitigate...).
These stories are usually presented as general reasons the U.S. needs to worry about China in an AI context, rather than reasons the U.S. needs to beat China to ASI, but I address them here anyway.
These stories doesn't make sense to me:
An adjacent point that comes up sometimes is that China is going to invade Taiwan, capture TSMC, and get all the compute. So, the U.S. needs to prepare to fight China over AI.
Even if China invades Taiwan, and even if the invasion is successful, China is unlikely to be able to take control of TSMC. So a Taiwan invasion would actually lengthen timelines.
Conclusion
Right now, export controls are seen as a prudent measure, while diplomacy toward a bilateral treaty is closer to wishful thinking.[19] These priorities are backwards. I hope I've been able to show that at the very least there are legitimate critiques of export controls that go beyond the obviously suspect arguments from Jensen Huang.
There are a couple recent events that I find encouraging. One is that total research transparency is in vogue now because it would underpin a bilateral treaty—despite the obvious infohazard and diffusion concerns that transparency raises. Another encouraging sign is that last year, the U.S. used the promise of selling Nvidia chips to Armenia to help broker a peace deal with Azerbaijan, suggesting that the U.S. is willing to use chip diplomacy when the terms are right.
To be sure, I don't know that the upcoming Trump-Xi summit is the optimal time to spend the export controls bargaining chip. For one thing, China could be more bought-into the risks and receptive to a treaty 6 months from now, when Chinese AI labs have had their own equivalents to the Hugging Face incident. It could also be that export controls are a better bargaining chip in the future if China becomes further convinced of the strategic importance of chips, raising the value it places on them.
But on average, the value of the bargaining chip declines as China sees the U.S. approach the RSI point of no return. If the U.S. reaches ASI with export controls still in place, I will likely think we made a mistake.
When I imagine China catching up to the U.S., I do feel some greater fear in response: it makes me feel less in control of the AI situation. But this feeling does not survive scrutiny. To the extent I feel like our lead gives us breathing room to figure out alignment, that's because I'm already being frog-boiled by the current trajectory of compute growth in the U.S.
Consider the situation from the perspective of an international social planner allocating chips between two countries in a suicide race to ASI when one country has a definitive lead. Wouldn't it be weird to give the chips to the leading country? There are some conceivable reasons you might reach such a counterintuitive conclusion, including if a wider gap makes a treaty more likely or the leader is sure to pause on the finish line, but as I've covered above, I don't think those reasons apply here.
Upon reflection, I know that I am no safer just because I happen to live in the country that's in the lead. Regardless of what we choose to do on the export controls front, we should get used to the feeling of not being as far ahead if we're serious about pacing.
Acknowledgements
Several people have contributed to my thinking on this topic over the past few years. Thank you to Daniel King, Peter Barnett, MF, Caleb Parikh, and NJ. None of them endorse the conclusions in this post—in fact, many of them probably disagree with most of it.
This is not the only theory of change. Another one that gets less attention is that export controls are instrumentally good because enforcing them requires location monitoring. For example, the Chip Security Act requires location verification for enforcement. Tracking chips is good for safety, the argument goes, plus it sets the stage for the additional verification mechanisms that would facilitate an eventual treaty. (Other people argue for the other direction: that verification mechanisms are good because they help enforce export controls, which is not the position I'm addressing here.)
I have a couple responses:
I'm assuming that OpenAI and Anthropic will buy some of the export controlled chips, in line with Dylan Patel's forecast that those two companies will have most of the world’s compute by 2028.
As we'll cover later, some of these chips might have been spent running more or better automated safety researchers during the intelligence explosion, but I wouldn't expect that to be worth the faster speed.
Consider the way Dario talks about compute allocation on Dwarkesh's podcast. The way he makes it sound, out of Anthropic's compute in a given year, inference demand comes first, and the remainder goes to research.
Probably external customers don't lexically take precedence over internal uses (if demand would consume 100% of compute, Anthropic would surely raise prices and preserve some for research). But as long as training compute isn't literally fixed, each marginal chip affected by export controls has some influence on training progress.
People who support export controls also support patching these loopholes, of course. Some work on that is already underway, including restoring the diffusion rule to close the remote access loophole. But it could end up being difficult to close these loopholes entirely.
Proponents of the backfire view might challenge why China has been able to keep up, despite existing export controls. Lennart Heim's answer is that China may have kept up relatively well so far, but worse than the counterfactual without export controls, and that gap will only widen. Why?
Isn't this a problem for all treaties, including the bilateral U.S.-China treaty I'm advocating for? To some extent, yes, but I expect the international agreement to have stronger oversight mechanisms. It will be the full weight of the U.S. government, instead of a team from Accenture and a handful of people from METR...
Though I would expect the U.S. lead to get longer if the labs in both countries get nationalized because the difference between total compute in the U.S. vs China is much larger than the difference between compute owned by e.g. OpenAI vs DeepSeek.
Bernie's data center moratorium bill is an example of treating export controls as a bargaining chip to incentivize global coordination.
Regardless of whether a longer lead makes the U.S. more inclined to negotiate, that effect may be dominated by increasing political polarization of AI safety in the U.S. making it more difficult to get U.S. buy-in. That is, with Trump calling AI safety a hoax, now might be the most open the U.S. will ever be to a treaty. If so, it's better to remove export controls now in exchange for a deal.
Although, I guess it's possible that some people in AI safety have exaggerated their hawkishness to gain backing from the national security community for export controls.
Access to training compute matters as well because each ASI will be training (and aligning) its successors, but I'm simplifying to treat each ASI as one entity.
I have found Dario's answers to this question uncompelling. Dwarkesh asked him:
Dario replied:
I agree this is a live possibility and deserves consideration, but Dario seems way overconfident to me about what the equilibrium will look like here. It could very well be more stable than the nuclear equilibrium. We should be pretty confident in his view for this consideration to outweigh the cost of delayed diffusion in developing countries. Dwarkesh rightly counters:
Dario's response is that he supports diffusion eventually, but only once the U.S. and other western liberal democracies have cemented their preferred global rules:
I'm just not convinced that the U.S. will ever allow diffusion in the sense that Dario is describing. At every future moment such a deal is considered, I expect the U.S. to deem the risks of diffusion too high to allow for meaningful technology-sharing. The U.S. already imposed export controls on Fable over trivial jailbreak concerns. That was the least fearful of AI proliferation I expect the U.S. to ever be.
If the U.S. starts training frontier-quality open source models in the next few years, I don't expect the U.S. to share those either:
Incidentally, the EAR authorities that Commerce cited to export control Fable actually don't legally govern API services (you need section 744.6 for that), but those authorities do apply to exporting model weights!
In Plan A, the U.S. government sets aside $13 trillion for citizens and $5 trillion for the rest of the world in 2032. That comes out to $45,000 per person in the U.S. and $1,200 per person elsewhere (excluding China, which has its own AI windfall in Plan A).
That seems way overoptimistic to me. They predict that U.S. GDP will be around $50 trillion in 2032, meaning 10% of GDP is given abroad. That's dramatically out of line with the 0.25% of GNP the U.S. spent on foreign aid in 2023.
(In Plan A, by 2035, those welfare numbers increase to $300 trillion for citizens and $40 trillion for the rest of the world. That means other countries go from getting 38% as much as Americans to 13% as much, which seems like the wrong direction for that trend. As the U.S. gets richer, I expect it to get more generous in relative terms—though not generous enough for it to be worth racing China to become global hegemon.)
Predatory lending
Such a preemptive strike would prevent the U.S. from using its strategic position to extract concessions from China. In Mandarin, the word for deterrence is weishe, which in addition to suggesting the Western meaning of dissuading an adversary from attacking, also means compelling an adversary to submit. China analyst Dean Cheng writes:
I've overall avoided discussion of MAIM until this point because if MAIM is true then most of this post is moot. MAIM claims that the strategic situation between the U.S. and China is currently one of deterrence. If either country gets to close to ASI, the other will strike to disable the other's AI project. The way to square this post with MAIM is:
For example, take Shakeel Hashim in Transformer: