To state the obvious:
Many of the same people who publicly make a great fuss of this don't seem to care that Nvidia is selling GPUs to China by way of Singapore. Their purpose is to drag up China as an excuse for continuing what they are doing. If China has more GPUs, all the better for them in the future; China will be a greater threat that they can use to continue what they are doing.
Their model hasn't chosen an explanation like "China will play the role of the movie bad guy at any cost to their own selfish interests". They aren't reasoning at GPT 3.5-level, only GPT-2 level. China is a scary word-vector, so they can argue that if the USA tries to slow down against extinction risk, then China won't slow down, because that's a scary thing so "China" is a word-vector that the scary thing can be attributed to.
1I think the point is that there aren't just two black and white possibilities:
- get extinct by developing superintelligence,
- stop AI R&D.
I would argue that it is much more difficult to understand at which point the AI is too dangerous to be improved than to understand at which point YOUR AI is "better enough" than YOUR ENEMY'S AI (China or the US) to provide you some strategic advantage. Under this premise, if your adversary is rational, once you stop your AI R&D, he may realize that his optimal choice is to do just a bit more R&D and only then stop, in order to get some strategic advantages out of it, and still avoid the catastrophic risk of extinction.
If both adversaries consider the other two possibilities:
- develop "just a bit more AI" to catch up with the other country,
- develop "just a bit more AI" to get some strategic advantage over the other country...
they can push the Nash equilibrium into "get extinct by developing superintelligence".
On the other hand, I'd say that in order to hope for some international cooperation you need to have some more nuance (maybe GPT-5.5 level) and model the fact that possibly both good and bad guys in their heart believe to be the good guys, don't want to see the world burn, and may be happy to take a leap of faith and stop/regulate AI R&D in the same way as we did for weapons/pollution: not in pursuit of their selfish interest, but because it is good for the world.
Suppose that alignment was TRIVIAL. Then it would make PERFECT sense for Nvidia (and for Nvidia-controlled politicians?) to sell the GPUs to China, since the result would merely accelerate both sides' progress and improve Nvidia's cash flow.
As for China failing to slow down, I would like @Daniel Kokotajlo to comment on whether China not slowing down in the tabletop exercises was due to the player roleplaying as China acting too in-character or due to quasirational considerations like the ones with which one could "justify" betrayal in the Prisoner's Dilemma.
One toy model is race dynamics, neither is incentivised to unilaterally slow down and they lack the mutual trust and coordination mechanisms to even seriously consider coordinating
The adversaries are assumed to believe in a severely flawed world model. Suppose that White House Will Ad Hoc Decide Who Can Individually Access GPT-5.6, but Musk, being a friend of Trump, informed Trump that Grok 5 is aligned because it underwent alignment training (which actually taught Grok to hide its thoughts, but Musk didn't understand it and Trump never learned the details). Then Trump would trust Musk and watch as Grok 5 gets released.
Basically this. Trump and the CCP neither believe nor alieve in the possibility of superintelligence; it does not inform their actions the same way that nuclear war and potential assassinations do. It is not clear what would have to happen for that to change.
ETA: Obviously that might change; many people have updated over the last few years. But I mostly don't understand what caused anyone to update any particular point in time in the past, and it gets more confusing as capabilities continue to increase why exactly the most recent thing was enough but the one before that wasn't.
There are plenty of movies with plots where the good guys are trying to prevent a disaster that will harm everyone, and the bad guys are too busy being bad guys to see the problem and ally with them so they instead foolishly get in the good guys' way. Under Hollywood logic, bad guys are automatically foolish.
There are also plenty of people who believe misuse is a serious problem, but not loss of control (and indeed some vice versa). The Administration's recent behavior suggests they suddenly started taking in the misuse problem seriously, not that they bought loss-of-control.
In practice, I'm not expert on CCP psychology, but I would guess they might be inherently rather predisposed to thinking in terms of loss of control problems, at least in certain respects.
This is a good time to be asking this question.
We've recently seen a massive shift from the White House. Honestly, it's still hard to imagine, but things can change fast.
It’s like while believing in the risk, they also kind of believe that it’s a totally uncompelling story that nobody in real geopolitics would ever be touched by.
If this is supposed to be a contradiction, I don't see any. "Human extinction/permanent disempowerment because of ASI is possible" and "almost nobody believes that human extinction/permanent disempowerment is possible, or that AI that is superintelligent in the relevant sense can even exist" can both be true at the same time, and right now they are. I doubt this will change in the absence of an extremely loud and clear warning shot on the level of Chernobyl. And even then it's not guaranteed that the response will be sane (after the recent ban on Mythos and opaque policy regarding which models will be available to whom, I've updated away from people responding to a warning shot in a reasonable manner).
Another hypothesis: geopolitical actors have limited bandwidth and their even considering your argument of stopping AI progress gives them less bandwidth to deal with <all the other sht they're fighting against> and so purely by virtue of this, they don't want to all come together and solve a problem that you think is important.
When I talk to people about what might be done about AI threatening approximately everything that everyone cares about, I notice a common oddity in their resistance to a variety of ideas. They seem to take for granted that certain entities—especially Trump and China—would be acting against their own interests, were they to cooperate or take proactive action to avert the building of dangerous AI.
The speaker often thinks there is a fairly substantial risk of the AI thus produced killing or disempowering everyone, including Trump and China. And I imagine in a situation where a certain course of action were going to produce a 20% chance of Trump being shot in the head or China being heavily nuked, that these parties would actually be considered to be ‘following incentives’ to avoid it. Yet they talk as though the idea of Trump or China responding to such risks is akin to the idea of these parties suddenly becoming zealous proponents of universal selfless love randomly.
It’s like while believing in the risk, they also kind of believe that it’s a totally uncompelling story that nobody in real geopolitics would ever be touched by.
Or that these parties actual utility functions are ‘be the bad guy’, which, you know, sometimes involves extreme selflessness as long as it puts everyone else at risk too.
Any other hypotheses for what’s going on?