I'm frankly confused here by your treatment of Trump here. The President of the United States actively mocks and denies one of the most pressing risks to humanity and the response is to shrug and ask people not to attack him? He is the leader of the free world and he is being held here to the standard of a late night talk show host.
The Trump Administration's initial response to another major national crisis, the COVID-19 pandemic, is instructive, and that history is not reassuring:
3.1. Threat insensitivity and policy overconfidence
Trump repeatedly claimed that the coronavirus crisis was unforeseeable: “… an unforeseen problem… What a problem. Came out of nowhere” (White House, 2020c). On another occasion, he said, “We're having to fix a problem that, 4 weeks ago, nobody ever thought would be a problem” (White House, 2020d). He then stated at one of his daily briefings, “I would view it as something that just surprised the whole world,” adding that it was “uncharted territory” (White House, 2020a). These claims were inaccurate and self‐serving. The threat of a pandemic was foreseeable and widely foreseen (Sanger et al., 2020).
Regarding the COVID‐19 pandemic, the early warning system worked, and, starting in January 2020, Trump repeatedly received prescient warnings from the intelligence community and officials, such as national security adviser Robert O'Brien, deputy national‐security adviser Matthew Pottinger, economic adviser Peter Navarro, and the head of HHS, Alex Azar, among others (Blake, 2020; Harris et al., 2020; Lipton et al., 2020; Woodward, 2021). The alarm had been loudly sounded, and the case for a robust policy response had been made (Woodward, 2021; Wright, 2021). Yet, although Trump eventually came to accept that COVID‐19 was a deadly and highly contagious disease, his staff described him as “slow to absorb the scale of the risk and to act accordingly,” and one of his closest associates said he was “baffled” by how the crisis had unfolded (Lipton et al., 2020).
Further on:
3.2. Denial and wishful thinking
Watching President Trump's response to the unfolding COVID‐19 pandemic was to observe a leader acting and communicating in a fashion congruent with psychological denial and wishful thinking (Ball et al., 2021). For months, the president continually attempted to reassure the public—and the stock market—via his statements at public appearances and tweets that the problem was “under control,” that it would soon go away, and that vaccines were just around the corner and could be pushed out to a receptive public at WARP speed—despite clear indications and expert assessments to the contrary. Again, however, analytical caution is needed, and behaviors attributable to motivated bias and misperception may conceal deception and conscious misrepresentation for personal or political benefit (Tetlock, 2000).
In other words, a critical question in interpreting and judging the president's crisis communication regarding the COVID‐19 pandemic is whether he genuinely believed that happy days would soon return and that the pandemic threat would soon recede due to the reduced travel from China (and subsequently from other countries too), seasonal weather changes, etc.—or whether he simply believed that it was in his own political and/or the general interest to act as if he did. The public record of his remarks (Wolfe & Dale, 2020), revelations from Bob Woodward's March 2020 interviews on Trump's private views about the danger of the virus (Woodward, 2021), and reporting based on interviews with administration and government officials involved in fighting the pandemic (Abutaleb et al., 2020), suggest a complex mix of politically motivated “tactical reassurance,” denial, and wishful thinking.
These are deadly traits in a national leader, and we're seeing the exact same script play out. Trump's perceived interests are threatened by a narrative he doesn't control, so he denies the narrative. He postpones rather than encourages actual work on disaster prevention and preparation for contingencies. He is overconfident in his own ability to shut down rogue AI that we may not even know is actively attacking us until it is too late.
His response and actions are a failure of leadership. Full stop.
What we get in this post re:Trump is essentially "it's okay guys, we know he's underinformed and may change his mind. Don't make him mad or it gets worse."
Like yeah, man. That's part of the problem.
"He likely has only been briefed by some combination of Bessent, Lutnick and Jensen."
Again, he is the President of the United States. He should be getting daily intelligence briefings and be one of the best-informed people on the planet. Instead he doesn't read them even when he does bother to get the briefing at all.
"I stand by my position that this could have meant any number of things. The way things ultimately went was far from determined, and jumping to the worst case conclusion only made that ending more likely."
So why is this not terrifying to you, that he is so easily swayed by emotions and attacks?
"You certainly cannot read this as ‘America will have zero AI guardrails’ when Trump has already instituted super secret AI guardrails and yanked the top AI model from the market for weeks on safety concerns. We are also now indefinitely in a world where the White House arbitrarily decides who gets access to Claude Mythos.
None of this is compatible with wanting no guardrails at all."
Yeah, he's famously had concepts of a plan before. His plan this time, as far as it exists and we have been allowed to know it, is mostly to bully companies into doing what he wants when he wants it. He's in a position now where he can make the rules up as the suit him, and he is clearly in no rush to get the guardrails into legislation where they have actual teeth.
"I know this is tragic, horrible and many other things. But.
If you can set those things aside it is also extremely funny."
Funny enough, I cannot set those things aside when the President of the United States is actively increasing the likelihood of human extinction.
There's a time and a place, but this isn't it.
"This is reality. This is Earth. This is 2026. We saw that the President of the United States talks this way and we all thought ‘yeah, we know, that’s just how it is.’"
I would add "and that is a major problem for all of us for obvious reasons."
"Even now, Trump is being misinterpreted. Of course Trump endorses the need for guardrails. Trump said on Sunday on Air Force One, to reporters, that ‘we can put guardrails.’ Trump is saying that there is no need to worry because he is such a great President that will ensure the right guardrails, not that we don’t need any at all."
Paying lip service to guardrails is not the same thing as actively working to get them enacted. Talk is cheap. The kinds of protections we need have to be set up well in advance, not done ad-hoc based on vibes.
To your point, I'm not here to make this partisan. I'm in full agreement that this specific issue isn't a matter of right vs left.
Mostly, I would very much like not to die, and he keeps making it more likely that I do.
I'll wrap here by saying that I do admire your reporting, even where I disagree with the politics. You've got a generally clear-eyed view of the situation on the ground and the comms savvy to make the work accessible. I appreciate the effort it takes to write as frequently and comprehensively as you do, and I hope that your future posts return to your usual standard of holding the people in power to account.
I don’t think that would be my top priority at OpenAI, given what else is happening, but it matters.
True, but plausibly it's demanding an action that's sufficiently fast, obvious, and meaningful-but-relatively-low-stakes so as to determine whether the party on the other side is worth engaging with at all.
This is our reality. I suppose we have to talk about it.
Everyone in a position to know is freaking out about AI potentially killing everyone this decade and wants to pace the frontier, and people are finally listening.
It only took a few days for the conversation to fully pivot to the counteroffensive, where the Usual Suspects and those they recruited attacked anyone and everyone who dared point out that we are in danger, with every attack they can think of, usually without substance or any attempt at understanding.
Sigh. I knew what I signed up for.
Table of Contents
Hold Your Fire
The most important thing, as this plays out, remains to not make things worse.
Please do not make this any more partisan or personal than it already is. Please do not attack Republicans, or Trump. That will only make things worse. Emphasize helpful voices on all sides.
That does not mean you do not point out that Wrong Post is Wrong, or that particular people are in bad faith or are lying in particular ways. Certainly you can call out the Usual Suspects like Jensen and Sacks who are actively driving us off a cliff.
What you do not do is go after Republicans as a group, or attack Trump personally in all the ways many of you will be sorely tempted to do.
For the same reason, to avoid reinforcing a partisan frame, I will refrain from including related statements from Democratic politicians.
If You Don’t Like the Weather
Before we get into details, I also emphasize: Nothing here is fixed.
Every month is different. Trump changes positions on a dime. Trump has gone back and forth several times on Anthropic and Dario Amodei within the last few months. If there is another major AI incident or the politics changes, or after the midterms, anything is possible.
And a lot of this can be posturing in advance of the US-China summit, to look tough and stake out a negotiating position, as could be China’s reactions to all this.
It does not look good. These statements could have somewhat crossed the Rubicon and will make it harder to double back. But that has not always stopped this man in the past, and there are clear signs that Trump does not understand the situation. He likely has only been briefed by some combination of Bessent, Lutnick and Jensen.
Trump Does Not Take Kindly
On Monday, as Trump made some rather concerning statements, I cautioned that the press was jumping to conclusions and misreading what Trump said. Yes, Trump was unhappy about the vibes from all this, but his beef was about the data centers.
This was his first post:
I stand by my position that this could have meant any number of things. The way things ultimately went was far from determined, and jumping to the worst case conclusion only made that ending more likely.
You certainly cannot read this as ‘America will have zero AI guardrails’ when Trump has already instituted super secret AI guardrails and yanked the top AI model from the market for weeks on safety concerns. We are also now indefinitely in a world where the White House arbitrarily decides who gets access to Claude Mythos.
None of this is compatible with wanting no guardrails at all.
There were those who said that my and Roon’s challenges of the media’s premature headlines were various versions of ‘insane cope’ and accused us of being naive or having poor reading comprehension.
Certainly those people score some Bayes points given what happened, and we lose them, given how many we would have stood to win at their expense had this gone the other way. That’s how it has to work. I take the L.
I still think I was ‘supposed to lose’ this one, and that the read was correct. At the time, we had insufficient evidence to know which way it would land.
If you think Trump’s first comment should have updated us all the way, and his later comments did not contain much information, I think you are incorrect. I still think it is likely that, at the time of his first comments, Trump himself did not know what he intended to do here.
Trump Goes Full ‘Hoax’
Then something got him fully on the warpath. I am guessing it was Jensen Huang.
Trump went Full Hoax.
This Is Not About Data Centers, Mr. President
And then:
It sure sounds like Trump thinks these warnings are a plot against data centers? Despite the fact that Dario Amodei and Sam Altman and Elon Musk are three of the top five people most heavily pushing to build American data centers, and actively funding a large fraction of them via their companies. Someone tell the President.
One can see how that compression got made. No, it makes no sense, but such is life.
Then the compression got solidified:
I am not going to try to litigate which missteps may or may not have contributed to this being the way it played out. The past is the past. This is our present.
I Am The Hoax Buster, I Am The Hoax Buster, I Am The Walrus
And then things went, well, to another level.
I know this is tragic, horrible and many other things. But.
If you can set those things aside it is also extremely funny. The song needs some work and unfortunately Grok can’t make him sing, but I tell you, there’s something there. He’s a singular talent. I couldn’t write that. You couldn’t write that.
By Astra’s count this will be at least the 20th such Hoax, joining Charlottesville, Russia, Russian bounties on American troops, the classified documents case, the New York civil fraud case, the accusations by E. Jean Carroll, that he has connections to Epstein, the accusations against Brett Kavanaugh, the affordability crisis, the official unemployment rate, the ‘suckers and losers’ story, the ‘bloodbath’ controversy, the Bubba Wallace noose incident, criticism of his response to Covid, climate change, first impeachment over Ukraine, second impeachment, January 6 as an insurrection, Biden winning the 2020 election and now that AI has existential risks.
This is reality. This is Earth. This is 2026. We saw that the President of the United States talks this way and we all thought ‘yeah, we know, that’s just how it is.’
Nvidia CEO Jensen Huang Is a Lying Liar
It is patently absurd that people claim not to notice that he is a constantly lying liar.
I say this as a shareholder. To claim that Coxon’s claims are ‘ignorant of the industry’s safety work’ means Huang is straight up lying. This has moved beyond mere bullshit. Do not pretend otherwise.
Later, at the All-In Summit, Jensen would moderate that.
Then he goes back to pretending this is all solved by ordinary product liability and market forces. If your AI kills everyone then we will sue you, and no one will buy your product anymore, and boy will you be sorry.
Presumably you noticed that no, we won’t sue you, on account of being dead and all.
Jensen is a maximally cynical commercial and political actor whose one and only goal is to sell Nvidia chips to whoever he can, especially China. This could not be clearer.
Jensen’s statements about AI existential risk, or any AI downsides, long ago stopped pretending to have any correlation with reality, other than what is good for Nvidia.
Jensen does not care if the world is destroyed, so long as he gets to have a great company. Which he already had, but this way it is slightly more great. Again.
At the All-In Tech Summit (of course), Jensen Huang went onstage to make his case, and took an about five minute phone call from Trump.
There is dispute between transcripts over who ‘they’ refers to here. The hosts offer a s transcript that attributes this to [doomers] and I believe that was the broad intent here.
Trump Quietly Draws Key Distinction
I do take solace in some word details. With a chaos agent like Trump, your careful interpretations can always get overtaken by events, things change constantly. I realize this is going to sound like splitting hairs. Fine.
But notice that he’s saying the ‘doomers’ are ‘playing right into the hands’ of people.
That is an implicit admission that, in Trump’s mind, the ‘doomers’ warning us are sincere. You do not ‘play into the hands of’ your co-conspirators or allies. This is the language you use to describe what in politics is called a Useful Idiot, someone who believes what they are saying in a way that acts to someone else’s advantage.
Trump is calling this a ‘hoax’ because he believes in, to attempt to coin a phrase, what one might call implication-based epistemology. Historically it has worked well for him. You may not like it, but this is what Peak Evidential Decision Theory looks like.
Calling For Pacing the Frontier Is Bad For AI Stock Prices
Zac Hill is the latest to remind us that if AI companies wanted to sell America on their new technology, they would do pretty much the opposite of what they are doing. Yet the product still sells itself.
This is all utterly obvious, and is why Jensen and Trump are so bent on calling this a ‘hoax.’ But many people still think the opposite, so let us be even more explicit:
Michael Burry has moved on from calling AI a bubble to calling it a cartel. For the reasons explained by Tabarrok, and demonstrated by the reactions of Jensen and Trump, this makes no sense. In his case I do not think he is lying. I think Burry is just in way too deep.
Matt Levine somehow disagrees. He thinks that Dario Amodei’s call was entirely sincere and also great marketing by expanding the total addressable market and reducing training costs. Matt Levine is always very funny, and excellent with financial infrastructure made of gears, and is great for making a steelman of all utterly ridiculous claims that something is an antitrust violation, great marketing or classically securities fraud. Yes, you can imagine such complaints, indeed I have seen them.
What Matt Levine has never been that interested in is whether the ludicrous steelman he constructs has any merit, or whether anyone could actually say it in good faith. That’s not his department. He is here to muse. I respect that.
Never change, Matt Levine, the jester can speak truth to the king:
People On The Internet Sometimes Lie
Ad hominem is famously almost the lowest level of discourse, but actually there is an additional lower level: Ad hominem attacks that are also exactly the opposite of true.
The opposite is true. Those worried about AI have always overwhelmingly favored nuclear power and we know climate change is real but such folks are almost never ‘climate doomers’ and have argued for years to focus elsewhere. This is straight lying. Either Erick is lying, or he has been lied to and did not care to check, at all.
Origins of Cynicism
A lot of people say that AI might kill everyone. A lot of them are in a position to know.
There are those for whom ‘they believe it’ does not compute as an explanation. Why?
I think those are the two of the three main true reasons for believing that everyone is lying about this, which I would summarize as:
Then, in all three cases: Therefore, they are lying, QED.
David Shor links us to a paper from 2018 called The Cynical Genius Illusion: Exploring and Debunking Lay Beliefs About Cynicism and Competence, as illustrative of this style of thinking. The galaxy brains are out in force.
This is both, people say, a cynical ploy to pump up valuations, and a cynical ploy to set up excuses if valuations go down.
Ineffective Egoism
Definitely do not panic about ‘Effective Altruists’ being called out by name.
This scapegoat campaign started long ago, spearheaded by Marc Andreessen.
No. They’re not.
I mean, don’t get me wrong, a16z has been trying, and they have in some ways made it a dirty word in politics in particular. But this kind of blind rage should if anything be welcomed.
For those who think that if an AI went rogue we would all collectively work together to deal with it, well, the US Department of War has gone rogue and no one seems inclined to do anything about it, perhaps because it is so obviously ineffective.
Somehow this is real.
First they ignore you, then they laugh at you, then they fight you, then you win.
This also invokes the Streisand Effect. Essentially no one knows what Effective Altruism is, or most of the ideas around AI existential risk. If you start railing against EA, especially in a highly polarized environment, what do you think happens?
Attempting to launch a new McCarthyism not only will not work, it will cause everyone to go ‘hey check out this thing they’re trying to do a McCarthyism on.’
And then they find out it’s about trying to do the most good via charity.
The McCarthyist Faction Attacks METR
This is a logical first target. Maybe I should be more concerned but mostly I find it counterproductive and more importantly hilarious.
This is the actual, real life front page of the New York Post, enshrining this story alongside past winners like ‘Headless Body in Topless Bar’:
Who wouldn’t want to be the ‘Wizards of AI’?
If there was a new version of Pangram that used AI to detect Hack Job Hit Piece Voice, you would not need it, because this is so classic. You see, [A] claims to be independent of [B], but [A] traces its roots back to [C], which once associated with [D], and [D] was a bad, bad person, who once was associated with [B], so checkmate [outgroup], we have discovered your conspiracy.
It continues in this vein throughout. You see, there are (gasp) Effective Altruists at METR, and some of the same people who invested in Anthropic also helped fund METR, like Jaan Tallinn, it must be some grand conspiracy. And so on. There are zero new facts, or even anything that is not common knowledge among those paying attention.
The ‘complete joke’ line was obtained from Perry Metzger, who goes around posting unhinged rants in reply to various Twitter posts including my own, and is chairman of the industry anti-regulation group Alliance for the Future. That offers a guess as to the origins of Barrabi’s hit piece.
So does this, which could be either a deep confusion at the White House, where they have been known to make such conflations, or the source making things up.
The ‘woke’ label is purely something they threw in the headline and first sentence, with neither any actual truth to it nor any evidence provided, as per usual. METR is not, in any sense whatsoever, ‘woke.’
The ‘globalist’ label in the headline is a complete non-sequitur. Usual playbook.
The other implicit argument is, look at these large photos of so many prominent METR employees who, unlike Dario Amodei, photograph well and look good.
That is not to say that there are no issues with potential conflicts of interest. Yes, of course we also will want auditors that bring additional perspectives, to work alongside METR, both at Anthropic and elsewhere. If you want the ‘fair play’ outside view hit piece, see SE Gyges.
The issues that are real here are well known, acknowledged, and were not avoidable. You need former lab employees and those whose backgrounds have links to the labs, because no one else has the technical expertise to do the job. And you need auditors to get paid enough to attract strong talent and not charge for their services, which means you need to hit up the philanthropists who care about such issues. Most industries have some form of exactly this problem.
It would be really great, you would totally own the libs, if you were to spin up your own outside view highly credible and technically competent monitoring organizations to compete in this space. My lord, I hope no one does that, we definitely don’t need a lot more such efforts. Would be a real shame.
It would be even worse for Anthropic and METR if you were to say that in order to generate a real check on Anthropic you need the audits and evaluators to be universal, mandatory and protected by law, and otherwise given more teeth to be a ‘real check.’
For additional fun color, this was the reporter’s previous hack hit job effort, which some say was even funnier but was tragically denied the front page:
To explain: This was an Obvious Nonsense hit piece against Dean Ball full of clear untruths, and also the insert picture that was supposed to be Dean Ball was for some reason instead Caleb Watney. See here for my full coverage from AI #181.
There are indeed all the signs of a coordinated smear campaign against METR to make them sound bad to right-wing echo chambers. The next few weeks are likely to suck in many ways for the METR folks, especially as some rather vicious people dig around for the tiniest bit of actual dirt. I do not expect this to work, indeed I expect it to backfire via the Streisand Effect.
Laugh it off as best you can. Stay strong. Hang it in the Louvre.
Other Key Republicans React
There are also many Republicans, with the mainstream arm of that led by Ron DeSantis but also increasingly many others, who increasingly understand the stakes or simply understand the politics.
The Usual Suspects may have Trump currently talking about a hoax, but this remains a bipartisan moment, and many are not going to fall in line. Even now the issue often remains remarkably bipartisan, and not only when you see Steve Bannon on stage with Bernie Sanders.
Axios’s post is largely about the bipartisan investigation of the HuggingFace attacks where Senator Hawley is taking point, but it also includes the more recent concerns.
House Speaker Mike Johnson continues to act sanely and this continues to be reported strangely. First Politico tried to make it sound like Johnson wanted to fully pass the buck to the companies, and now Bloomberg says he ‘risks Trump’s wrath’ over his very milquetoast statements:
Indeed, Johnson said Monday that there will be a White House meeting with AI executives by early next week at the latest, prior to the China summit.
Even now, Trump is being misinterpreted. Of course Trump endorses the need for guardrails. Trump said on Sunday on Air Force One, to reporters, that ‘we can put guardrails.’ Trump is saying that there is no need to worry because he is such a great President that will ensure the right guardrails, not that we don’t need any at all.
Again, he has a super secret prior restraint testing regime already in place and his administration yanked Fable 5 from the market in a panic. There is no ‘wrath’ here.
Republican Senator Rick Scott says the AI companies have said what they are building could kill mankind, so pretty obvious they need to slow down and get their house in order so we can move forward safely, with America leading the way or else China will lead, which will hurt, especially for our kids, whose safety our good corporate citizens must protect online.
I’ve read this three times and I still can’t figure out to what extent he understands the situation. Which is progressively making it funnier.
Ron DeSantis continues to call for channeling our technological innovations rather than letting the AI labs do whatever they want.
Republican Utah Governor Spencer Cox calls for Congress to establish a consistent national framework for AI, requiring incident reporting, protecting whistleblowers, ensuring independent evaluation and imposing strict export controls.
(The Tweet is by AI but the full statement is human written.)
David Sacks Stops Being Plausibly Constructive
JD Vance finds the whole thing ‘a little weird’ with companies asking for regulation, saying it feels ‘a little bit to me like a Trojan horse.’
David Sacks was initially hostile but in a potentially constructive way. He has now retreated to joining the other Usual Suspects like his co-host Jason and is back to fully lying about what is being requested, as per usual. You can say ‘let’s impose restrictions on only the frontier’ and the Usual Suspects will flat out claim you said the opposite, over and over, every time, no matter what.
Federal Trade Commission Chooses Danger
I am not going to pretend Andrew Ferguson actually thinks that leading AI labs working together on AI safety is suspicious or sounds like moat digging.
That view does not make physical sense. This whole move is obviously bad for business. That is why the boss does not like it. Alternatively, the boss thinks is a bargaining chip and he should get something in return. Fair enough.
Chris Lehane Heel Face Turn
OpenAI is walking the walk in some ways, but what about its policy team?
Alex Bores claims, consistent with my observations and reporting, that a16z, Chris Lehane and Leading the Future have, at least until recently, consistently followed the standard industry lobbyist playbook. You try to kill all the bills, then if you cannot kill the bill you claim to support the bill, and try to weaken the bill or shove your provisions into the bill, then turn around and say you supported the bill all along. Which they are now trying at the federal level.
If OpenAI and Greg Brockman and Chris Lehane were to all collectively pull their support and funding from Leading the Future, and disavow its activities, that would be an excellent way to show that this new pivot to safety is fully legit. Leading the Future could also fully pivot to data center concerns.
Matthew Yglesias takes the political perspective, where if you are an OpenAI employee who wants collective action your first step is to demand that OpenAI cease and desist here, and you should threaten to quit on this issue.
I don’t think that would be my top priority at OpenAI, given what else is happening, but it matters. I do think that it would be a top priority at Meta, where Mark Zuckerberg has been actively blocking action all the way at the top, and it makes sense to say first things first you have to stop before I will work for you, at any price.
The good news is that perhaps this is being addressed. Chris Lehane may be in a Heel Face Turn.
Lehane announced, per Bloomberg, OpenAI support for various legislative initiatives, ‘whatever we can get through.’ Quite the change of tune, if he means it.
A Matter of Trust
More substantively, Lehane is announcing that Google, OpenAI and Anthropic are working together on safety, and that OpenAI sees no need for an antitrust exemption to start doing so.
This is very good to see. As usual, it means the work begins, not ends, but it is a start.
OpenAI is also exploring the legality of a coordinated industry slowdown. The worry is that this might conflict rather more forcefully with antitrust law, potentially making it illegal to agree to not risk killing everyone.
I have always been in the camp that felt this was rather silly, this would never actually be a barrier in practice. It would still be very helpful to formally put this to bed and I am glad OpenAI is working on that, especially with the Federal Trade Commission now saying it is not inclined to make this easy.
China Calls It Fearmongering
China is responding as if calls to slow down Anthropic and OpenAI are a plot to ‘slow China down’ and be anti-competitive in some way. This is consistent with their standard rhetorical position.
There were a lot of warnings about China in Dario’s essay, and calls to slow them down via chip export controls. But if we wanted to actually be anti-competitive, we would just impose the controls and then keep building better AIs as fast as possible.
The story that this is primarily about China and being anti-competitive does not actually make sense. This is politics, so making sense is optional.
Pick Up The Phone
A key question in all this is: What would a deal with China look like?
Part of that is asking: What would China want as part of that deal? Why would China care about ‘America invests more in safety’ when they don’t care about safety in that sense, and they are happy to have America push forward faster so they can fast follow, the way they do everywhere else? Where their plan is distill, make it smaller, faster and cheaper, win the market, diffuse the tech, sell products and improve the world, but they’re not trying to get to superintelligence.
What I realized was, if that’s what Xi turns out to want, then you don’t need a deal with China, any more than we need a deal with Germany.
In that world, China is not going to be investing the sums it would take to actually catch and surpass the top American labs on big frontier models, even if we slow down. The enemy here is not China, it is ‘lose to China,’ the boogeyman pushing everyone forward.
All we need from China in that world is some amount of good stewardship. We need them not to allow model releases that would destroy the internet or cause a plague. Well, I have some good news, the PRC really likes the internet, and it does not like plagues.
Meanwhile, China has put out its version 3.0 of its AI Safety Governance Framework. Here are some translated quotes, this is prosaic stuff but they do pay attention:
Watch the hands, not the mouth, and so on.
If You Want To Beat China So Badly You Should Act Like It
As Roge Karma points out in The Atlantic, our current AI policy is contradictory.
If we are in an existential race with China, where ‘whoever AI wins!’ and nothing else can compare in importance, then the United States needs to act like it.
At a bare minimum, that starts with enforcing strong export controls on chips, and also on other ways for China to use American compute for training frontier models.
If one were fully serious, one would also do things like make it easy to brain drain China of its best AI engineers. If one were really fully serious, there would be a major push for all forms of electrical power and energy, and for things like building more housing especially in San Francisco, and for deregulating barriers to AI diffusion in places like the FDA, and so on. These are things I would strongly support.
What instead maddeningly happens is that when talking about the risk we might all die, competition with China on AI in particular becomes the only thing that matters, and then in other contexts we shrug and it is business as usual.
Never Go Full Hoax
Trump went full hoax. Other than the usual suspects and their Twitter vibe warriors who want full accelerationism at all times, and those who take orders from the boss, he is alone on an unpopular and unstable island. Speaker Johnson, Scott, Cox and DeSantis are far from the only Republicans to break ranks.
Even if the situation stabilizes for now, and will likely make it as far as the China summit, the next AI incident is coming. Trump responds very differently to concrete threats than to hypotheticals. And the ‘jailbreak demo’ from Amazon, that got Fable yanked, shows us that things can move into the ‘concrete threat’ basket without anyone having to get hurt first.
Meanwhile, the lab employees are continuing to freak out and make their voices heard, and the labs are starting to collaborate on their own.
Do not write things off. Do not let the partisan conflict intensify or become the story. We can still turn this around.
We can even use AI. For things.
Trump Uses AI For Things