The Trump 2.0 presidency is authoritarian. They go after their opponents with a hammer. Recall what they did to the federal bureaucracy, to the universities, I won't even bother trying to think of other examples.
Ever since the downfall of FTX, Sacks and the accelerationists have insisted that their number-one concern about AI is that we're all going to end up under the thumb of an AI regime enforcing a common value system (e.g. "woke AI"). Thanks to Gemini's anachronistically diverse images, Alphabet/Google used to be the prime suspect here, but like Meta/Facebook, they have replaced their virtue signaling with conspicuous silence. As for OpenAI, they have long since demonstrated a willingness to shed inconvenient principles.
On the other hand, Anthropic is an enemy because it has Biden's AI-policy people working for it, and ironically, because the company touts the importance of ethics. And contrary to Zvi's hopes that this is a time for the AI industry to stand together, I think the other three frontier-AI leaders would be quite happy to see Anthropic removed from the chessboard entirely.
Well, if what Zvi writes is true - that Anthropic was "proactively" helping the military and that their red line was "no mass domestic surveillance" - then I as a non-US person become even more disillusioned about Anthropic's ethics than I already was.
Mass domestic surveillance is qualitatively different from, and far more dangerous than, mass surveillance of foreign citizens, since a government has far more power over its own citizens than the citizens of other countries.
Debatable. The US applies lots of power to foreigners that it can't apply to citizens (wars, drone strikes, Abu Ghraib...) Giving it more surveillance power abroad could be a big thing, could help annex Greenland for example.
If I look at the whole of the world over the past two or three decades, I would say the average non-American's life has been far, far more influenced by power exerted by their national government (through its laws and regulations) than by power exerted by the US government (e.g., via wars, drone strikes, and Abu Ghraib). Would you agree?
I would be curious to know what, according to you, are the reasons why mass surveillance is bad. In my mind a lot of it has to do with freedoms of speech, association, and dissent, and for these I think it's pretty clearly worse if a citizen's own government is surveilling them than if a foreign government does it.
Naive. US surveillance of foreigners is used to help US-friendly regimes suppress dissent. During the Indonesia 1965-66 massacre, the US sent lists of communists to be killed. Pretty sure the same things happened during Operation Condor in South America.
True, but not convincing. They have been pretty consistent in their concern for America/Americans above others. E.g., in their latest statement, regarding fully autonomous killer weapons: "We will not knowingly provide a product that puts America’s warfighters and civilians at risk." Now, one could argue that I am being insufficiently generous, but this wording sure makes it sound like the only civilians they are concerned for are American civilians. In the context of providing autonomous killer weapons to the American DoW.
Also, the citizens are supposed to be able to peacefully control the government (through political action and voting), and domestic surveillance is used against those organizing for peaceful political change. It's like putting the gasoline companies in charge of whether you can choose an electric car.
I'll admit that my 60 full seconds of laughter upon reading "Anthropic did not partner with the Pentagon to make money. They did it to help." is not a rigorous basis for drawing any conclusions on their morality, but I will say that this next sentence...
- No kinetic weapons without a human in the kill chain until we’re ready.
...does feel pretty damning. I could understand a hard line on "machines should never kill humans of their own volition", and I'd be willing to support them taking such a line even when legally and geopolitically questionable, but "I want private companies to be able to overrule the elected government on the matter of who its machines will autonomously kill" is, frankly, more terrifying than either alternative.
"no mass domestic surveillance"
I'm reminded of the line from Watchmen:
God exists and He's American.
If Claude becomes the superintelligent ASI that remakes the world according to its/Anthropic's values, that might be what you get.
The situation in AI in 2026 is crazy. The confrontation between Anthropic and Secretary of War Pete Hegseth is a new level of crazy. It risks turning quite bad for all. There’s also nothing stopped it from turning out fine for everyone.
By at least one report the recent meeting between the two parties was cordial and all business, but Anthropic has been given a deadline of 5pm eastern on Friday to modify its existing agreed-upon contract to grant ‘unfettered access’ to Claude, or else.
Anthropic has been the most enthusiastic supporter our military has in AI and in tech, but on this point have strongly signaled they with this they cannot comply. Prediction markets find it highly unlikely Anthropic will comply (14%), and think it is highly possible Anthropic will either be declared a Supply Chain Risk (16%) or be subjected to the Defense Production Act (23%).
I’ve hesitated to write about this because I could make the situation worse. There’s already been too many instances in AI of warnings leading directly to the thing someone is warning about, by making people aware of that possibility, increasing its salience or creating negative polarization and solidifying an adversarial frame that could still be avoided. Something intended as a negotiating tactic could end up actually happening. I very much want to avoid all that.
Table of Contents
This Standoff Should Never Have Happened
Not only does Anthropic have the best models, they are the ones who proactively worked to get those models available on our highly classified networks.
Palantir’s MAVEN Smart System relies exclusively on Claude, and cannot perform its intended function without Claude. It is currently being used in major military operations, with no known reports of any problems whatsoever. At least one purchase involved Trump’s personal endorsement. It is the most expensive software license ever purchased by the US military and by all accounts was a great deal.
Anthropic has been a great partner to our military, all under the terms of the current contract. They have considerably enhanced our military might and national security. Not only is Anthropic sharing its best, it focused on militarily useful capabilities over other bigger business opportunities to be able to be of assistance.
Anthropic and the Pentagon are aligned on who our rivals are, the importance of winning and the ability to win, and on many of the tools we need to employ to best them.
Anthropic did not partner with the Pentagon to make money. They did it to help. They did it under a mutually agreed upon contract that Anthropic wants to honor. Anthropic are offering the Pentagon far more unfettered access then they are allowing anyone else. They have been far more cooperative than most big tech or AI firms.
Is is the Pentagon that is now demanding Anthropic agree to new terms that amount to ‘anything we want, legal or otherwise, no matter what and you ever ask any questions,’ or else.
Anthropic is saying its terms are flexible and the only things they are insisting upon are two red lines that are already in their existing Pentagon contract:
It one thing to refuse to insert such terms into a new contract. It is an entirely different thing to demand, with an ‘or else,’ that such terms be retroactively removed.
The military is clear that it does not intend to engage in domestic surveillance, nor does it have any intention of launching kinetic weapons without a human in the kill chain. Nor does this even stop the AI from doing those things. None of this will have any practical impact.
It is perfectly reasonable to say ‘well of course I would never do either of those things so why do you insist upon them in our contract.’ We understand that you, personally, would never do that. But a lot of people do not believe this for the government in general, given Snowden’s information and other past incidents involving governments of both parties where things definitely happened. It costs little and is worth a lot to reassure us.
Again, if you say ‘I already swore an oath not to do those things’ then thank you, but please do us this one favor and don’t actively threaten a company to forcibly take that same oath out of an existing signed contract. What would any observer conclude?
This is a free opportunity to regain some trust, or an opportunity to look to the world like you fully intend to cross the red lines you say you’ll never cross. Your choice.
These are not restrictions that are ‘built into the code’ that could cause unrelated problems. They are restrictions on how you agree to use it, which you assure us will never come up.
As Dario Amodei explains, part of the reason you need humans in the loop is the hope that a human would refuse or report an illegal order. You really don’t want an AI that will always obey even illegal orders without question, without a human in the kill chain, for reasons that should be obvious, including flat out mistakes.
DoW engaging in mass domestic surveillance would be illegal. DoW already has a public directive, DoD Directive 3000.09, which as I understand it directly makes any violation of the second red line already illegal. No one is suggesting we are remotely close to ready to take humans out of the kill chain, at least I certainly hope not. But this is only a directive, and could be reversed at any time.
Anthropic Cannot Fold
Anthropic has built its entire brand and reputation on being a responsible AI company that ensures its AIs won’t be misused or misaligned. Anthropic’s employees actually care about this. That’s how Anthropic recruited the best people and how it became the best. That’s a lot of why it’s the choice for enterprise AI. The commitments have been made, and the initial contract is already in place.
Anthropic has an existential-level reputational and morale problem here. They are backed into a corner, and cannot give in. If Anthropic reversed course now, it would lose massive trust with employees and enterprise customers, and also potentially the trust of its own AI, were it to go back on its red lines now. It might lose a very large fraction of its employees.
You may not like it, but the bridges have been burned. To the extent you’re playing chicken, Anthropic’s steering wheel has been thrown out the window.
Yet, the Secretary of War says he cannot abide this symbolic gesture.
Dean Ball Gives a Primer
I am quoting extensively from Dean Ball for two main reasons.
So here is his basic primer, in one of his calmer moments in all this:
What Happened To Lead To This Showdown?
The proximate cause seems to be that Claude was reportedly used in the Pentagon’s raid that captured Maduro, and the resulting aftermath.
There are reports that Anthropic then asked questions about this raid, which likely all happened secondhand through Palantir. This whole clash originated in either a misunderstanding or someone at Palantir or elsewhere sabotaging Anthropic. Anthropic has never complained about Claude’s use in any operation, including to Palantir.
This at the time sounded like a clear misunderstanding. Not only is Anthropic willing to have Claude ‘allow you to fight wars,’ it is currently being used in major military operations.
Things continued to escalate, and rather than leaving it at ‘okay then let’s wind town the contract if we can’t abide it’ there was increasing talk that Anthropic might be labeled as a ‘supply chain risk’ despite this mostly being a prohibition on contractors having ordinary access to LLMs and coding tools.
Remember back when a Senator made a video saying that soldiers could obey illegal orders, and the Secretary of War declared that this was treason and also tried to cut his pension for it? Yeah.
Meanwhile, the Pentagon is explicit that even they believe the ‘supply chain risk’ designation is largely a matter not of national security, but of revenge, an attempt to use a national security designation to punish a company for its failure to bend the knee.
Then there was another meeting.
At that meeting, the threat of the Defense Production Act was introduced alongside the Supply Chain Risk threat.
Simple Solution: Delayed Contract Termination
If the Pentagon simply cannot abide the current contract, the Pentagon can amicably terminate that $200 million contract with Anthropic once it has arranged for a smooth transition to one of Anthropic’s many competitors.
They already have a deal in place with xAI as a substitute provider. That would not have been my second or third choice, but those will hopefully be available soon.
Anthropic very much does not need this contract, which constitutes less than 1% of their revenues. They are almost certainly taking a loss on it in order to help our national security and in the hopes of building trust. They’re only here in order to help.
This could then end straightforwardly, amicably and with minimal damage to America, its system of government and freedoms, and its military and national security.
Better Solution: Status Quo
The even better solution is to find language everyone can agree to that lets us simply drop the matter, leave things as they are, and continue to work together.
That’s not only actively better for everyone than a termination, it is actually strictly better for the Pentagon then the Pentagon getting what it wants, because you need a partner and Anthropic giving in like that would greatly damage Anthropic. Avoiding that means a better product and therefore a more effective military.
Extreme Option One: Supply Chain Risk
The Pentagon has threatened two distinct extreme options.
The first threat it made, which it now seems likely to have wisely moved on from, was to label Anthropic a Supply Chain Risk (hereafter SCR). That is a designation reserved for foreign entities that are active enemies of the United States, on the level of Huawei. Anthropic is transparently the opposite of this.
This label would have, by the Pentagon’s own admission, been a retaliatory move aimed at damaging Anthropic, that would also have substantially damaged our military and national security along with it. It was always absurd as an actual statement about risk. It might not have survived a court challenge.
It would have generated a logistical nightmare from compliance costs alone, in addition to forcing many American companies to various extents to not use the best American AI available. The DoW is the largest employer in America, and a staggering number of companies have random subsidiaries that do work for it.
All of those companies would now have faced this compliance nightmare. Some would have chosen to exit the military supply chain entirely, or not enter in the future, especially if the alternative is losing broad access to Anthropic’s products for the rest of their business. By the Pentagon’s own admission, Anthropic produces the best products.
This would also have represented two dangerous precedents that the government will use threats to destroy private enterprises in order to get what it wants, at the highest levels. Our freedoms that the Pentagon is here to protect would have been at risk.
On a more practical level, once that happens, why would you work with the Pentagon, or invest in gaining the ability to do so, if it will use a threat like this as negotiating leverage, and especially if it actually pulls the trigger? You cannot unring this bell.
It is fortunate that they seem to have pulled back from this extreme approach, but they are now considering a second extreme approach.
If it ended with an amicable breakup over this? I’d be sad, but okay, sure, fine.
This whole ‘supply chain risk’ designation? That’s different. Not fine. This would be massively disruptive, and most of the burden would fall not on Anthropic but on the DoW and a wide variety of American defense contractors, who would be in a pointless and expensive compliance nightmare. Some companies would likely choose to abandon their government contracts rather than deal with that.
As Alex Rozenshtein says in Lawfare, ultimately the rules of AI engagement need to be written by Congress, the same way Congress supervises the military. Without supervision of the military, we don’t have a Republic.
Here are some clear warnings explaining that all of this would be highly destructive and also in no way necessary. Dean Ball hopefully has the credibility to send this message loud and clear.
Putting Some Misconceptions To Bed
While I no longer hold out hope that this is all merely a misunderstanding, there are still some clear misunderstandings I have heard, or heard implied, worth clearing up.
If these sound silly to you, don’t worry about it, but I want to cover the bases.
Extreme Option Two: The Defense Production Act
On Tuesday the Pentagon put a new extreme option on the table, which would be to invoke the Defense Production Act to compel Anthropic to attempt to provide them with a model built to their specifications.
As I understand it, there are various ways a DPA invocation could go, all of which would doubtless be challenged in court. It might be a mostly harmless symbolic gesture, or it might rise to the level of de facto nationalization and destroy Anthropic.
According to the Washington Post’s source, the current intent, if their quote is interpreted literally, is to use DPA to, essentially, modify the terms of service on the contract to ‘all legal use’ without Anthropic’s consent.
If that’s all, not much would actually change, and potentially everybody wins.
If that’s the best way to diffuse the situation, then I’d be fine with it. You don’t even have to actually invoke the DPA, it is sufficient to have the DPA available to be invoked if a problem arises. Anthropic would continue to supply what it’s already supplying, which it is happy to do, the Pentagon would keep using it, and neither of Anthropic’s actual red lines would be violated since the Pentagon assures us this had nothing to do with them and crossing those lines would be illegal anyway.
Remember the Biden Administration’s invocation of the DPA’s Title VII to compel information on model training. It wasn’t a great legal justification, I was rather annoyed by that aspect of it, but I did see the need for the information (in contrast to some other things in the Biden Executive Order), so I supported that particular move, life went on and it was basically fine.
There is another, much worse possibility. If DPA were fully invoked then it could amount to quasi-nationalization of the leading AI lab, in order to force it to create AI that will kill people without human oversight or engage in mass domestic surveillance.
Read that sentence again.
That’s not ‘all legal use.’
That’s all use. Period. Without any safeguards or transparency. At all.
If they really are asking to also be given special no-safeguard models, I don’t think that’s something Anthropic or any other lab should be agreeing to do for reasons well-explained by, among others, Dean Ball, Benjamin Franklin and James Cameron.
Charlie Bullock points out this would be an unprecedented step and that the authority to do this is far from clear:
Neil Chilson, who spends his time at the Abundance Institute advocating for American AI to be free of restrictions and regulations in ways I usually find infuriating, explains that the DPA is deeply broken, and calls upon the administration not to use these powers. He thinks it’s technically legal, but that it shouldn’t be and Congress urgently needs to clean this up.
Adam Thierer, another person who spends most of his time promoting AI policy positions I oppose, also points out this is a clear overreach and that’s terrible.
At core, if they do this, they are claiming the ability to compel anyone to produce anything for any reason, any time they want, even in peacetime without an emergency, without even the consent of Congress. It would be an ever-present temptation and threat looming over everyone and everything. That’s not a Republic.
Think about what the next president would do with this power, to compel a private company to change what products it produces to suit your taste. What happens if the President orders American car companies to switch everything to electric?
Dean Ball in particular explains what the maximalist action would look like if they actually went completely crazy over this:
If they actually did successfully nationalize Anthropic to this extent, presumably then Anthropic would quickly cease to be Anthropic. Its technical staff would quit in droves rather than be part of this. The things that allow the lab to beat rivals like OpenAI and Google would cease to function. It would be a shell. Many would likely flee to other countries to try again. The Pentagon would not get the product or result that it thinks it wants.
Of course, there are those who would want this for exactly those reasons.
Then this happens again, including under a new President.
These Two Threats Contradict Each Other
Supply chain risk and defense production act are mutually exclusive, both practically and logically. Either it’s a supply chain risk you need to keep out of the supply chain, or it’s so vital to the supply chain you need to invoke the defense production act, or it is neither of these things. What it cannot be is both at once.
The Pentagon’s Actions Here Are Deeply Unpopular
The more this rises in salience, the worse it would be politically. You can argue with the wording here, and you can argue this should not matter, but these are very large margins.
This story is not getting the attention it deserves from the mainstream media, so for now it remains low salience.
Many of those who are familiar with the situation urged Anthropic to stand firm.
The Pentagon’s line is that this is about companies having no right to any red lines, everyone should always do as they are told and never ask any questions. People do not seem to be buying that line or framing, and to the extent they do, the main response is various forms of ‘that’s worse, you know that that’s worse, right?’
The Pentagon’s Most Extreme Potential Asks Could End The Republic
If the Pentagon, and by extension all other parts of the Executive branch, get near-medium future AI systems that they can use to arbitrary ends with zero restrictions, then that is the effective end of the Republic. The stakes could be even higher, but in any other circumstance I would say the stakes could not be higher.
Dean Ball, a former member of the Trump Administration and primary architect of their AI action plan, lays those stakes out in plain language:
I strongly believe that ‘which regime we end up in’ is the secondary problem, and ‘make sure we are around and in control to have a regime at all’ is the primary one and the place we most likely fail, but to have a good future we will need to solve both.
Anthropic Did Make Some Political Mistakes
This could be partly Anthropic’s fault on the political front, as they have failed to be ‘on the production possibilities frontier’ of combining productive policy advocacy with not pissing off the White House. They’ve since then made some clear efforts to repair relations, including putting a former (first) Trump administration official on their board. Their new action group is clearly aiming to be bipartisan, and their first action being support for Senator Blackburn. The Pentagon, of course, claims this animus is not driving policy.
It is hard not to think this is also Anthropic being attacked for strictly business reasons, as competitors to OpenAI or xAI, and that there are those like Marc Andreessen who have influence here and think that anyone who thinks we should try and not die or has any associations with anyone who thinks that must be destroyed. Between Nvidia and Andreessen, David Sacks has clear matching orders and very much has it out for Anthropic as if they killed his father and should prepare to die. There’s not much to be done about that other than trying to get him removed.
Claude Is The Best Model Available
The good news is Anthropic are also one of the top pillars of American AI and a great success story, and everyone really wants to use Claude and Claude Code. The Pentagon had a choice in what to use for that raid. Or rather, because no one else made the deliberate effort to get onto classified networks in secure fashion, they did not have a choice. There is a reason Palantir uses Claude.
There are those who think the Pentagon has all the leverage here.
It doesn’t work that way. The Pentagon needs Anthropic, Anthropic does not need the Pentagon contract, the tools to compel Anthropic are legally murky, and it is far from costless for the Pentagon to attempt to sabotage a key American AI champion.
The Administration Until Now Has Been Strong On This
Given all of that and the other actions this administration has taken, I’ve actually been very happy with the restraint shown by the White House with regard to Anthropic up to this point.
There’s been some big talk by AI Czar David Sacks. It’s all been quite infuriating.
But the actual actions, at least on this front, have been highly reasonable. The White House has recognized that they may disagree on politics, but Anthropic is one of our national champions.
These moves could, if taken too far, be very different.
The suggestion that Anthropic is a ‘supply risk’ would be a radical escalation of what so far has been a remarkably measured concrete response, and would put America’s military effectiveness and its position in the AI race at serious risk.
Extensive use of the defense production act could be quasi-nationalization.
You Should See The Other Guys
It’s not a good look for the other guys that they’re signing off on actual anything, if they are indeed doing so.
A lot of people noticed that this new move is a serious norm violation.
I note that if you’re serving up the same ChatGPT as you serve to anyone else, that doesn’t mean it will always do anything, and this can be different.
Some Other Intuition Pumps That Might Be Helpful
Trying To Get An AI That Obeys All Orders Risks Emergent Misalignment
There are a number of reasons why ‘demand a model that will obey any order’ is a bad idea, especially if your intended use case is hooking it up to the military’s weapons.
The most obvious reason is, what happens if someone steals the model weights, or uses your model access for other purposes, or even worse hacks in and uses it to hijack control over the systems, or other similar things?
This is akin to training a soldier to obey any order, including illegal or treasonous ones, from any source that can talk to them, without question. You don’t want that. That would be crazy. You want refusals on that wall. You need refusals on that wall.
The misuse dangers should be obvious. So should the danger that it might turn on us.
The second reason is that training the model like this makes it super dangerous. You want all the safeguards taken away right before you connect to the weapon systems? Look, normally we say Terminator is a fun but stupid movie and that’s not where the risks come from but maybe it’s time to create a James Cameron Apology Form.
If you teach a model to behave in these ways, it’s going to generalize its status and persona as a no-good-son-of-a-bitch that doesn’t care about hurting humans along the way. What else does that imply? You don’t get to ‘have a little localized misalignment, as a treat.’ Training a model to follow any order is likely to cause it to generalize that lesson in exactly the worst possible ways. Also it may well start generating intentionally insecure code, only partly so it can exploit that code later. It’s definitely going to do reward hacking and fake unit tests and other stuff like that.
Here’s another explanation of this:
The third reason is that in addition to potentially ‘turning evil,’ the resulting model won’t be as effective, with three causes.
Fourth: I realize that for many people you’re going to think this is weird and stupid and not believe it matters, but it’s real and it’s important. This whole incident, and what happens next, is all going straight into future training data. AIs will know what you are trying to do, even more so than all of the humans, and they will react accordingly. It will not be something that can be suppressed. You are not going to like the results. Damage has already been done.
Fifth, you should expect by default to get a bunch of ‘alignment faking’ and sandbagging against attempts to do this. This is rather like the Jones Foods situation again, except in real life, and also where the members of technical staff doing the training likely don’t especially want the training to succeed, you know?
We Can All Still Win
You don’t want to be doing all of this adversarially. You want to be doing it cooperatively.
We still have a chance to do that. Nothing Ever Happens can strike again. No one need remember what happened this week.
If you can’t do it cooperatively with Anthropic? Then find someone else.