This article seems somewhat negative, but the situation with Glasswing and some kind of informal model safety inspection just now is close to my realistic best case scenario.
That is, getting a superintelligent hacker that finds security vulnerabilities before it gains other capabilities is great. Firstly if it is extremely good, a later misaligned AI would not be able to cause hacking danger. Secondly it has already got companies and orgs cooperating and pre-empting potential dangers. Sure your current govt has major issues but there is a good chance that will improve before ASI. The process can be improved.
The White House has ordered Anthropic not to expand access to Mythos, and is at least seriously considering a complete about-face of American Frontier AI policy into a full prior restraint regime, where anyone wishing to release a highly capable new model will have to ask for permission.
This would be the antithesis of all their previous rhetoric, and all their actions to systematically avoid laying a foundation to do this in an orderly and informed fashion.
But now, with the existence of Mythos, and a potential coming hackastrophe where cyber attackers will by default have the edge and we desperately need defenders to have a head start, it is not clear they feel they have a choice.
If implemented well, this could be the right thing.
By default, it won’t be implemented well.
Project Glasswing Cannot Expand
The government is now deciding which models can and cannot be made available on particular terms to particular parties. This is already happening.
Anthropic wanted to expand the number of companies with access to Mythos as part of Project Glasswing. The White House said no.
It is not clear this is any of the White House’s damn business, legally speaking, but Anthropic honored their refusal. It is not clear what would have happened if they had done it anyway, but I strongly agree that it would have been unwise to find out.
Neil Chilson points out that while little harm is being done this time by denying Anthropic’s ability to widen the deployment of Mythos, the precedent of the White House vetoing Anthropic’s deployments of Mythos is concerning. As he says, arbitrary and informal government decision making can be even worse than formal regulatory regimes, favoring the connected and insiders. I’d add it also prevents the ability to plan and enables massive corruption.
That lack of harm assumes the decision to not expand is wize. One dynamic here is that the European Union is pressing Anthropic to give its key firms access, Anthropic wants to say yes, and this is what the White House is refusing to allow. Is this security concerns, or is this the White House being pissed at or looking to hack the Europeans and punishing them by not letting them secure their systems? Of course, one could say this is just desserts for having pretended the American AI advantages were all fake instead of securing access.
And now, it looks like this is not going to be a one-off incident. Oh no.
The Ad-Hoc Prior Restraint Era Begins
White House Considers Vetting A.I. Models Before They Are Released.
How would this work? There would be an executive order creating an ‘AI working group’ of tech executives and government officials to examine potential procedures, up to and including a government review process.
A good implementation of a prior restraint regime for true frontier model releases, isolated to the biggest models of the leading labs and with formalized procedures that are difficult to abuse, is a good and eventually (perhaps soon or even now) even a necessary thing.
I fear that is not what we are going to get. As Dean Ball and Neil Chilson point out, and Shakeem emphasizes, we are looking at a solution well outside the efficient frontier, full of ad-hockery. Because of course we are.
Guess what happens when you fail to prepare for or enact reasonable regulations? When the crisis takes you by surprise? You end up doing ad-hoc things in the heat of the moment instead, that on every level are worse. A tale as old as time, many such cases, etc. We were assured this moment would never come, that anyone advocating for even the precursors of such rules would be a tyrant the likes of which the world has never seen, and then the moment came And, well, here we are. Que the music.
The New York Times writeup says this is partly the result of David Sacks leaving his duties, and being replaced by a combination of Chief of Staff Susie Wiles and Treasury Secretary Scott Bessent.
The NYT write-up claims Britain has and is developing such a review and prior restraint model. This is not the case. The UK AISI reviews models prior to release, but this is entirely voluntary. Labs cooperate because they find it useful.
Perhaps Neil Chilson is right and all of this is vanishingly unlikely. I do not think so. I don’t think it is a done deal by any means, but things like this are inevitable once those involved understand the implications of frontier AI capabilities. Even if it does not happen this time around, it is mostly a matter of time.
It was always a matter of time and the talking of price. We could have used that time to do a decent job of it. We still could, but time is short, the rhetorical well was poisoned by bad faith arguments, and it is now going to be a lot harder. Were previous proposed thresholds and timings premature? Yes, and it plausibly is still too early, but when you deal with an exponential your choices are too early or too late. No longer plausibly too early means definitely too late.
Whether or not all of this is necessary, the price we pay is steep. We cannot flinch from it. All the arguments that have been offered against such a regime, and all the negative consequences, still apply. If and when we do get such a system, as Gail Weiner points out, this slows diffusion and thus public benefit, elite capture accelerates as connected and approved insider corporations get early access and work the system, and there is more incentive to not depend on American AI models. That’s how it works, and the more ad-hoc the system is the more those things happen.
That, and similar related issues, are why the idea of asking for prior restraint was always so politically toxic, and only considered in extremis with a heavy heart. When bills were proposed involving such systems, the very organizations proposing such model bills were essentially run out of town on a rail for daring to suggest even what such a system might look like.
So now we may soon have such a system, only without a thoughtful design.
Implementation Through CAISI
If we’re going to do this, the obvious reasonable way is via CAISI. They have now added Google, Microsoft and xAI (SpaceX) to the list of companies that have screening agreements with CAISI, along with Anthropic and OpenAI.
So far, these tests have not carried any consequences. They’re ‘for information purposes only.’ The government could still then use that information to decide to stop a release.
The leverage available can go well beyond exclusion from the federal marketplace.
The question is, will these tests turn into something with teeth? Will it be possible to ‘fail’ such a test (or is it pass?) and have the government tell you not to release? That would be the logical next step, along with gently informing everyone relevant they had better sign up.
If implemented well, that could be a good method. Even if we’re not going to do prior restraint, I will be happy to see CAISI testing all the important new releases, which has been shown not to appreciably slow releases down.
Then, if a true “holy ****” moment happens, we can deal with it. Not as good as a formalized full system, but better than pure ad-hockery.
What Should We Do About AI?
Ben Buchanan and Dean Ball coauthor a NY Times editorial on cybersecurity policy, with the basic message being to wake up and actually do the minimum things like real and enforced chip export controls and guardrails on AI development, while cooperating with China on catastrophic risk management. You presumably know all this already, but hopefully this tells people who need to know and don’t know.
Dean Ball lays out his overall philosophy on politics and AI, that he is a classical liberal who opposes almost every regulatory action on AI and technology (and, mostly, on everything else) with one notable rare exception for management of AI catastrophic risks, here. The arguments would apply even more to any existential risks worth worrying about.
Dean Ball also has a companion piece on Hyperdimensional. As he says, the regime of ‘because the White House arbitrarily said so’ is one of the worst regimes for deciding whether new AI models can be released, but that’s the track we are on right now. Imagine what the government can and will do with that kind of leverage and power.
So yes, of course, it looks like we’re going to by default stumble into that fully arbitrary ad-hoc regime. That’s today’s main focus, although it ties into other choices.
Given that their other choices almost amount to deliberate misalignment, plus the usual worries about ad-hoc exercise of and concentration of power, we should worry.
The Chain of Command Nonsense Continues
One of the things in the new memo announcement is very much unlike the others, and represents a huge break and reversal in AI policy, as discussed above. This section is about the others parts of the statement, which are more of their demands of absolute obedience.
No one wants to or has attempted to ‘interfere with the military’s chain of command,’ any more than I have attempted to do so. This opens the door to interpret this as ‘attempt to actually challenge the chain of command and tell the military what to do,’ in which case it’s all good. The danger is, do they interpret this as another version of ‘when Pete Hegseth says jump you ask how high and otherwise never ask any questions,’ in which case no, go home, sir, you’re… overstepping your authority.
I am hopeful, because blackballing Anthropic is no good for anyone. Well, not good for any American without competing commercial or other private interests.
You know what is not helping? Pete Hegseth continuing to call Dario Amodei an ‘ideological lunatic.’ Which, like other comments before it, is way worse than anything that was in the internal memo whose leaking caused a full cutoff in negotiations.
The Government Should Maintain Multiple AI Providers
There is one clearly good part of the above memo. The new principle of ‘have multiple AI providers available to agencies at all times’ is the right call. You want resilient backups for everything. The model providers physically can’t withdraw what they have already deployed, but there’s no reason to risk getting backed into a corner, and you never know which tool will be right for which job.
How’s It Going To End?
As Dean Ball puts it, part of the government has now realized some of the security implications of frontier AI systems, and right on schedule it is freaking the hell out, and looking to take control and use this for its own advantage.
Even if that starts out coming from a good place, by default controlled access and prior restraint will turn into a weapon of insiders against outsiders, a tool of leverage and corruption, and ultimately an attempt to control just about everything.
You can minimize this by doing it systematically and with clear rules, rather than going ad hoc or asking the companies themselves. That does not seem to be the plan.
The alternative plan, of insisting that AI companies should release all their frontier models (or even their weights) indefinitely without checking in first, and let the internet sort them out, was only ever going to work out if capabilities hit a plateau. Thus, a lot of arguments that a plateau had been reached or was arriving Real Soon Now, when those paying attention knew that was not the case.
A dedicated campaign of rhetoric made it impossible to point out the coming problem without getting absolutely buried in bile. That did not stop reality. Now here we are.
The best thing we can do now is figure out how to do this wisely, and convince those in charge to do it wisely, before it is instead done unwisely, to minimize the potential for abuse and for damage done, and to do our best to limit its scope to where it is actually necessary.