This is a special post for quick takes by Philipreal. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
The United States Government has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States. This includes Anthropic employees who are foreign nationals.
Anthropic is currently disabling Fable 5 and Mythos 5 for all customers.
16We have reviewed a report that we believe is the basis of the government's directive and validated that the level of capability displayed there is widely available from other models (including OpenAI’s GPT-5.5), and is used every day by the defenders who keep systems safe.
...If this standard was applied across the industry, we believe it would essentially halt all new model deployments for all frontier model providers.
Export controls on AI are probably inevitable (and perhaps a good thing if done well), but this is pretty clearly retaliatory / punitive. OpenAI will presumably release a comparable model in the next few weeks or months, it will be similarly jailbreak-able, but not subject to the same restrictions. Then what?
Not everyone in the administration has the same agenda.
There are likely people who want export controls on all models and people who wants them on no models and putting on on Anthropics model was the easiest to get consensus for at this point in time.
That does not mean that OpenAI is safe in the future as those people who want more export controls can leverage the win.
I'd say, even if I'm a bit mad for not getting to use it anymore, this lowered my P(doom). Basically as I understand the government realized AI capabilities can't be just restricted with a simple safeguard (e.g you ask Fable 5 to find bugs in your codebase, and it will discover vulnerabilities too). And then set a precedent for banning AI models (at least by using ITAR restrictions). Other existing laws are only about "voluntary testing".
I think that it's good that a government body just demonstrated it's willing to pull a frontier model offline, that could have made billions in tax revenue, on short notice, and the company had to comply immediately. Even if it was through a primitive mechanism, before this the only regulations were only about mandatory reporting and voluntary testing.
If this repeats across models from different companies, and on different countries too, it removes a significant part of the economic incentives for developing more intelligent frontier models, as military applications are only a small fraction of the usage.
Something that also played a factor in my estimate, is that while Anthropic's system card for Mythos say that "it's the most aligned model to date", this is mostly based on having a better accuracy at refusing questions. To me, the model is not really aligned in a "deep" way, it has a high rate of reckless and deceptive behaviors, and I've seen glimpses of them when I used it.
However, the risk still exists, as even if access to frontier models requires heavy vetting, they might still be significant to the broader economy, so this might reduce misuse risks, but not risks from ASI loss of containment/control.
Edit: After reading Max H's comment, I figure this might be retaliatory due to the surrounding conflict between Anthropic and the US government and might not repeat with OpenAI's next models, in that case it would increase my P(doom).
I think that it's good that a government body just demonstrated it's willing to pull a frontier model offline
The government didn't pull a frontier model offline, it cut off access for non-Americans.
In the short term yes, in the medium term frontier AI companies, forced by the US government, can restrict access to frontier models to US Americans. Which would accelerate the US economy over the rest of the world, while the justification will be "national security".
How are they going to limit access to American citizens? I have no idea whether that's possible.
Some form of ID verification, or (less precisely) IP geo-blocking. This is already done on many websites for other purposes (copyright, local laws, sanctions etc).
They can still use geo-blocking rather than no blocking at all. They also don't have to call it "weapon export control legislation". Apart from geo-blocking, they can refuse service payments from foreign countries. Fable wasn't free. ID verification is also still on the table.
I don't see how this is a remotely feasible approach, how many non-Americans are there working for FAANG alone who use these models? Let alone their international operations. If this sticks I think it basically kills deployment of new models
In the worst outcome this effectively slows down US-based AI development, while motivating a lot of non-US citizens to move to more open development environments elsewhere. Most of the promise of AI is still unrealized and I believe premature restrictions can cause the opposite effect, as one can potentially see in the chip export restrictions to China.
Presumably the model currently used internally by Anthropic employees is not Mythos 5, it's Mythos 5.1 or something?
I sort of suspect this might be a storm in a teacup and the models will come back once the US gov realises that there's no universal jailbreak. But it does set a interesting new precedent if they plan to be consistent about it. I would have assumed Anthropic has the best anti-jailbreak measures given all their work on constitutional classifiers.
The US gov likely believes that their justification is flimsy and minimal. Their intent is likely to set a broad precedent for applying ITAR restrictions.
ITAR already bans 'uplift' to foreign militias by banning provision of 'advanced defense services', which includes most firearms training, under ITAR. The same goes for exporting certain classes of optics (night vision).
There are a ton of things, including computer software, restricted under ITAR. The legal precedents here are very strong.
So it's time for anyone who wanted regulation and slowdown to celebrate, the US government has chosen a regulatory regime for frontier models! And they picked a really aggressive one!
As of 21:54 ET (~1 hour after the statement on twitter), I still have access to Fable. Seems unclear when this goes into effect.
Well. That's the natural end result of Anthropic et al.'s attempts to frame the AI Risk as being about a competition with China, isn't it? Inasmuch as it hurts Anthropic, that's just deserts.
We believe this is a misunderstanding and are working to restore access as soon as possible.
I don't think this is a misunderstanding at all. I think "we found a jailbreak!!!" is just an excuse.
Here's to hoping this means the USG is now going to choke the AI industry to death by trying to nationalize it.[1] Some good tidings on that front already:
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.
Honestly... This may not be the most appropriate reaction, but boy do I find it funny.
- ^
Though it's likely just going to make it all worse somehow.
1I struggle to understand what the disagreement with this comment is, I'd appreciate it if someone could fill me in. What other logical response would there be if people in the US government came to accept Anthropic's messaging on the issue? Certainly we can't expect them to try and reach out to China for collaboration on the topic, given the policies of this government that we've seen thus far
I struggle to understand what the disagreement with this comment is
I'm guessing it's some of:
- Disagreement that this is due to Anthropic's anti-China messaging (which was anti-helpful for the AInotkilleveryoneism cause), as opposed to the DoW retaliating against Anthropic after it steadfastly refused to bow to the DoW's demands (which was arguably heroic) – and therefore disagreement that this is just deserts for Anthropic.
- Disagreement that there's any chance that the USG taking destructive actions may lead to on-net positive outcomes, and with the nihilistic-ish vibes of my hoping for things to go this way.
- Disagreement with my overall lighthearted/gleeful tone, which, again, may be considered inappropriate for this moment – if one is sympathetic to Anthropic, mourns Fable,[1] or expects things to develop badly from there.
I think it's fair to downvote that comment based on those, I did expect it to be controversial.
(I'd also be curious to know for certain what the downvotes are about, though!)
- ^
Which I actually do too, it was definitely a step change in how pleasant it was to talk to.
I wrote this nine years ago, maybe it was Anthropic's approach all along: https://www.lesswrong.com/posts/fTrEqNnYYSNXNSRcg/allegory-on-ai-risk-game-theory-and-mithril#8kyxf3bAmQBHrstm2
The one crazy company gets into so much trouble with a government that everyone else just kind of takes a pause and waits to see how it turns out...but the paperwork never ends and the energy/material resources that were available for advancing that tech are reallocated.
There, we are probably in a pause, do something useful with it please.