That is better than I expected from OpenAI.
I was similarly impressed with Anthropic's When AI Builds Itself. It focuses on the same issue, RSI, with very careful evidence, and the same honesty without mentioning the ticklish issue of it maybe killing us all (but they do prominently say "losing control"). They don't propose specific solutions, although they do express hopes for democratic control and say that we've controlled and monitored the development of other dangerous technologies before.
What Do We Want?
They state these desirea:- Address frontier risks to national security and public safety.
- Advance democratic governance.
- Promote transparency.
- Protect innovation.
- Build adaptive institutions.
Those are great, and the full descriptions are better, these go beyond the basic applause lights although yes there are plenty of applause lights: A common theme throughout the document is that OpenAI wants concentrate responsibility in CAISI, and avoid concentrating power in the NSA. I strongly agree with this and believe it is important. These priorities are in conflict, but this is an excellent start. The later parts of the document offer an implementation plan, which at the high level seems like clearly good ideas, that constitute a good start:A National Framework
OpenAI proposes basing the national framework on California’s SB 53, New York’s RAISE Act and Illinois’s SB 315, as enforced by a strengthened CAISI:- Severe risk evaluations and mitigations.
- Transparency requirements, including publishing frontier safety frameworks.
- Independent assessment and auditing.
- Critical safety incident detection and reporting.
- Model weight security requirements.
- Whistleblower protections.
- Meaningful accountability mechanisms. Liability. No blanket safe harbors.
In exchange, Congress would preempt state laws around frontier safety. In principle, if details are good, this could be The Way, the Minimum Viable Deal that someone trying not to die could consider accepting. There are five big concerns.- Meaningful accountability has to actually mean meaningful accountability, in addition to preserving liability for the underlying harms. Million dollar fines for violations do not cut it at this level. Billion dollar fines probably don’t, either, not on their own. We need to be able actually force compliance.
- If we move these bills to the Federal level, then we all know who is in charge of implementation and enforcement. There is a real possibility that we pass a real bill with real consequences if enforced, and it just… isn’t. Or is enforced selectively and capriciously. Being able to only counts if you actually do it.
- By default what happens is we take the watered down political compromises of the state bills, and then there are further compromises, watering them down further, when these are already pretty weaksauce bills. The initial offering risks being the best offer, the thing you then have to defend, rather than a first step. We need to go beyond the union of the bills, and beyond SB 315, before this becomes a deal worth taking, especially given the risks of non-enforcement. If we get into ‘this deal’s getting worse and worse all the time’ mode, it’s time to bail.
- If we accept preemption of all ‘frontier risks’ this risks meaning that states can enact counterproductive laws about AI but not the ones that matter, and there is the risk Congress never again acts. We want to be very careful with exactly what we preempt. It’s fine and even good to rule out similar provisions (e.g. we’ve got the auditing and frameworks and transparency, specifically) but if it was all of ‘frontier risk’ and we despair of getting Congress to act again then that’s a very price to pay for modest transparency alone.
- This could be treated as ‘we did it we fixed frontier risk’ and, well, very much no.
The basic problem is, if we agree to center all this enforcement at the Federal level, and accept preemption, then it needs to be enforced, and enforced with teeth, and the preemption properly tailored, and we have very good reason to worry this won’t be the case. And we have to worry that the deal we think we get doesn’t become one end of a negotiation or slippery slope. Once the preemption is in place, we don’t know that further modifications would be in the good direction. Nathan Calvin similarly likes a lot of other things here, but is justifiably extremely wary of accepting preemption and giving up this reverse federalism. I agree that this part of the deal needs to be evaluated very carefully. It’s all good to say ‘implement appropriate safeguards’ and ‘meaningful enforcement mechanisms’ but by default you end up with not those things. Still, this is the serious proposal, or at least a serious starting point. I would seriously consider something built along similar lines.Building State Capacity And CAISI
The first part is scary, because it is necessary but could easily backfire. The second part is flat out necessary. It starts out with properly authorizing and funding CAISI, giving them hiring authorities and public-service pathways and secure classified compute and a mandatory evaluation process that goes through CAISI rather than the NSA. Independent technical assessments should be supported. Mandatory evaluation carries obvious dangers, but if it is time-boxed and does not on its own block deployment it seems hard to argue. As in, you need to get the evaluations, within a time frame that won’t appreciably slow you down. That’s something that actively helps frontier labs. It doesn’t mean you must block development if the numbers don’t come out right. The tricky part is, if the answer comes back flashing red, what do you do? In practice, if the red flashes loud enough, this will cause the national security state and executive to act whether or not they are given specific authorization to do that, and also all the major labs will take the hint and cooperate. That’s the same logic of why the current ‘voluntary’ executive order is not all that voluntary.Whole-Of-Government Resilience
This is a grab bag of other good things worth doing:- Facilitate collaboration and international coordination on frontier AI safety, especially communicating around progress towards RSI.
- Protect America’s compute advantage.
- Restrict the adoption of unevaluated frontier AI systems. Yes, if you haven’t done the safety evaluations yet, don’t adopt the associated frontier models. Duh.
- Ensure defensive capabilities scale faster than offensive capabilities.
- Prepare for future resilience challenges
A bunch of this is rather vague, and it’s not a complete list, but yes, sure.Reasonableness Rising
Thus, I am very happy to see OpenAI proposing this blueprint. This is a highly reasonable document, well exceeding expectations. If this becomes OpenAI’s position, including that of the PACs it supports, and they stop using various underhanded tactics especially false flags, then I would be fine with that. We shall see. The discussions are going in good directions. My hope is to be in a position, soon, where it is hard to predict where my biggest concerns will lie with seriously proposed new bills.