We need commitments for 25% compute for safety by 205 ECI points, progressively increasing with further capabilities.
(Or at least push for governmental pressure on companies to do so.)
Or, rather, we need specific, middle-ground commitments with concrete demands and conditions that are triggered. The example proposed is not random and also, obviously, not enough: asking for 25% compute for safety on the eve of TED-AI - estimated by AI Futures Model to be 210 - seems like a bad joke compared to the demands from compute-based plans like How to Pace the AI Frontier or Pacing AI via Compute Reallocation, which often asked for research compute to be limited to <20%. But it is something that, IMO, can happen and can set important institutional precedents that serve to prepare for a more sophisticated pause.
Why I chose 25% compute by 205 ECI:
Nuclear plants often have 20-30% at the lower bound of their capital costs going into safety protocols and procedures. Source,Source (look at Appendix 1 - Hitachi 3b). I pick a midpoint of 25%.
The worst nuclear disaster resulted in $700 billion in damage. The most damaging AI attack is arguably the 2026 Binance hack, with ~$3-6 billion in damage. At the time of this attack, the strongest public model is GPT-6 Astra, rated at 169 ECI.
A doubling of METR's time horizon used to correspond to an increase of 5 ECI points. For simplified purposes, we'll assume this means an increase by 5 ECI points represent a 2x increase in danger. Obviously, this is heavily simplified and inaccurate (increase in available compute, emergent capabilities as agents become stronger (ESPECIALLY RSI)), but I'll temporarily use it for now.
Therefore, as 7 doublings are needed for Binance to match Chornobyl, 169 + 35 = 204. I added 1 to round up so it looks better.
Pros of this plan:
It won't substantially narrow, or even narrow at all, the US-China gap. YTD ECI growth of the US was 17 points (GPT-5.2 Pro at 157 to GPT-6.2 Astra at 174); YTD ECI growth of China was 15 points (DeepSeek-V3.2 at 146 to Kimi K3.1 Pro at 161). A naive calculation of slowing down progress by 25% would leave the US progressing at nearly 13 points (and likely dramatically undersell the growth advantage that American labs will enjoy via RSI and compute leads), which is nearly China's pace, leave China 5 years before catching up rather than 1, and is much less radical than a full-blown AI pause, therefore much more likely to get through in the US Congress (and also makes it a lot harder for US labs to defend than against a full pause: 25% is small enough of a demand).
It'll also set precedents for imposing similar regulations on Chinese open-source models (i.e., inference bans for unsafe open-source models), and we can use a variety of market mechanisms (for example, if Western open-source-focused labs "refine" Chinese models like Mistral and Thinking Machines by aligning them and then said made-safe models can be allowed to run inference) to pressure the Chinese into complying with safety requirements themselves and prevent Western "refiners" from eating their lunch, even on a surface level.
An early, concrete demand tied with numbers can produce things that tech companies should find more difficult to walk back on, with easier support to get through into actual legal action, producing important experience for future institutions.
With Hugging Face, it'd be easy to convince governments that risks can also come from powerful, internal agents, and thus allow for third-party agents to evaluate internal models' capabilities (and ideally failure modes) as well, preventing being blinded by surprise capabilities of internal models. (I'd say that it'd be very important data to be aware, at all times, where each lab is on the ECI scale.)
Governments will have precedents regulating AI companies and can more effectively execute such actions in the future.
We can pressure AI companies to make commitments before reaching 205 ECI, which we would then progressively increase linearly until it reaches 205 ECI. This will either force them to substantially increase their immediate safety spend, or have to commit to much larger safety spending in the future.
For example, if OpenAI committed to 1.75% of its compute to safety, at GPT-6.2 Astra, being at 174 ECI, then they'd have to increase their spend by 0.75% per ECI point and spend 55% at 235 ECI (on the eve of ASI, estimated at 241). Otherwise, if they committed to 9.5% instead, to reduce the increase by 0.5%/ECI point, that'd still represent a 4.5x increase in safety spending.
Most importantly, public responses can dramatically intensify if more advanced AI indeed produces as transformative impacts and damages as imagined; and with precedents, governments would find themselves much more ready to act upon public opinion. For example, if TED-AI arrived and rogue incidents proved especially damaging or threatening, the 25% threshold could easily increase to 55% instead, with 1% more for each new ECI point.
Cons of this plan:
Obviously, it's not enough. It's nowhere near enough (I agree that a <20% limit on compute for R&D is ideal), and should not be treated as anything but a temporary, less-than-a-stopgap initial proposal. Despite the many pros above, this one alone undermines a good chunk of its possible gains.
Providing opportunities for institutions is not building institutions, and AI companies will almost certainly try to manipulate the young institutions that might emerge from this (like audits, compute measurements, or multi-lab safety coordination mechanisms) to their favor.
More specifically, ECI itself might be subjected to being Goodharted or regulatory capture: they'd preserve most of their benefits as an evaluator for competing models to AI companies, while being much less obstructive, if they just make it so that future ECI points are 4/3th more expensive than previous ones. This is both expected and can be combated, in part or in full, via an increase in third-party inspectors' numbers, government intervention, increased funding for third-party inspectors, etc., but I don't claim that we can fully mitigate this. However, I do think that Goodhart's law or regulatory capture will happen nonetheless whenever we come upon them, so it's better that we do it earlier so that we can fix the mistakes.
Governments might come to believe that the solution to AI safety is to regulate them hard, especially if prosaic alignment produced very impressive surface results, and not permanently stopping ASI from being a thing under current regimes. Furthermore, they might miss subtle dangers, such as a highly competent schemer whose misalignment models might evade human understanding. From my POV, this is the status quo for governments anyways, and so there is little to be lost from at least giving them more exposure to and experience in the problem's nuances.
It might bury more radical (and much more necessary) proposals. Agreed. Even if I don't think current plans are gaining traction in the public at least to the extent that they can pass Congress in 2027, and that they might benefit from a preliminary effort like this establishing precedent (a hasty, inexperienced coordination right before ASI could be less than ideal, for example); I agree that it might lead to false complacency and less attention to much more important plans like AI Futures Model's.
Overall, I think there's much to be gained from getting something like this passed in 2027, as staging grounds for more advanced policies in the future (2028-2030 and onwards). I'd love to hear further feedback on what other cons and pros you think exist in this plan!
We need commitments for 25% compute for safety by 205 ECI points, progressively increasing with further capabilities.
(Or at least push for governmental pressure on companies to do so.)
Or, rather, we need specific, middle-ground commitments with concrete demands and conditions that are triggered. The example proposed is not random and also, obviously, not enough: asking for 25% compute for safety on the eve of TED-AI - estimated by AI Futures Model to be 210 - seems like a bad joke compared to the demands from compute-based plans like How to Pace the AI Frontier or Pacing AI via Compute Reallocation, which often asked for research compute to be limited to <20%. But it is something that, IMO, can happen and can set important institutional precedents that serve to prepare for a more sophisticated pause.
Why I chose 25% compute by 205 ECI:
Pros of this plan:
For example, if OpenAI committed to 1.75% of its compute to safety, at GPT-6.2 Astra, being at 174 ECI, then they'd have to increase their spend by 0.75% per ECI point and spend 55% at 235 ECI (on the eve of ASI, estimated at 241). Otherwise, if they committed to 9.5% instead, to reduce the increase by 0.5%/ECI point, that'd still represent a 4.5x increase in safety spending.
Cons of this plan:
Overall, I think there's much to be gained from getting something like this passed in 2027, as staging grounds for more advanced policies in the future (2028-2030 and onwards). I'd love to hear further feedback on what other cons and pros you think exist in this plan!