Here's an attempt to recap the previous discussion about "Global Shutdown" vs "Plan A/Controlled Takeoff", trying to skip ahead to the part where we're moving the conversation forward rather than rehashing stuff.

Cruxes that seemed particularly significant (phrased the way they made most sense to me, which is hopefully reasonably ITT passing)

...

How bad is Chinese Superintelligence? For some people, it's a serious crux whether a China-run superintelligence would be dramatically worse in outcome than a democratic country. 

...

"The gameboard could change in all kinds of bad ways over 30 years." Nations or companies could suddenly pull out in a disastrous way. If things go down in the near future there's fewer actors to make deals with and it's easier to plan things out.

...

Can we leverage useful work out of significantly-more-powerful-but-nonsuperhuman AIs? Especially since "the gameboard might change a lot", it's useful to get lots of safety research done quickly, and it's easier to do that with more powerful AIs. So, it's useful to continue to scale up until we've got the most powerful AIs can we can confidently control. (Whereas Controlled Takeoff skeptics tend to think AI that is capable of taking on the hard parts of AI safety research will already be too dangerous and untrustworthy)

...

Is there a decent chance an AI takeover is relatively nice? Giving the humans the Earth/solar system is just incredibly cheap from percentage-of-resources standpoint. This does require the AI to genuinely care about and respect our agency in a sort of complete way. But, it only has to care about us as a pretty teeny amount

[Edit: this was an interesting disagreement but I don't know anyone for whom it's strategically relevant, except in what arguments to publicize about whether if anyone built it, everyone would die]

...

And then, the usual  "how doomed are current alignment plans?". My impression is "Plan A" advocates are usually expecting a pretty good chance things go pretty well if humanity is making like a reasonably good faith attempt at controlled takeoff, whereas Controlled Takeoff skeptics are typically imagining "by default this just goes really poorly, you can tell because everyone seems to keep sliding off understanding or caring about the hard parts of the problem")

...

All of those seem like reasonable things for smart, thoughtful people to disagree on. I do think some disagreement about them feels fishy/sus to me, and I have my takes on them, but, I can see where you're coming from.

Three cruxes I still just don't really buy as decision-relevant:

1. "We wouldn't want to pause 30 years, and then do a takeoff very quickly – it's probably better to do a smoother takeoff." Yep, I agree. But, if you're in a position to decide-on-purpose how smooth your takeoff is, you can still just do the slower one later. (Modulo "the gameboard could change in 30 years", which makes more sense to me as a crux). I don't see this as really arguing at all against what I imagined the Treaty to be about.
    
2. "We need some kind of exit plan, the MIRI Treaty doesn't have one." I currently don't really buy that Plan A has more of one than the the MIRI Treaty. The MIRI treaty establishes an international governing body that makes decisions about how to change the regulations, and it's pretty straightforward for such an org to make judgment calls once people have started producing credible safety cases. I think imagining anything more specific than this feels pretty fake to me – that's a decision that makes more sense to punt to people who are more informed than us.
    
3. Shutdown is more politically intractable than Controlled Takeoff. I don't currently buy that this is true in practice. I don't think anyone is expecting to immediately jump to either a full-fledged version of Plan A, or a Global Shutdown. Obviously, for the near future, you try for whatever level of national and international cooperation you can get, build momentum, do the easy sells first, etc. I don't expect, in practice, Shutdown to be different from "you did all of Plan A, and, then, took like 2-3 more steps, and by the time you've implemented Plan A in it's entirety, it seems crazy to me to assume the next 2-3 steps are particularly intractable."
   1. I totally buy "we won't even get to a fully fledged version of Plan A", but, that's not an argument for Plan A over Shutdown.
   2. It feels like people are imagining "naive, poorly politically executed version of Shutdown, vs some savvily executed version of Plan A." I think there are reasonable reasons to think the people advocating Shutdown will not be savvy. But, those reasons don't extend to "insofar as you thought you could savvily advocate for Plan A, you shouldn't be setting your sites on Shutdown."

Thanks for writing this paper. 

Why do we need to halt for so long? In short, AI alignment is probably a difficult technical problem, and it is hard to be confident about solutions. Pausing for a substantial period gives humanity time to be careful in this domain rather than rushing. Pausing for a shorter amount of time (e.g., 5 years) might reduce risk substantially compared to the current race, but it also might not be enough. In general, world leaders should weigh the likelihood and consequence of different risks and benefits against each other for different lengths of a pause. Section 2 discusses some of the reasons why the AI alignment problem may be difficult. Generally, experts vary in their estimates of the difficulty of this problem and the likelihood of catastrophe, with some expecting the problem to be very hard [Grace et al., 2025, ControlAI, 2025, Wikipedia, 2025]. Given this uncertainty about how difficult this problem is, we should prepare to pause for a long time, 8 in case more effort is needed. Our agreement would allow for a long halt, even if world leaders later came to believe a shorter one was acceptable. We also contend that there are other problems which need to be addressed during a halt even if one presumes that alignment can be quickly solved, and these problems are also of an uncertain difficulty. These include risks of power concentration, human misuse of AIs, mass-unemployment, and many more. World leaders will likely want at least years to understand and address these problems. The international agreement proposed in this paper is primarily motivated by risks from AI misalignment, but there are numerous other risks that it would also help reduce.

I agree with a lot of this, but I do think this paper ambiguates a bit between "we need to halt for decades" and "we might need to halt for decades". I agree with the latter but not the former,. 

I also think that in the cases where alignment is solvable sooner, then it might matter a lot that we accelerated alignment in the meantime. 

I get that it's scary to have to try to bifurcate alignment and capabilities progress because governments are bad at stuff, but I think it's a mistake to ban AI research, because it will have very negative consequences on the rate of AI alignment research. I think that we should try hard to figure out what can be done safely (e.g. via things like control evals), and then do alignment work on models that we can empirically study that are as capable as possible while incurring minimal risks. 

Serial time isn't the only input that matters: having smarter AIs is helpful as research assistants and to do experiments directly on the smarter AIs, having lots of compute to do alignment experiments is nice, having lots of money and talent going into AI alignment is helpful. I think you guys should emphasize and think about the function you are trying to maximize more clearly (i.e. how much do you really care about marginal serial time vs marginal serial time with smart AIs to do experiments on). 

Nice. I've only read the quoted paragraphs from the main body:

Despite prohibitions on large-scale training, AIs could continue getting more capable via improvements to the algorithms and software used to train them. Therefore, the coalition adopts appropriate restrictions on research that contributes to frontier AI development or research that could endanger the verification methods in the agreement. These restrictions would cover certain machine learning-related research and may eventually expand to include other AI paradigms if those paradigms seem likely to lead to ASI. Coalition members draw on their experience restricting research in other dangerous fields such as nuclear and chemical weapons. These research restrictions aim to be as narrow as possible, preventing the creation of more capable general AI models while still allowing safe and beneficial narrow AI models to be created. The coalition encourages and makes explicit carve-outs for safe, application-specific AI activities, such as self-driving cars and other uses that provide benefits to society.

Verification that members are adhering to research restrictions is aided by the fact that relatively few people have the skills to contribute to this research—likely only thousands or tens of thousands, see Appendix A, Article IX. Nations verify compliance by way of intelligence gathering, interviews with the researchers, whistleblower programs, and more. This verification aims to use non-invasive measures to ensure that researchers are not working on restricted topics. Additionally, inspectors verify that the medium-scale training allowed by the agreement uses only approved methods (e.g., from a Whitelist) and does not make use of any novel AI methods or algorithms; these would be evidence that restricted research had taken place.

And then skimmed the appendix "ARTICLE VIII — Restricted Research: AI Algorithms and Hardware". I'm very happy to see any kind of proposal that includes a serious attempt to ban AGI research. I want to also mention "The Problem with Defining an "AGI Ban" by Outcome (a lawyer's take).".

My low-effort attempted very short summary of this aspect of the treaty is:

Part of the treaty sets up a committee that decides what counts as AGI research. That stuff is banned.

Is that basically right? (And there's more discussion of how you'd enforce that, and a few examples of non-AGI research are given, and some comparisons with nuclear bombs and Asilomar are given.)

I don't have a better idea, so this seems as promising as I can think of. I'm curious though for ideas about

• Some sort of category that could actually work here.
  • You need a category that's politically viable and also captures most or all of AGI research.
  • It's ok but sad and also maybe politically more difficult to exclude a bunch of stuff that isn't AGI research.
  • For example (some bad ideas to spur better ideas from others):
    • You could say "you have to declare a specific task you're trying to solve, like protein interactions or self-driving cars; the task can't be super broad / heterogeneous, like predicting text".
    • You could say "no running searches over very algorithmically rich spaces, or strongly enriching or speeding up existing searches".
    • You could say "your system has to demonstrate by lesion experiments that it requires domain-specific knowledge".
    • You could say "if other researchers can copy-paste your system into another domain, and it works well, it's banned".
    • You could say "if our red team is able to use your system to do X and Y and Z, then it's banned".
    • You could say "you have to state reasons why your research is interesting/promising, and those reasons have to sound like the promisingness comes from something domain-specific".
  • It might be possible to targetedly make some boundaries more coordinatable-on. Cf. https://tsvibt.blogspot.com/2025/11/constructing-and-coordinating-around.html
• And/or, some way of having really good governance.
  • It seems like if the committee goes awry, it's very easy to pretend you've drawn a good boundary, but you haven't.
  • It seems likely for the committee to go awry, given how much pressure there could be for less restriction.
  • Curious for any thoughts on how to do that, e.g. examples of governance with significant responsibility to develop unclear policies that went well despite strong pressures.

Coalition members would each consolidate AI chips into a smaller number of declared data centers, where their usage can be declared and monitored for assurance that they are only being used for allowed activities. The number of chips permitted in any one unmonitored facility would be strictly limited. (We suggest the equivalent of 16 H100 chips, a conglomeration that would cost approximately $500,000 USD in 2025).

This seems a bit too low to me. According to epoch AI, the largest training runs currently take 5e26 FLOP, and so at 50% utilization this cluster would take^[1] ~ 5e26/(0.5*16*1e15) s = 6.25e10 s = 1981 years, far longer than the expected lifetime of the ban. Since current training runs have not yet caused a catastrophe, we should allow runs with less effective compute. For example, if want it to take ~50 years to reach frontier LLMs, and conservatively assuming a 10x algorithmic improvement over existing frontier methods, we should allow ~64 H100 GPU clusters. 

1. ^{^}

   Not to mention that frontier training runs would be impossible on this cluster due to memory constraints. 

Note: you can get a nice-to-read version of the Treaty on https://www.ifanyonebuildsit.com/treaty I'm not sure if there's any notable differences between that and the paper but I'm guessing it's mostly the same.

The content in article XIII to XV is absolutely crucial to get roughly right. 

From game theory perspective clear sanctions are extremely important. 

I think even more explicity and specifics would be good there. But I could guess that the terms need to be heavily negotiated to pass.

// Regarding the whole core idea that signatories can regulate non-signatories, this seems absolutely wild politically. If the idea centers around a power coalition, then the withdrawal terms seem "undercooked" and the whole idea seems unrealistic to pass in the current era.

I mean, playing devil's advocat here, what superpower would sign on to shooting itself in the foot like this? Enabling an entity to which it yields its own power, which if successful, from which they can never take the power back? 

USSR didn't even agree to US giving up its nukes to UN council back post-WW2. And current US seem very self-centered as well.

Do you have some estimation of the chances of the withdrawal terms passing?