This is a special post for quick takes by TurnTrout. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Sometimes people object that pausing AI progress for e.g. 10 years would lead to a "compute overhang": At the end of the 10 years, compute will be cheaper and larger than at present-day. Accordingly, once AI progress is unpaused, labs will cheaply train models which are far larger and smarter than before the pause. We will not have had time to adapt to models of intermediate size and intelligence. Some people believe this is good reason to not pause AI progress.
There seem to be a range of relatively simple policy approaches which mitigate the "compute overhang" problems. For example, instead of unpausing all progress all at once, start off with a conservative compute cap[1] on new training runs, and then slowly raise the cap over time.[2] We get the benefits of a pause and also avoid the problems presented by the overhang.
EG "you can't use more compute than was used to train GPT-2." Conservatism helps account for algorithmic progress which people made in public or in private in the meantime.
This is only a simple policy approach in an extremely theoretical sense though, I'd say.
Like it assumes a perfect global compute cap with no exceptions, no nations managing to do anything in secret, and with the global enforcement agency being incorruptible and non-favoritist, and so on. You fail at any of these, and the situation could be worse than if no "pause" happened, even assuming the frame where a pause was important in the first place.
(Although, full disclosure, I do not share that frame).