How much slower does takeoff go with 10× less compute?

bhalstead

About 6x slower in the median case, with an 80% CI of 3.5x to 8x.

Setup

Define the "R&D compute" (in, say, H100-equivalents) of an AGI company at a given time to be the total compute in use across the following categories:

Compute used to run experiments, which are used to search for software improvements,
Compute used to run automated researcher agents,
Compute used in final training runs for frontier models.

With less R&D compute, the company would need to make cuts across these categories. For now, we'll assume they maintain the same allocation across the three areas, that is, we'll assume they achieve the reduction in the total via a reduction to each category.

The capability level of your best AI model is represented by its effective training compute, . is a stock, with units of "2025-FLOP". evolves according to

where is training system performance (a flow, units: FLOP/yr) and is software efficiency (units: 2025-FLOP per FLOP). We will call this quantity effective training system performance.

Training system performance and software efficiency tend to grow exponentially over time, so the "rate of AI progress" will be operationalized as the relative growth rate of (i.e. the time derivative of , which equals ). We will use the symbol for this.

Huh? Why this model?

Other models of AI takeoff (the AI Futures Model, Tom Davidson's FTM, Epoch AI's GATE) compose effective compute multiplicatively from two stocks:

where is cumulative training compute (a stock, units: FLOP, satisfying ). In other words, they apply today's software efficiency to every FLOP ever spent on training, "pulling outside the integral". Our formulation instead applies to the flow: , so . An algorithmic improvement raises the value of each FLOP spent from now on, but never retroactively upgrades FLOP spent under older algorithms.

A large, sudden compute reduction is precisely where the formulations come apart. Physically, the cut is a cut to flows: the rates at which FLOP are spent on experiments, agents, and training all drop by 10x. The multiplicative model has no training flow among its state variables, only the stock $C$, so the cut has to be translated into a statement about $C$. There are two natural translations, and neither is good:

Cut the stock: reduce $C$ by 10x. Then effective compute, i.e. capability, instantly drops by an order of magnitude. But nothing about a cut to the flow claws back FLOP already spent: the company keeps its best model's weights, can keep improving that model (with further RL, for example), and will use it to accelerate its remaining R&D. Treating a 10x cut in spending rates as a 10x cut in cumulative spending is like assuming the company must retrain from scratch on the smaller budget. This overestimates the slowdown.
Preserve the stock: let $C$ continue accumulating at the reduced flow ($C' = T/10$), or freeze it entirely. The stock bookkeeping is now fine; the problem is the multiplication. Since $C$ barely moves on the relevant timescale while $S$ keeps growing, capability keeps growing at roughly the rate of software progress, which is most of the overall rate, so to first order the cut slows progress by about half or less. But $E = C \cdot S$ applies each new algorithm to the entire accumulated stock, including all the FLOP spent before the algorithm existed. It's a bit like assuming that improvements discovered on the reduced budget magically upgrade training runs completed years earlier. Actually realizing an algorithmic improvement in a model requires spending FLOP under that algorithm, and the cut company can only spend them at a tenth the rate. This underestimates the slowdown.

The remaining simplification is treating AI progress as a single continuous training run that has been underway since the beginning of time. (Mathematically, the integral defining $E$ runs from minus infinity to the present.) In reality, frontier training runs sometimes start from scratch, have different lengths, and may only be underway during parts of the year. We think this still captures the dynamics relevant to compute reductions, but it directionally favors the compute-reduced project: some capabilities may require architectural changes or otherwise require starting from scratch, i.e. re-accumulating $E$ from zero, which is far more costly at a reduced flow, and this model assumes you never need to do this. In other words, if anything, a real 10x cut probably buys somewhat more slowdown than the model estimates.

Software improves via the semi-endogenous growth law

That is, the relative growth rate of software efficiency is research effort divided by a difficulty term that rises with the software level already achieved. parameterizes how much harder further improvements get as climbs (the "fishing out" of ideas). (Human and AI labor also enter research effort, but experiment compute carries almost all of the weight in the relevant regime, and the labor cut matters less and less as automation saturates.)

We are going to compare two quantities:

The amount of time elapsed between a pair of AI capability milestones, if compute and labor inputs were to freeze at the point the first milestone is reached
The amount of time elapsed between the same pair of milestones, if instead compute were reduced by a factor of 10 and frozen as the first milestone was reached.

Finding a mathematical expression for the slowdown factor

When we cut R&D compute by in this model, since assumed proportional reductions across all categories, the immediate effect is a reduction in , and therefore also a reduction in the rate of AI progress .

What happens after that? To figure this out, we can look at how changes over time. Differentiating,

This term equals the relative growth rate of , , or the growth rate of effective training system performance . We will call the growth rate of effective training system performance . The equation above then becomes

(This has the form of the "logistic differential equation", with carrying capacity .) Qualitatively, it says that when effective training system performance grows faster than capabilities (), then the growth rate of capabilities increases (). Conversely, when capabilities are growing faster than ETSP is, then capabilities growth will slow.

Hence, after falls by , the subtraction on the right hand side is positive, meaning that is increasing. That is, the rate of capabilities progress will initially fall by , then pop back up again, until effective compute is growing at the same rate as software efficiency. It will continue to increase so long as exceeds , gradually approaching the fixed point where and .

If training system performance (the raw hardware) remains constant after the reduction, then is just the growth rate of software efficiency, determined by experiment compute and AI labor, which today proceeds at perhaps 1 OOM per year. With less compute for experiments, and less compute to run automated researchers, the growth rate falls, but probably by less than . Say it falls by some factor with .

Now we have to determine how the rest of takeoff goes. Both trajectories have to traverse the same interval of capability, from the level at the cut to whatever level counts as the end of takeoff, and the time this takes is

In steady state , so if the slowed trajectory's is lower at every capability level (with the same N throughout), the integrand is larger everywhere and the total time is longer. So we want to show that the ratio of the two trajectories' software growth rates at matched capability is constant, and to find it.

Dividing the default trajectory's software law of motion by that of the reduced-compute trajectory yields

where we are making the approximation that research effort exhibits constant elasticity to experiment compute over the relevant range.^[1] Intuitively, the factor is the direct slowdown at that capability level, and the second factor adjusts for the increased "fishing out" effect associated with the greater software content of the trajectory with reduced compute.

Now we determine his determines how much more software the reduced-compute trajectory holds. At the fixed point , we have . Taking logs,

We can write this identity once for each trajectory at the same , and subtract:

Rearranging yields

Hence the software ratio (at steady state) is indeed independent of capability level. By substituting, finally we can solve for :

This makes sense in the extreme cases:

with infinite , i.e. with extremely low returns to software R&D, the slowdown is the full 10×
with near zero, i.e. with high returns to software R&D, the slowdown is just research effort reduction .

Estimating using the AI Futures Model

After modifying the AI Futures Model to use the continuous training run ODE described above, I simulated the effect of compute reductions while sampling from our parameter distributions. The various approximations involved in the derivation held relatively well:

The slowdown factor does not depend greatly on the specific capability level at which the reduction occurs. Here, with the relevant parameters held fixed at our median estimates, you can see that N ranges from 5.2 to around 6:

There is significant uncertainty in our estimates of N, mostly driven by our uncertainty about the current rate of software progress. The main other important parameter determining N in the AI Futures Model is the current-day software progress elasticity to experiment compute, which we estimated by polling AI researchers.

^{^}
In the AI Futures Model this is approximately true, since during takeoff, research effort is highly bottlenecked on experiment compute.

41

How much slower does takeoff go with 10× less compute?

41

Setup

Finding a mathematical expression for the slowdown factor

Estimating using the AI Futures Model

41

41