We’d like to know how much limits on compute scaling will constrain AI R&D. This post doesn’t have answers, but it does attempt to clarify thinking about how to use economic models to explore the question.
The Standard Model of Idea Production
A general “Jones-style” model of idea production[1] is
,
is total stock of ideas which have been found within a field
is capital (for our purposes, this is physical compute)
is researcher hours
The Greek letter parameters are assumed to be constant and nonnegative, and they describe how idea production responds to changes in , , and .
You may want to refer to this image multiple times while you read
We’re interested in applying this model once physical compute scaling is done, which makes a constant.[2]
The assumption that the exponent on is at most one () is downstream from the idea that over time, as all the low-hanging fruit is picked, new ideas in a field become harder to find. The empirical data say that the so-called ‘standing on shoulders’[3] effect is actually negative ().[4]
As for researcher hours , in standard fields this isn’t boosted by new discoveries like it is for AI R&D. The exponent is estimated to be somewhere around .[5]
We cannot be very confident in these numerical estimates, as there is no clean way to measure things like ‘idea stock’ and so econometrics resorts to a variety of clever indirect methods which wind up generating lots of different estimates without a lot of consistency.
The Automation Feedback Loop
AI has the potential to automate AI R&D. This creates a unique field to model, as in most fields increasing the idea stock doesn’t automatically make the researchers more effective.
AI researchers can also try to increase effective compute by developing more efficient algorithms.
So our updated model looks something like this:
Here we multiply capital and researcher hours by terms representing how much they are effectively increased by more idea stock.[6]
Here’s an updated reference image — we’ll get to δ in a bit
Separating and from clarifies the meaning of each parameter and maintains the convention that these numbers are positive. Here, measures how quickly finding new ideas gets more difficult, measures how much the idea stock increases effective compute for constant physical compute, and measures how much the idea stock increases AI researcher effectiveness.
If researchers are all AIs running on compute and compute is constant, we can simplify by setting as well (although there will be direct substitution happening where the number of artificial researchers of a certain capability level can be traded for more compute and vice versa, this will reach some kind of equilibrium).
So our simplified equation becomes this:
... where we’ve collapsed all our constants into .
Well, now it’s easy to see that the question “Will there be a Software Intelligence Explosion” is roughly equivalent to asking if will be true.[7]
While the model usually assumes parameters are constant, we can expect to drop as we approach the theoretical limit of our physical compute. However, if current AI systems are far from the theoretical limits of intelligence per unit of compute, it’s possible that if is high it could stay that way for a while.[8]
is probably the most important unknown here. It’s not yet possible to measure this and we don’t have any historical analogues. It’s here that Erdil and Besiroglu have intuitions which differ most strongly from those of the AI Futures Project, for example.
Constant Elasticity of Substitution vs the Jones-style model
Previous discussions of compute bottlenecks by Epoch, Davidson, and Whitfill & Wu have framed this question using a CES (Constant Elasticity of Substitution) production function:
Here is experimental compute, is cognitive labor, and controls whether they are substitutes (goods that can replace each other in consumption) or complements (goods that are consumed together). The debate then turns on the value of , generally agreed to be negative, with smaller magnitudes meaning faster takeoffs: Epoch argues , Davidson argues is between and , and Whitfill & Wu get wildly different empirical estimates depending on whether “compute” means total research compute (, positive![9]) or frontier experiment scale ().
We can and should think carefully about how to interpret this empirical instability of , but it’s also worth looking beyond CES.[10]
The question of whether there will be a software intelligence explosion is a question about whether a dynamic feedback loop causes accelerating growth over time. But CES is static: it describes how contemporaneous inputs combine into contemporaneous output within a given period.
Even if Epoch is right that and you cannot speed up R&D much by throwing more cognitive labor at fixed compute in a single period, that doesn’t settle the question of whether the accumulating stock of ideas (i.e. algorithmic improvements) make your fixed inputs more productive across time.
This means we need Jones-style ODE like above. CES and Jones are not alternatives to each other: You could put a CES inside a Jones-style model as the within-period aggregator of and . CES by itself doesn't represent the feedback from accumulated ideas into future productivity, which is where the explosion question lives.
My Takeaways
It’s important to think not just about whether AI can autonomously increase the effectiveness of AI researchers, but also about how large this increase will be ().
In traditional models of idea production, the exponent on is much lower than we should expect it to be in AI R&D. The feedback loops involved mean that traditional assumptions won’t necessarily hold.
If a software intelligence explosion does happen we’ll be seeing some historically large production model parameters.
This model is also applicable as compute continues to scale. However, applying it in that context would require modeling a separate production function for compute. There’s no closed-form solution for this case, and it’s not a crux for determining ASI timelines in the same way as the constant-compute case.
Notably, I am modeling feedback from ideas into the productivity of compute and labor, but not the reverse channels, like how more compute might enable qualitatively new experiments that open up new idea-space.
We’d like to know how much limits on compute scaling will constrain AI R&D. This post doesn’t have answers, but it does attempt to clarify thinking about how to use economic models to explore the question.
The Standard Model of Idea Production
A general “Jones-style” model of idea production[1] is
The Greek letter parameters are assumed to be constant and nonnegative, and they describe how idea production responds to changes in
We’re interested in applying this model once physical compute scaling is done, which makes
The assumption that the exponent on
As for researcher hours
We cannot be very confident in these numerical estimates, as there is no clean way to measure things like ‘idea stock’ and so econometrics resorts to a variety of clever indirect methods which wind up generating lots of different estimates without a lot of consistency.
The Automation Feedback Loop
AI has the potential to automate AI R&D. This creates a unique field to model, as in most fields increasing the idea stock
AI researchers can also try to increase effective compute by developing more efficient algorithms.
So our updated model looks something like this:
Here we multiply capital and researcher hours by terms representing how much they are effectively increased by more idea stock.[6]
Separating
If researchers are all AIs running on compute and compute is constant, we can simplify by setting
So our simplified equation becomes this:
... where we’ve collapsed all our constants into
Well, now it’s easy to see that the question “Will there be a Software Intelligence Explosion” is roughly equivalent to asking if
While the model usually assumes parameters are constant, we can expect
Constant Elasticity of Substitution vs the Jones-style model
Previous discussions of compute bottlenecks by Epoch, Davidson, and Whitfill & Wu have framed this question using a CES (Constant Elasticity of Substitution) production function:
Here
We can and should think carefully about how to interpret this empirical instability of
The question of whether there will be a software intelligence explosion is a question about whether a dynamic feedback loop causes accelerating growth over time. But CES is static: it describes how contemporaneous inputs combine into contemporaneous output within a given period.
Even if Epoch is right that
This means we need Jones-style ODE like above. CES and Jones are not alternatives to each other: You could put a CES inside a Jones-style model as the within-period aggregator of
My Takeaways
Thanks to economist Peter Pedroni for a long conversation which clarified my understanding of this. Thanks to @JustisMills, @apolloandersen, and @Tom Davidson for comments on drafts of this post.
This model is due to Charles “Chad” Irving Jones. A simplified version of this which ignores capital is called the Jones Model.
This model is also applicable as compute continues to scale. However, applying it in that context would require modeling a separate production function for compute. There’s no closed-form solution for this case, and it’s not a crux for determining ASI timelines in the same way as the constant-compute case.
A term coined by Jones; see page 1071 of this book chapter for his explanation of the term and the contrasting “fishing out” effect.
See Ma & Samaniego (2020). Macroeconomic shocks and productivity: Evidence from an estimated Ideas’ production function, page 4.
See the appendix of Are Ideas Getting Harder to Find?
Notably, I am modeling feedback from ideas into the productivity of compute and labor, but not the reverse channels, like how more compute might enable qualitatively new experiments that open up new idea-space.
Again:
It seems like it’s currently high! See Epoch’s “Algorithmic progress in language models”
Positive
Peter Pedroni: “I agree ... completely that the CES production function is not appropriate.”