After @Daniel Kokotajlo invited me to the AI Futures office I ended up talking to Eli and Alex for about an hour, and feel like I have a decent understanding of the model:

Summary of the AI Futures Model

• Compute and effective compute
  • Actual compute $C\left(t\right)$ is stock of compute at time t
  • Effective compute $E\left(t\right):=C\left(t\right)\cdot \text{software efficiency}$ is used as the main measure of AI capabilities. It is defined as the "amount of training compute we’d need to train models as performant as the frontier models at time $t$ using the training process of the present-day".
  • Compute is allocated as fixed percentages between training, experiments, and automated coders
• Effective labor
  • The % of tasks automatable is a logistic function of log effective compute $E\left(t\right)$
  • Once a task can be automated, it will still get more efficient over time by a multiplier ${\eta }_i\left(t\right)$
    • ${\eta }_i\left(t\right)$ is zero for non-automated tasks. When effective compute reaches the level $E_i$ required to automate it, it increases as a power law ${\eta }_{\text{init}}\cdot {\left(\frac{E\left(t\right)}{E_i}\right)}^{{\eta }_{\text{slope}}}$.
  • Human coding labor $L_{C,H}\left(t\right)$ and automation compute $C_{aut,i}$ are optimally allocated between tasks
  • Overall coding labor for task i is the sum of human and AI labor $G_i=L_{C,H,i}+{\eta }_i\cdot C_{aut,i}$
  • Aggregate coding labor $L_C\left(t\right)$ is CES between the labor applied to all different tasks, with low substitutability ${\rho }_c=-2$ by default, meaning tasks only substitute slightly for each other
    • $L_c\left(t\right)={\left({\int }_0^1{G}_i\right(t{)}^{{\rho }_c}di)}^{1/{\rho }_c}={\left({\int }_0^1{\left(L_{c,H,i}\right(t)+{\eta }_i(t)\cdot C_{aut,i}(t\left)\right)}^{{\rho }_c}di\right)}^{1/{\rho }_c}.$
  • Finally, serial coding labor ${\tilde{L}}_C\left(t\right)=L_C(t{)}^{\lambda }$, indicating diminishing returns of about $\lambda =0.5$ to adding more labor in parallel
• “Experiment throughput” $X\left(t\right)$ is CES between serial coding labor and experiment compute
  • $X\left(t\right)={\left(\alpha {\tilde{C}}_{xpm}\right(t{)}^{{\rho }_x}+(1-\alpha ){\tilde{L}}_C\left(t{)}^{{\rho }_x}\right)}^{1/{\rho }_x},0<\alpha <1,{\rho }_x<0$
  • Labor and compute are slight complements (median estimate ${\rho }_x=-0.155$)
  • There are also diminishing returns to compute, with ${\tilde{C}}_{\text{xpm}}=C_{\text{xpm}}^{\zeta }$ where $\zeta =0.65$ by default
• Research taste $T\left(t\right)$
  • Human research taste is lognormally distributed with median researchers defined as 1x taste and 99.9th percentile (+3.1SD) researchers assumed to have 3.70x research taste
  • An Automated Coder–level AI has research taste $T_{AC}$
  • AI research taste increases as a power law in effective compute (AI “research taste IQ” is $T_{rate}\cdot \log E\left(t\right)+const$ standard deviations above the human median, which is then passed through an exponential to get research taste)
  • AIs replace whatever humans they’re better than. The aggregate research taste of the company is the mean of all remaining researchers. This means it initially increases slowly as AIs replace the worst researchers, then speeds up as everyone starts using the AIs’ research taste which keeps improving.
• Research effort RE(t) = research taste * experiment throughput
• Then software efficiency $S\left(t\right)$ follows the Jones model $\dot{S}=S^{1-\beta }RE\left(t\right)$
  • $\beta$ is how much harder AI R&D gets as software efficiency advances
• Finally this feeds back into effective compute $E\left(t\right)=C_{train}\left(t\right)S\left(t\right)$
  • A taste-only singularity happens when $m>\beta$, where $m$ = doublings of research taste per doubling in effective compute. This would cause improvements to go faster and faster until approaching physical limits. Eli's parameter choices give 38% chance of taste-only singularity, but many of the non-singularity samples still get to ASI quickly, with the 50th percentile sample getting from AC to ASI in 5 years.
  • For various reasons Eli and Daniel's all-things-considered views have harder takeoff than the model predicts, with Eli's median for AC -> ASI 2 years, and Daniel's median 1.5 years.

Notes on Sensitivity analysis

• Time to AC is very sensitive to how superexponential time horizon growth is, and also to
  • The present doubling time
  • Time horizon for automated coder
• Time from AC to ASI is very sensitive to the “automated research taste slope” $T_{rate}$: how much “research IQ” AIs gain per doubling of effective training compute. But many other factors could slow down the AC-to-ASI duration to >6.5 years:
  • Median-to-top-human jumps above SAR needed to reach TED-AI
  • The software efficiency growth rate in 2024
  • Median to 99.9th% human research taste multiplier
  • Slowdown from 10x less experiment compute
  • Research progress rate in the limit of infinite coding labor: mostly because it’s highly uncertain (their 90% CI is 2.0-201)
  • Automated research taste of an AC

Biggest uncertainties to track

(not necessarily that I disagree, just need to think about it more)

• Effective compute vs time horizon: how do all the assumptions look when we eliminate time horizon from the model and use other methods to model effective compute growth? I’m sketched out by the huge error bars on time horizon superexpontiality → time to AC
• Ryan thinks >70% of code at Anthropic was written by AIs already in October 2025 but it’s mostly low-value code. Code varies dramatically in value, and AIs can expand the number and type of low-value tasks done rather than just substituting for humans. This may be a separate effect from AIs doing extra work on tasks that can be automated, which is not tracked by the model.
  • It might be that coding ability and research taste are two ends of a continuous spectrum from small-scale to large-scale tasks.
• Research taste:
  • Someone really needs to do experiments on this, it’s possible now. David Rein and I are actively thinking about it
  • Is human research taste modeled correctly? Eg it seems likely to me that the 0.3% of top humans add more than 0.3%*3.7x to the “aggregate research taste” of a lab because they can set research directions. There are maybe more faithful ways to model it; all the ones Eli mentioned seemed far more complicated.
  • Is modeling AI research taste as exponential in human standard deviations valid? I have no idea whether someone 9 standard deviations above the human median would be able to find 3.7^(9/3) = 50x better research ideas or not
• Is CES valid for experiment throughput at these extreme values of labor and compute? It seems like a superhuman AI researcher might learn to run experiments more efficiently, decreasing the compute required for each experiment. The estimates for experiment throughput parameters were all about humans getting 10x compute, infinite labor, etc. Or, they could coordinate better (especially with all the human ex-coders to help them), and decrease the parallelization penalties for labor and/or compute. I’m not sure if this would be different from adjusting research taste.

Thoughts in no particular order:

• Kudos for what seems to be lots of thoughtful work incorporated into this model.
• There are a lot of parameters. Maybe this is necessary but it's a bit overwhelming and requires me to trust whoever estimated the parameters, as well as the modeling choices.
• I couldn't find a block of equations that represents the whole model, or an index of variables in one place, and it's difficult to go between math and exposition especially when the equations are hidden in dropdowns, so I still feel like I don't have a good picture. I had Claude do this and read through it, and it looks reasonable but some parts are still not defined in Claude's summary, I think because the whole page is rendered in javascript and it couldn't access it. I would love to visit the AI Futures office again to understand the model better.
• I find the use of time horizon as such a crucial intermediate variable confusing and am scared of potential assumptions around it.
  • Time horizon is underdefined on years long tasks. I know I talked to the AI Futures team about what you now wrote up as METR-HRS-Extended to operationalize it, but it's unclear what a 3-year time horizon really means (when extrapolating the trend with superexponential adjustment) given factors like the increased number of details and interaction with longer tasks. Does the trend mean that in X years, an AI will competently substitute for a human for a 3-year long project with the same level of feedback from a manager, or be able to do the human's job with less feedback?
  • The function that relates time horizon and research speedup in the real world is very unclear. I'm trying to collect data on this and it's highly nontrivial to model and interpret^[1] so I'm skeptical of any use of the time horizon trend to predict uplift that doesn't either have a simple, robust model or something validated by experiment.
  • The description implies that time horizon is only used in the first phase (pre-AC), but I don't understand how it's used. Humans and AIs will probably go from their current state to perfectly substitutable in some kind of continuous progression and I couldn't find the formula for this. Also when I changed the "How much easier/harder each coding time horizon doubling gets" parameter by small amounts, the forecasted time from AC to ASI changes significantly (2.7 years at 0.90, over 4 years for 1.00), so it looks like stages 2 and 3 are affected as well.
  • It seems to me like a time horizon of 3 years or 125 years is complete overkill for automation of enough coding for the bottleneck to shift to experiments and research taste.
• Why not assume that compute is allocated optimally between experiment, inference, etc. rather than assuming things about the behavior of AI companies?
• I wish the interface were faster to update, closer to 100ms than 1s to update, but this isn't a big deal. I can believe it's hard to speed up code that integrates these differential equations many times per user interaction.

1. ^{^}

   Eg looking at transcripts to determine where humans are spending their time when they give Cursor tasks of a certain length

I didn't really define software intelligence explosion, but had something in mind like "self-reinforcing gains from automated research causing capabilities gains in 6 months to be faster than the labor/compute scaleup-driven gains in the 3 years from 2023-2025", and then question I was targeting with the second part was "After the initial speed-up from ASARA, does the pace of progress accelerate or decelerate as AI progress feeds back on itself?"

A 23.5x improvement alone seems like it would qualify as a major explosion if it happened in a short enough period in time

Seems about true. I claim that the nanogpt speedrun suggests this is only likely if future AI labor is exponentially faster at doing research than current humans, with many caveats of course, and I don't really have an opinion on that.

We already know that there is of course a fundamental limit to how fast you can make an algorithm, so the question is always "how close to optimal are current algorithms". It should be our very strong prior that any small subset of frontier model training will hit diminishing returns much quicker than the complete whole.

This is not as small a subset of training as you might think. The 53 optimizations in the nanogpt speedrun touched basically every part of the model, including the optimizer, embeddings, attention, other architectural details, quantization, hyperparameters, code optimizations, and Pytorch version. The main two things that limit a comparison to frontier AI are scale and data improvement. It's known there are many tricks that work at large scale but not at small scale. If you believe the initial 15x speedup is analogous and that the larger scale gives you a faster, then maybe we get something like a 100x speedup atop our current algorithms? But I don't really believe that the original nanoGPT, which was a 300-line repo written to be readable rather than efficient [1], is analogous to our current state. If there were a bunch of low-hanging fruit that could give strongly superlinear returns, we would see 3x/year efficiency gains with small increases in labor or compute over time, but we actually require 5x/year compute increase and ~3x per year labor increase.

A software intelligence explosion is completely possible with linear speedups in cumulative effort. Indeed, it is possible with sublinear increases in cumulative effort.

Agree I was being a bit sloppy here. The derivative being infinite is not relevant in Davidson's model or my mind, it's whether the pace of progress accelerates or decelerates. It could still be very fast as it decelerates, but I'm not really thinking in enough detail to model these borderline cases, so maybe we should think of the threshold for very fast software-driven progress as r > 0.75 or something rather than r > 1.

[1]: "In addition, llm.c still has a lot of pending optimizations and people haven't tried to tune the training in the style of cramming, so I'd say we're likely to see significant improvements on this number." 

Diminishing returns in the NanoGPT speedrun:

To determine whether we're heading for a software intelligence explosion, one key variable is how much harder algorithmic improvement gets over time. Luckily someone made the NanoGPT speedrun, a repo where people try to minimize the amount of time on 8x H100s required to train GPT-2 124M down to 3.28 loss. The record has improved from 45 minutes in mid-2024 down to 1.92 minutes today, a 23.5x speedup. This does not give the whole picture-- the bulk of my uncertainty is in other variables-- but given this is existing data it's worth looking at.

I only spent a couple of hours looking at the data [3], but there seem to be sharply diminishing marginal returns, which is some evidence against a software-only singularity. 

At first improvements were easy to make without increasing lines of code much, but then improvements became small and LoC required became larger and larger with increasingly small improvements, which means very strong diminishing returns-- speedup is actually sublinear in lines of code. This could be an artifact related to the very large elbow early on, but I mostly believe it.

If we instead look at number of stars as a proxy for amount of attention on the project [4], there are no diminishing returns. The data basically suggest speedup is linear in effort [1], which is consistent with a world where 3x/year increases in labor and compute are required to sustain the historical trend of ~3x/year algorithmic speedups observed by Epoch. However, this still points against a software intelligence explosion, which would require superlinear speedups for linear increases in cumulative effort.

Given that the speedup-vs-stars and speedup-vs-improvement-# graphs are linear but speedup-vs-LoC is sublinear, our guess should be that returns to research output are somewhat sublinear. In the language of Davidson's semi-endogenous growth model, this means $\beta >1$ [2]. Of course there are massive caveats about extrapolation to future models.

In Davidson's model, the requirement for a software intelligence explosion after research is automated is $r:=\frac{\lambda \alpha }{\beta }>1$, where $\lambda <1$ represents inefficiency of parallel work and $\alpha <1$ is the elasticity of research output to cognitive labor at a fixed compute budget. If $\beta >1$, this mathematically means $r<1$ and we don't get an SIE.

So I think an SIE will only happen if one or more of the below is true:

• Cognitive quality of future AIs being so high that one hour of AIs researching is equivalent to exponentially large quantities of human researcher hours, even if they don't train much more efficiently to the same capability level. This is the most important question to answer and something I hope METR does experiments on in Q1.
• Some other difference between nanogpt and frontier AI research, e.g. maybe research ability is easier to improve than base model loss
• Once we get AGI, something other than human labor, AI labor, or compute scales exponentially on a much faster timescale, e.g. training data
• A paradigm shift to a totally different architecture that wouldn't be captured in a dataset only 1.5 years long
• AIs with 2x the efficiency cause research to happen more than 2x faster, despite outweigh the diminishing returns from parallel work AND being bottlenecked by compute, making $\lambda \alpha >1$. I can't think of a way this could happen, and to get an SIE it would have to be even larger than 1.

[1]: This was previously observed in a tweet from Epoch in February but now we have about twice the data.

[2]: $\beta =0$ would mean exponential improvements, while $\beta =1$ implies linear improvement over time at constant labor/compute. So $\beta >1$ means improvements are actually slower than linear.

[3]: A few minutes ideating, almost an hour writing a prompt for Claude 4.5 Opus, then 30 minutes making graphs and such.

[4]: It's unclear whether to say that stars represent instantaneous effort or total cumulative effort on the project. If we interpret it as instantaneous effort, then we would see diminishing returns. Also it's unclear whether stars are measuring $\lambda \beta$ or $\beta$; if $\lambda \beta$ it might imply slightly increasing returns.