The recent METR report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027.

I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind.

I haven't seen an unredacted version of this report; this is purely about the public report. Because so much of the underlying evidence was redacted from the public report, I think conclusions about things other than the observed 2x faster rate of progress on benchmarks depend significantly on trust for the authors of the report. (E.g., conclusions about whether this speed up is from RSI depend on trust.) I do trust the authors of the report a lot to be careful and rigorous and generally do a good job. (Note that I also work at METR.) Less extensive redactions would greatly reduce this need for trust.

Since in my current role I may have access to other confidential information that could be relevant to timelines one way or another, I unfortunately cannot state my all-things-considered views on ASI timelines or the probability of a software intelligence explosion right now. Here, I’ve done some analysis of the implications using my previous public statements.

A 2x acceleration in overall AI progress from RSI corresponds to a much larger than 2x RSI speed up for AI software R&D (algorithms and data); I did an estimate using my public preexisting modeling of drivers of AI progress and the public report which yielded a very rough estimate of 3.8x for how much RSI is currently accelerating AI software progress (relative to no AI automation). In the AI 2027 scenario, this much acceleration happens around March 2027.

Here are the worked details of this estimate based on my public modeling. There are two main reasons RSI speed up of AI software R&D must be larger:

• Not all of AI progress is from better software (some is from scaling up compute in training runs). I've previously speculated that around 2/3 of AI progress is from software improvements which implies that 2x overall AI progress acceleration requires 2.5x faster software progress. (Math: (2 years of progress - 1/3 of a year from compute) / (2 / 3 of a year of progress for each year of software progress.) ^[1]
• Diminishing returns: A fixed amount of progress requires more work over time, so to ongoingly maintain 3x faster software progress via AI automation, you need increasingly large speed boosts from AI automation (as in, speed ups relative to how fast software R&D would go without AI automation). Based on this report, it seems that there is a substantial amount of additional AI progress from automation that has occurred.

We can estimate the effects of diminishing returns by first determining how many additional years of software progress we've seen (relative to the preexisting trend) and then estimating how much software R&D would slow down after a year of software progress if you held inputs fixed due to diminishing returns. I've previously estimated software R&D would slow down by around 2x for each year of software progress if you held inputs fixed and based on the report, it seems as though we've perhaps seen roughly 0.35-1.0 years of additional software progress incorporated into trained models so far (geometric mean of ~0.6). This combines to 2. t * 2^(~0.6) = ~3.8.

(Relevant public modeling and numbers I'm using: here and here.)

I say "It says AI progress is greatly sped up by RSI right now, by around 2x." More precisely, the report authors estimate that AI progress has recently been accelerated by around 2x (though with large uncertainty and some redacted evidence pointing towards larger acceleration while other evidence is unclear) and believe this is largely due to RSI.

I say "In AI 2027, this happens roughly at March 2027." AI 2027 gives numbers in terms of AI automation speed up for AI software R&D, not overall speed up, so you need to do some kind of conversion. I did this by estimating the current AI automation speed up and then matching this to AI 2027. If you instead try to estimate the overall AI progress speed based on AI 2027's modeling, you get that 2x happens somewhere around February to April which is consistent.

The 4.5x estimate I give above is for how much AI is accelerating software R&D progress, not how much the labor input to AI software progress is effectively accelerated (serial labor acceleration). In other words: a different quantity you could estimate is the value of AI automation relative to making your researchers/engineers operate X times serially faster (while keeping compute fixed) and for what value of X you’d be indifferent between these. Software R&D is driven by more than just labor (e.g., surveys generally indicate experiment compute is believed to be an important input) so we’d expect a substantially larger serial labor acceleration than the overall acceleration to software R&D progress.

Crossposted from X/twitter

1. I fixed an error in my modeling here where I forgot to subtract off the 1/3 of a year of progress from compute scale up, which changes this number from 3 to 2.5. ↩︎