As of October 2026, benchmark scores appear to be growing at 1.9x their pre-October 2025 rate
... isn't this graph actually kind of a bear signal?
No, seriously. The hypothesis we're tracking is RSI, i. e., that AI capability improvements make additional AI capability improvements easier. This should mean that the acceleration multiplier is growing: from one model release to another, the slope of the graph should be becoming ever steeper, such that the graph as a whole stops being well-modeled as linear.
What the graph is showing, instead, is a one-off slope change somewhere around the start of 2026, past which the slope remained the same. That is, there was some sort of "unlock" in early 2026 that let AI make Anthropic employees more productive[1], and then additional capability improvements no longer made further capability improvements easier. For the entire year.
I suppose the bull case is that RSI is just "discrete" instead of continuous? I. e., that there will be a few more discrete slope changes like this, and each slope change will bring the next slope change sooner, up until AI is able to handle all research tasks and we get a multiplier of "approximately positive infinity" or something. But this isn't how the argument usually goes, I think.
Edit: Employees' self-reported uplift is more like how RSI should look like, a straight line on the productivity multiplier graph. But these data are considered much more unreliable, and if it's to be taken seriously together with the other graph, it would imply that sustaining the current 2x growth would require continual exponential improvement in employee productivity.
E. g., the "now only bottlenecked on ideas & compute, no longer bottlenecked on code" stuff from the interviews.
We went from the evidence indicating "it was a one-time jump at Mythos Preview" to "oh there was some constant acceleration" to "oh there was a bunch of constant acceleration" and now METR is saying (paraphrased) "we can't really rule out that actually the acceleration is growing, and we have some redacted evidence indicating this, though AECI doesn't necessarily indicate it". So I'm expecting (with some probability) evidence in the future that will update me toward growing acceleration.
Yeah, that makes sense to me. Agreed.
Twitter thread.
Ryan Greenblatt:
Ajeya Cotra:
Daniel Kokotajlo: says the METR report confirms his worry from two days ago, which was:
See also GradientDissenter's take.