Thane Ruthenis's Shortform

Thane Ruthenis

I am pretty annoyed with the messaging around METR's latest result.

Our chief concern is RSI dynamics, i. e., AI capability progress making subsequent AI capability progress easier. The way it would show up on graphs is by making straight lines steeper and steeper. If we observed 1.5x acceleration in one time period, we should observe 2x acceleration in the next time period, 2.5x in some subsequent one, et cetera. This is what RSI implies: a line continually going up on the graph of productivity multipliers. In other words, research acceleration at T+1 should be a function of research acceleration at T. This is what scenarios like AI 2027 assume: that there are compounding multiplicative dynamics at play.

But that is not what the graph in the report, if taken at face value, shows. The graph shows a single slope change around the start of 2026 – something that made Anthropic permanently better at AI research – and then no additional productivity improvements all throughout 2026. This is better modeled as a one-off "productivity unlock", rather than smoothly predictable RSI.^[1]

In particular, it's not straightforwardly extrapolable into claims of the form "we will reach 4x acceleration by January 2027, 8x acceleration by March", etc. Taken at face value, the graph says we reached 2x acceleration by early 2026, and that we're still at 2x acceleration. Will we stay at 2x all throughout 2027 as well? Will there be a different discontinuous slope change that puts us at 10x next week?^[2] Who knows.

There are two model breaks: (1) the current speed-up to AI research from AI is much larger than previously predicted, (2) the current speed-up to the speed-up is much lower than was expected at this speed-up value. This means that whatever this 2x-speedup figure is, it's not the same type of figure as the compounding multiplier in AI 2027. (Say, maybe the straight line on the multipliers graph should have a vertical offset, because significant aspects of the speed-up are non-compounding?)

Yet, people are blindly plugging that number into their compounding-multiplicative-dynamics models, and then interpreting that as, wow, we're just straightforwardly N months ahead of the AI 2027 schedule, ASI first thing next year.

Look, I don't even necessarily disbelieve the claims that people are trying to frame these results as making. Yes, surely there are some compounding multiplicative dynamics at play somewhere, and plausibly AI progress (on verifiable tasks et cetera) is currently going 2x faster than baseline largely due to boosts from AI itself. Sure, why not? Sounds plausible enough.

But, like, what that specific piecewise-straight line on that specific graph is literally saying is not what people are taking it as.

^{^}
There's even a plausible mechanism for that, see interviews where Anthropic employees say things like "now only bottlenecked on ideas & compute, no longer bottlenecked on code". If they got basically-completely-unbottlenecked around Mythos...
^{^}
Say, as LLMs finally grok research taste and become able to contribute fully autonomously.

Thane Ruthenis1hΩ225220

7Daniel Kokotajlo1h

Their conclusion is based on much more evidence than just that graph, it's just that a lot of the other evidence was redacted.

Vladimir_Nesov20mΩ6130

a lot of the other evidence was redacted

I don't expect such other evidence to be convincing either, for the kinds of evidence that seems compatible with the rumors currently circulating online. The issue is the whole frame of smoothly obtaining broadly-unbounded acceleration (without paradigm-shifting algorithmic unlocks such as continual learning in a strong sense), and relevance of claims like "around 2/3 of AI progress is from software improvements" when they're interpreted as predictions about what happens if we hypothetically permanently fix model ... (read more)

6Thane Ruthenis35m

Which conclusion? Again, I'm not necessarily questioning the current 2x speed-up. I'm questioning the inferences of the form "2x speed-up now implies 2.5x in the next time period", etc. Those inferences are directly contradicted by the graph if read straightforwardly, and indeed I don't think METR itself asserted them as a conclusion at all? (They say they don't observe further bottlenecks or diminishing returns, and that this means compounding multiplicative dynamics may kick in and lead to further acceleration. But they don't say that they're already observing such aggressive compounding dynamics, as we should expect them to have observed if this multiplier is the same beast as the multiplier in AI 2027 or in Ryan's estimates.)

Thane Ruthenis's Shortform

9

Ω 5

Thane Ruthenis's Shortform

9

Ω 5