HowManyAgents's Shortform

HowManyAgents

HowManyAgents's Shortform — LessWrong

Comment Permalink

Progress per unit of spending is going down. Putting METR's recent trend on a spending axis you see that although progress per unit of time has gone up, progress per unit of spending is logarithmic in R&D expenditure. I had never thought about this before and this personally updates me quite far away from foom soon even though until recently I was thinking foom was likely. Am I right to do so?

Showing 3 of 4 replies (Click to show all)

8dprkay2h

It's confusing to think through since a linear increase in ECI corresponds to roughly exponential capability gains. Fable 5.1 has an ECI of 167, compared to 125 for Haiku 3.5. METR's time horizon makes this more explicit: a doubling there tracks very well with ~5 ECI points. Here's an updated graph based on that: A problem with dollars is that it can just represent competition between labs for the same pool of compute and talent. Not that this has been hugely impactful—the graph has Q3 '26 spending at 5.46x Q3 '25, but R&D compute growth was probably only ~4.5x. Fwiw I don't think foom is congruent with the deep learning paradigm (where experiments and training runs take time) but I could just have a stricter definition. Like would you consider something like AI 2027 foom?

HowManyAgents29m10

Thanks this is useful. Yes AI2027 would be foom to me.

14Vladimir_Nesov4h

One thing that's misleading about LLMs is that they have actual scaling laws with compute, which are distinct from the possible experience curve things that are also happening. And all of this is happening at some pace over time, so time also gets conflated with scale. My model is basically that progress should be thought of as happening depending on scale, not depending on time; and that gradual algorithmic progress is more like an experience curve effect that therefore slows down if/when scaling slows down. This just doesn't say anything about paradigm-shifting innovations, which can happen at any moment and very quickly lead to strong ASI. (So my contrarian take is that slow takeoff is actually slow rather than fast, while fast takeoff remains very fast indeed even after years of observing a slow takeoff.)

See in context