It's important to distinguish between two different takeaways you could have from the recent METR report:
1. Shorter timelines: AI will soon fully automate AI R&D
2. Intelligence explosion more likely: Once AI fully automates AI R&D, AI progress will accelerate until we get superintelligence
I will walk through my tentative views on each.
Strong update towards shorter timelines
I think this is pretty clear. AI is already much better at AI research than I had realized and it is improving at a faster rate than I had realized.
So we're less far to go to full automation than I thought and we're moving towards it more quickly.
Moderate update towards an intelligence explosion.
This one is more complicated.
The key questions is about whether the feedback loop of AI improving AI is strong enough to accelerate over time, even holding compute fixed.
We can break this feedback loop down as follows:
more ECI -> more R&D labour -> more total R&D effort -> more ECI -> ...
Now, let's assess what we can learn about each stage of this feedback loop.
Stage 1: more ECI -> more R&D labour
When you increase ECI, how much more quality adjusted labour do you get at AI R&D?
Well, recently the amount of quality-adjusted labour increased pretty fast! A moderate increase in ECI led to a big increase in labour.
So stage 1 is stronger than I used to think.
Stage 2: more R&D labour -> more total R&D effort
Can improved labour have the same effect as increasing both compute and labour? Or does more labour not make much difference because compute experiments become a big bottleneck?
Well, we've already had a ~2x speed from more labour without getting bottlenecked by compute. That rules out strong "bottleneck" views that I've seen economists endorse.[1]
But it doesn't rule out compute bottlenecks biting a bit later on. In my opinion, these were always the most plausible bottleneck views.
It also doesn't speak to the possibility that much algorithmic progress depends on scale-dependent algorithms. If so, alg progress might be much slower if we couldn't increase the scale of training compute and benefit from new algorithms that only emerge at new scales. (Though METR's claim of "no foreseeable bottlenecks" is evidence against this.)
So for me there is only minor update on the strength of stage 2. Compute bottlenecks are still plausible. IE far from inevitable.
Stage 3: more total R&D effort -> more ECI
Some of the speed-up in recent progress wasn't driven by increasing R&D efforts, but just by it being easier to make AI progress than before. Eg the speed-up in progress from RL.
This is evidence that stage 3 is more powerful. Doubling R&D inputs led to more progress than we thought. (Though only a little bit more, if you think most of the acceleration was due to AI increasing labour inputs.)
The more sceptical you are of stage 1 - that AI has driven the recent faster progress - the more you should update about stage 3. Once you accept that progress has indeed become faster, you have to update upwards on one of the stages!
Summary
So, to summarise how my views on each stage have changed:
- more ECI -> more R&D labour: much stronger than I thought
- more R&D labour -> more total R&D effort: stronger than I thought, but compute bottlenecks still plausible
- more total R&D effort -> more ECI: a little bit stronger than I thought
Overall, this is definitely evidence that there will accelerating progress. But the evidence doesn't feel overwhelming at all. Still could go either way. I feel less bullish than AIFP seem to be.
And remember, even without accelerating progress, AI progress will still become 2-3xfaster if we automate AI R&D. And we can't handle that.
[1] On a strong bottleneck view, if one the growth of input R&D input accelerates but the other doesn't, progress is bottlenecked by the latter. Compute growth hasn't accelerated, only labour. Strong bottleneck views predict that the labour acceleration won't make much difference because of the compute bottleneck.
It's important to distinguish between two different takeaways you could have from the recent METR report:
1. Shorter timelines: AI will soon fully automate AI R&D
2. Intelligence explosion more likely: Once AI fully automates AI R&D, AI progress will accelerate until we get superintelligence
I will walk through my tentative views on each.
Strong update towards shorter timelines
I think this is pretty clear. AI is already much better at AI research than I had realized and it is improving at a faster rate than I had realized.
So we're less far to go to full automation than I thought and we're moving towards it more quickly.
Moderate update towards an intelligence explosion.
This one is more complicated.
The key questions is about whether the feedback loop of AI improving AI is strong enough to accelerate over time, even holding compute fixed.
We can break this feedback loop down as follows:
more ECI -> more R&D labour -> more total R&D effort -> more ECI -> ...
Now, let's assess what we can learn about each stage of this feedback loop.
Stage 1: more ECI -> more R&D labour
When you increase ECI, how much more quality adjusted labour do you get at AI R&D?
Well, recently the amount of quality-adjusted labour increased pretty fast! A moderate increase in ECI led to a big increase in labour.
So stage 1 is stronger than I used to think.
Stage 2: more R&D labour -> more total R&D effort
Can improved labour have the same effect as increasing both compute and labour? Or does more labour not make much difference because compute experiments become a big bottleneck?
Well, we've already had a ~2x speed from more labour without getting bottlenecked by compute. That rules out strong "bottleneck" views that I've seen economists endorse.[1]
But it doesn't rule out compute bottlenecks biting a bit later on. In my opinion, these were always the most plausible bottleneck views.
It also doesn't speak to the possibility that much algorithmic progress depends on scale-dependent algorithms. If so, alg progress might be much slower if we couldn't increase the scale of training compute and benefit from new algorithms that only emerge at new scales. (Though METR's claim of "no foreseeable bottlenecks" is evidence against this.)
So for me there is only minor update on the strength of stage 2. Compute bottlenecks are still plausible. IE far from inevitable.
Stage 3: more total R&D effort -> more ECI
Some of the speed-up in recent progress wasn't driven by increasing R&D efforts, but just by it being easier to make AI progress than before. Eg the speed-up in progress from RL.
This is evidence that stage 3 is more powerful. Doubling R&D inputs led to more progress than we thought. (Though only a little bit more, if you think most of the acceleration was due to AI increasing labour inputs.)
The more sceptical you are of stage 1 - that AI has driven the recent faster progress - the more you should update about stage 3. Once you accept that progress has indeed become faster, you have to update upwards on one of the stages!
Summary
So, to summarise how my views on each stage have changed:
- more ECI -> more R&D labour: much stronger than I thought
- more R&D labour -> more total R&D effort: stronger than I thought, but compute bottlenecks still plausible
- more total R&D effort -> more ECI: a little bit stronger than I thought
Overall, this is definitely evidence that there will accelerating progress. But the evidence doesn't feel overwhelming at all. Still could go either way. I feel less bullish than AIFP seem to be.
And remember, even without accelerating progress, AI progress will still become 2-3x faster if we automate AI R&D. And we can't handle that.
[1] On a strong bottleneck view, if one the growth of input R&D input accelerates but the other doesn't, progress is bottlenecked by the latter. Compute growth hasn't accelerated, only labour. Strong bottleneck views predict that the labour acceleration won't make much difference because of the compute bottleneck.