This is a special post for quick takes by Lukas Finnveden. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Ajeya has a simple model of future accelerating growth in the post here. Maybe the first doubling of the pace took 12 months. So with twice the pace, maybe the next doubling will take 6 months, then 3 months, then 1.5 months, etc, hitting limits within 12 months.
I think this argument doesn't really work as stated, but that a different argument still ends up supporting the idea that growth will accelerate until it hits extremely high levels.
The problem with the model is that it assumes that the first doubling came entirely from AIs getting more capable in that first year, and that AI's increasing ability to help with AI R&D was enough to not only overcome the diminishing returns from "ideas getting harder to find", but also to double the pace of progress on top of that.
But in fact (if the numbers from the METR assessment are right) what happened in the last year was that the combination of the 'normal' increase in exogenous inputs (e.g.: 3x scale-up of compute per year) and AI's increasing ability to help with AI R&D was enough to double the pace of progress.
This is importantly different, because if this "normal" increase in exogenous input was essential for fully doubling the pace of progress, then we can't extrapolate that the next 6 month period will be enough to double the pace of progress again. Because in the 6 month period, we'll only have half as big of an increase in exogenous inputs (maybe a 1.7x increase in compute rather than the yearly 3x increase).
From very roughly eyeballing the numbers, my current wild guess would be that, over the last year, the following three effects were roughly similarly large.
(At the beginning of the year, there was also a bump in exogenous inputs on top of the normal rate of scaling up exogenous inputs. But I'm ignoring that since it was likely a temporary thing.)
This would suggest that the "AI R&D speed-up per ECI point" is now high enough that, even without further exogenous inputs (i.e. if we froze compute and human labor at current levels), we might very roughly keep up the historical rate of progress, and it's unclear whether the rate of progress would be slightly decelerating over time or slightly accelerating over time towards theoretically infinite growth in finite time.
If we add exogenous inputs too, then further accelerations of the rate of progress seem likely. But if "AI R&D speed-up per ECI point" isn't high enough to speed-up growth on its own, then we wouldn't be on a trend towards reaching theoretically infinite growth in finite time; it would be more like seeing another doubling of the pace every year. (Which would also be very scary, to be clear.)
However, if the "AI R&D speed-up per ECI point" is currently at the point where we're teetering at the brink of self-sustaining and self-accelerating growth, I think "AI R&D speed-up per ECI point" is likely to soon increase and put us squarely into self-accelerating growth territory. The reason for this is that:
Or less theoretically and more practically: "AI R&D speed-up per ECI point" has been going up from near-0 to some significant value over the last 1-2 years, so absent some specific reason to expect this trend to stop or reverse, we should expect it to continue for a bit longer.
So overall, the recent news has made me update towards a software-only singularity being possible, and towards it happening soon.
This is assuming a measurement of "AI R&D speed-up per ECI point" that doesn't take into account "ideas are getting harder to find". Because we're trying to estimate if "AI R&D speed-up per ECI point" is large enough to overcome the effect from "ideas are getting harder to find".