AGI-pilled people often don't take seriously either the possibility of many years of RSI-without-ASI, or the possibility of a sudden ASI at some point. I think the baseline scenario involves both, and ignoring either one of them predicts very different effects of possible policies.

The basic case of recursive self-improvement is self-building, an LLM autonomously building a new version of an LLM like itself, without necessarily making important non-routine improvements, at which point the extent of self-improvement becomes quantitative. This is a project involving verifiable tasks, that LLMs seem to be very on track to become capable of being trained for using RLVR (in a broad sense, with graders that can invoke LLMs). The LLMs of 2026 are probably about 1T active params, 10T total, trained for 1e27 FLOPs with 300 MW of compute capacity. The LLMs of 2031 might be about 40T active params, 1,000T total, trained for 2e29 FLOPs with 10 GW of compute capacity, 10x more expensive per token, and slightly faster than the LLMs of today. The next big model scale-up comes with Nvidia Kyber racks in 2028 (when most of the buildout happens; a bit of it starts in 2027), so it's likely self-building starts working then. And then the 8x Kyber Feynman systems of 2029 or 2030 might enable models capable of self-improvement to a more meaningful degree than mere self-building.

Being AGI-pilled about LLMs paints the picture of not just AGI as a normal technology, but RSI as a normal technology. Prosaic RSI of preparing new training data, including new RLVR tasks and environments, and then building the next LLM solves generalization by introducing the out-of-distribution situations encountered in practice into the training data for the next model. This is quite slow when using the current cookbook, even if the process is fully automated, performed by the LLMs themselves. Yet this promises full automation (and automation-of-automation) for all engineering-type or otherwise verifiable tasks, no matter how specialized. Thus we get AGI, which is slow at adapting to new situations, but capable of general learning and fast at executing on what it already knows. This doesn't automatically lead to ASI, and the RSI process isn't inherently destabilizing.

Not being ASI-pilled is then a giant blind spot, where you expect a "smooth exponential" that is on-trend for the LLMs, a controlled chain reaction of prosaic RSI, possibly leading to some sort of similarly smooth industrial explosion, but not to a software-only intelligence explosion, where cognition goes supercritical. I think the valid point is that prosaic RSI with LLMs doesn't predict ASI any time soon, and thus RSI can continue smoothly for many years.

But while this is happening, nothing prevents a breakthrough (at any time) that makes learning of deep skills fast rather than slow, at the speed comparable to token generation rather than model releases, cutting any plans built on the foundation of the "smooth exponential" short. Hundreds of gigawatts of global AI compute in the 2030s can rapidly scale any such breakthrough to overwhelming effect, with all the engineering details taken care of by the LLMs, after some steps of prosaic RSI if that's necessary for them to really understand what needs to be done to scale the breakthrough. Not expecting this possibility risks failing to prepare for it (and then suddenly it's too late). In contrast, not seeing the possibility of prosaic RSI merely delays when the policies about the ASI danger (apart from shutting it all down) start talking about the relevant context in which they need to be able to work.