This is a special post for quick takes by Buck. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I think that superintelligent AI within a year is ~33% likely
On the meaning of "superintelligence": Does this very likely involve a crucial architecture/methods change (whether invented by LLMs or not), such as continual learning, or could "superintelligence" arise from mere application of current methods (so that it can qualify as "superintelligence" before such changes are applied)? Must it be capable of inventing such changes soon thereafter? Are the reasons for credence completely agnostic on this, so that the relevant models or illegible expectations aren't making this distinction?
On the nature of reasons for credence: How much of this is within-model probability for the lines-on-graphs kinds of arguments (benchmarks, research speedup estimates/measurements), as opposed to the overall illegible expectation that includes things not captured by such models (or that includes other kinds of models)?
(For me, the main crux is the uncertain level of inherent difficulty in continual learning in a strong sense or any other sufficiently strong unhobbling methods that enable fast climbing on the tech tree, as opposed to merely reaching higher from the then-current cultural/technological standpoint while only climbing slowly. Based on the current LLMs and remaining scaling, it seems very likely they get to full automation of next-model building by 2032, probably by 2028, which are roughly the next rounds of scaling comparable to 1 year between Sep 2025 and Sep 2026, taking 2 and then 4 years starting about now. But this only gets the world to ASI in a strong sense if (1) LLMs/humans unhobble the learning speed by inventing something significantly novel that currently doesn't work, so that the AIs start climbing the tech tree quickly, or (2) LLMs suddenly get way smarter than I expect them to get while remaining within feasible scaling and while keeping their LLM/RL nature/methods mostly unchanged, becoming able to reach way higher on the tech tree even if they can't climb it quickly, and picking the relevant unhobblings even if they are very difficult to invent, or (3) all of this fails, then after the robot-industrial explosion even the essentially unchanged LLM/RL AIs build orders of magnitude more compute within 10-20 years, a brute-force way to start climbing the tech tree much faster.
So if the crucial unhobblings are relatively easy, we get strong ASI by 2028-2032, otherwise we only get it by 2038-2050. A ban/pause is also a possibility, especially if there's no strong ASI for a few years. The "world-class discovery benchmark" showed first serious signs of life in 2026, so it gets saturated in the next 1-2 rounds of scaling, for some human-level sense of "world-class discovery" that includes invention of key LLM/RL unhobblings if they are "relatively easy". And a lot of "neolab" AI startups are aiming at continual learning at various levels of seriousness in the relevant sense, so it could even be invented by humans in 2027-2030 if LLMs aren't yet scaled enough then to become capable of next-model building. So I'm putting 50% on the before-2032 side, with half of it before late 2028, shared by the key unhobblings being very easy and invented by humans, and the first round of climbing the "world-class discovery benchmark" beyond its current nascent state. Within 2026-2027, that maybe leaves 15%, which includes the 2028-scale ~200T total param models getting trained in advance by late 2027, though without slipping to 2028 there probably won't be enough time or resolve-to-immediate-ruin to both elicit the unhobblings and apply them, a process that might need to involve training new models.)