A major plausible class of worlds in which we don't get superhuman coders by end of 2026 is worlds where the METR trend continues at roughly the same or only slightly greater slope than the slope it had in 2025. Right?
Yes, but 2025 saw two trends: Claude 3.5 Sonnet -- o3 and o3 -- GPT-5.1CodexMax with different doubling times. IIRC the earlier trend would cause superhuman coders to appear by 2028 and the later trend (which was arguably invalidated by Claude 4.5 Opus and its ~5h time horizon; see, however, two comments pointing out that the METR benchmark is no longer as trustworthy as it once was and my potemtial explanation of the abnormally high 50%/80% time horizon ratio) had superhuman coders arrive in 2030 or outright hit a wall[1] before becoming superhuman.
As for the OP's idea that coding agents are used to improving coding agents and reaching the SC, this could be unlikely because they don't improve the underlying LLM. I remember the now-obsolete benchmarks-and-gaps model which required the SCs not just to saturate the RE-bench, but learn to actually do long tasks and handle complex codebases, which in turn requires either a big attention span of the LLM itself or careful summarisation of each method's specification, of formatting, of other methods' names, etc.
P.S. The latter scenario would be particularly difficult to predict as it might involve the time horizon in the METR sense behaving like . In this case the horizon would grow ~exponentially until the very last couple of doublings.
Or become neuralese with consequences as disastrous as the lack of Safer-1 to test alignment.
Andrej Karpathy posted 12 hours ago (emphasis mine):
This seems to be a big update since his Dwarkesh episode published on Oct 17 (though I know these things can take a while to get edited, so the gap could be even bigger), where he said:
This is just me guessing, but Claude Opus 4.5 released just one month ago, and Opus 4.5 + Claude Code seems like the big shift for a lot of people.
In fact, Boris Cherny, creator of Claude Code, commented on Karpathy's post saying (emphasis mine):
To be clear, a lot of these PRs might be "quite small, a few lines and bug fixes" (cf. this comment by another Anthropic employee). Boris had just asked users for feedback, then closed 19 PRs the next morning. Still, 200 PRs in a month without opening an IDE is something [1].
AI Accelerating AI
It seems like we might be entering something like a self-improving feedback loop for the system "humans + AI": employees at the labs are developing AI coding agents using these same AI coding agents, with the horizon length of these models increasing on a faster exponential than we thought (cf. Opus 4.5), and potentially not even an exponential.
This isn't AI autonomously improving itself, but the feedback loop between training better AI models and having these models accelerate the automation of AI R&D seems to be tightening [2].
The "Coding Overhang"
In July 2020, after GPT-3, Andy Jones asked if we were in an AI Overhang, because (at the time) it felt like companies could just be scaling models like GPT-3 to many more orders of magnitude and get much more "intelligence".
With coding agents and reasoning / test-time compute, it seems to me that what Karpathy (& Boris) are describing is some sort of "Coding Overhang" where people at the cutting edge, and especially members of technical staff, are trying to catch up with ~10x improvements that are purely user-dependent skill-issues.
In what worlds do we not get Superhuman Coders by the end of 2026?
Note: As the creator of Claude Code, Boris is obviously incentivized to promote it.
See this older 2021 post I wrote about self-improving {humans + AI} systems, or this video explaining Tom Davidson's full takeoff model for more intuitions.