Thank you @elifland for reviewing this post. He and AI Futures are planning to publish updates to the AI 2027 Timeline Forecast soon.

AI 2027 (also launched in this LW post) forecasts an R&D-based AI takeoff starting with the development of Superhuman Coders^[1] within a frontier lab.

FutureSearch co-authored the AI 2027 Timeline Forecast. We thought the other authors’ forecasts were excellently done, and as the piece says:

All model-based forecasts have 2027 as one of the most likely years that SC [Superhuman Coders] being developed, which is when an SC arrives in the AI 2027 scenario.

Indeed. But overall, FutureSearch (two full-time forecasters, two contract forecasters, and this author, Dan Schwarz) think superhuman coding will arrive later — median 2033 — than the other authors (hereon "AI Futures"): median 2028 from Nikola Jurkovic, and median 2030 from Eli Lifland^[2].

Here, we briefly explain how our views diverge on what it would take for OpenAI/DeepMind/Anthropic/xAI to develop superhuman coding.

This post can also serve as a much shorter piece for those who didn’t read the whole (24-page!) piece on the AI 2027 site. (See also Scott Alexander's shorter piece on the AI Futures blog on modeling coding time horizons.)

The Forecast

As @Max Harms wrote in another AI 2027 LW reaction piece,

we can hallucinate that FutureSearch has approximately the market timelines, while Eli Lifland (and perhaps other authors) has a faster, but not radically faster timeline.

I think this is broadly correct, even though the “market” timeline here is hard to interpret because as Max says, Superhuman Coding ≠ AGI.

Our published view in AI 2027 — blue line below — is fairly in-line with similar crowd forecasts, while being much slower than the AI Futures:

We produced, but did not publish on AI 2027, an all-things-considered forecast with a median in Nov 2033, vs the "within-model" forecast of Jan 2032 above:

So while 2027 is conceivable, why do we give a median more than 8 years in the future?

The Path to Superhuman Coding

AI Futures produced, and FutureSearch then adopted, a clever methodology for this forecast that we endorse and used ourselves. It’s written in the piece as “Method 2”, and it has two steps:

Forecast the time for RE-Bench ^[3] to be saturated
Forecast the time for everything else needed from a model that aces RE-Bench for it to act as a super

...

Posts

Wikitag Contributions

Comments