The last few days, David and I did the same exercise which IIUC produced AI 2027: predict the next month, condition on those predictions, predict the month after that, etc. We assumed that:
there is no secret sauce of intelligence missing from today's LLM-based AIs
METR's "time horizon" metric is a basically reasonable way to think about LLM capabilities
Besides just running forward median predictions, for each month we also came up with a list of potential surprises, of a similar flavor to "news breaks that Jensen Huang had an affair with a Chinese spy" or "OpenAI releases a proof that the Exponential Time Hypothesis is false", and we randomly rolled a couple of those for each month.
The random rolls came out moderately worse than I'd actually expect, even conditional on the assumptions. But they weren't that far out of distribution, and...
We died in January. Two months from now.
Regulations hadn't even been passed, let alone implemented; if the government was't already enforcing a full Pause in December they didn't matter at all. AIs were starting to be able to run profitable businesses, which made it possible for the first time for AIs to exfiltrate and operate fully independently online; that also didn't matter at all. There was some crazy stuff going on with AI company finances, and none of it mattered at all. The Hackening hadn't even gotten seriously underway yet.
The only thing which mattered was a single random event: "OpenAI announces that their internal automated AI research swarm has found a way to improve the scaling power-law coefficient on data by 20%.". That wasn't written to be a "we die" event, just a plausible sounding algorithmic advancement. When we ran the numbers, it was enough for immediate supercritical RSI.
has found a way to improve the scaling power-law coefficient on data by 20%
Isn't this literally a secret sauce though? The only difference being that the resulting secret sauced system is still an LLM. The "20%" is making it sound not crazy, but to a scaling law true believer (Dario?) that probably sounds like "the sun gets 20% hotter"
But in the AI2027 model, humans don't die immediately after software singularity. Does this event somehow makes all of the post-mid-2027 happen faster also?
4johnswentworth
Ok, reasonable.
First, I assign a decent chance that LLMs are missing some secret sauce of intelligence which won't be quickly sorted out. Maybe 25-35%.
In worlds where LLMs are not missing some secret sauce of intelligence, or where the secret sauce will be quickly sorted out, I think January is on the quick side but not out of distribution. Like, maybe 10% chance that things go that fast, conditional on secret sauce not being a blocker. We did have an unlucky roll in the exercise, hitting the algorithmic jump that early.
I do think the estimates of takeoff in mid-to-late 2027 look pretty reasonable at this point, as a slightly-faster-than-median guess.