So this post is an argument that multi-decade timelines are reasonable, and the key cruxes that Ege Erdil has with most AI safety people who believe in short timelines are due to the following set of beliefs:
Ege Erdil doesn't believe that trends exist that require AI to automate everything in only 2-3 years.
Ege Erdil doesn't believe that the software-only singularity is likely to happen, and this is perhaps the most important crux he has with AI people like @Daniel Kokotajlo who believe that a software-only singularity is likely.
Ege Erdil expects Moravec's paradox to bite hard once AI agents are made in a big way.
This is a pretty important crux, because if this is true, a lot more serial research agendas like Infra-Bayes research, Natural Abstractions work, and human intelligence augmentation can work, and also it means that political modeling (like is the US economy going to be stable long-term) matter a great deal more than is recognized in the LW/EA community.
Here's a quote from the article:
I don’t see the trends that one would extrapolate in order to arrive at very short timelines on the order of a few years. The obvious trend extrapolations for AI’s economic impact give timelines to full remote work automation of around a decade, and I expect these trends to slow down by default.
I don’t buy the software-only singularity as a plausible mechanism for how existing rates of growth in AI’s real-world impact could suddenly and dramatically accelerate by an order of magnitude, mostly because I put much more weight on bottlenecks coming from experimental compute and real-world data. This kind of speedup is essential to popular accounts of why we should expect timelines much shorter than 10 years to remote work automation.
I think intuitions for how fast AI systems would be able to think and how many of them we would be able to deploy that come from narrow writing, coding, or reasoning tasks are very misguided due to Moravec’s paradox. In practice, I expect AI systems to become slower and more expensive as we ask them to perform agentic, multimodal, and long-context tasks. This has already been happening with the rise of AI agents, and I expect this trend to continue in the future.
I don’t see the trends that one would extrapolate in order to arrive at very short timelines on the order of a few years. The obvious trend extrapolations for AI’s economic impact give timelines to full remote work automation of around a decade, and I expect these trends to slow down by default.
Actually, I think the most obvious extrapolation of revenue indicates around 4 years to full remote work automation: current annualized AI company revenue is around $20 billion, we see 3x revenue growth per year, AI companies might internalize 10% of the value, so for ~$20 trillion of value due to remote work automation, we might expect ~$2 trillion in revenue which would naively happen in a bit over 4 years (late 2029).
(I don't think revenue extrapolation is a great forecasting methodology on my views for multiple reasons, but still seems worth engaging with to some extent.)
I think there are plausible reasons for this trend to speed up or to slow down (speeding up seems plausible even without AI accelerated AI R&D making a difference), and more minimally we'll see big impacts prior to full automation. Slowing down seems more likely than speeding up or staying at the same rate, but doesn't seem overwhelmingly likely (maybe I'd say around 30% chance that the trend stays at or above exponential up to $2 trillion in global revenue).
I discuss this sort of extrapolation and related considerations in more detail here.
So, I think a key takeaway from revenue extrapolation is it makes <4 year timelines look quite plausible (though it doesn't suggest they are >50% likely)!
So this post is an argument that multi-decade timelines are reasonable, and the key cruxes that Ege Erdil has with most AI safety people who believe in short timelines are due to the following set of beliefs:
This is a pretty important crux, because if this is true, a lot more serial research agendas like Infra-Bayes research, Natural Abstractions work, and human intelligence augmentation can work, and also it means that political modeling (like is the US economy going to be stable long-term) matter a great deal more than is recognized in the LW/EA community.
Here's a quote from the article: