The case for multi-decade AI timelines [Linkpost]

Noosphere89

This is a linkpost for https://epoch.ai/gradient-updates/the-case-for-multi-decade-ai-timelines

So this post is an argument that multi-decade timelines are reasonable, and the key cruxes that Ege Erdil has with most AI safety people who believe in short timelines are due to the following set of beliefs:

Ege Erdil doesn't believe that trends exist that require AI to automate everything in only 2-3 years.
Ege Erdil doesn't believe that the software-only singularity is likely to happen, and this is perhaps the most important crux he has with AI people like @Daniel Kokotajlo who believe that a software-only singularity is likely.
Ege Erdil expects Moravec's paradox to bite hard once AI agents are made in a big way.

This is a pretty important crux, because if this is true, a lot more serial research agendas like Infra-Bayes research, Natural Abstractions work, and human intelligence augmentation can work, and also it means that political modeling (like is the US economy going to be stable long-term) matter a great deal more than is recognized in the LW/EA community.

Here's a quote from the article:

I don’t see the trends that one would extrapolate in order to arrive at very short timelines on the order of a few years. The obvious trend extrapolations for AI’s economic impact give timelines to full remote work automation of around a decade, and I expect these trends to slow down by default.
I don’t buy the software-only singularity as a plausible mechanism for how existing rates of growth in AI’s real-world impact could suddenly and dramatically accelerate by an order of magnitude, mostly because I put much more weight on bottlenecks coming from experimental compute and real-world data. This kind of speedup is essential to popular accounts of why we should expect timelines much shorter than 10 years to remote work automation.
I think intuitions for how fast AI systems would be able to think and how many of them we would be able to deploy that come from narrow writing, coding, or reasoning tasks are very misguided due to Moravec’s paradox. In practice, I expect AI systems to become slower and more expensive as we ask them to perform agentic, multimodal, and long-context tasks. This has already been happening with the rise of AI agents, and I expect this trend to continue in the future.

This is a linkpost for https://epoch.ai/gradient-updates/the-case-for-multi-decade-ai-timelines

Ege Erdil doesn't believe that trends exist that require AI to automate everything in only 2-3 years.
Ege Erdil doesn't believe that the software-only singularity is likely to happen, and this is perhaps the most important crux he has with AI people like @Daniel Kokotajlo who believe that a software-only singularity is likely.
Ege Erdil expects Moravec's paradox to bite hard once AI agents are made in a big way.

Here's a quote from the article:

I don’t see the trends that one would extrapolate in order to arrive at very short timelines on the order of a few years. The obvious trend extrapolations for AI’s economic impact give timelines to full remote work automation of around a decade, and I expect these trends to slow down by default.
I don’t buy the software-only singularity as a plausible mechanism for how existing rates of growth in AI’s real-world impact could suddenly and dramatically accelerate by an order of magnitude, mostly because I put much more weight on bottlenecks coming from experimental compute and real-world data. This kind of speedup is essential to popular accounts of why we should expect timelines much shorter than 10 years to remote work automation.
I think intuitions for how fast AI systems would be able to think and how many of them we would be able to deploy that come from narrow writing, coding, or reasoning tasks are very misguided due to Moravec’s paradox. In practice, I expect AI systems to become slower and more expensive as we ask them to perform agentic, multimodal, and long-context tasks. This has already been happening with the rise of AI agents, and I expect this trend to continue in the future.

Macroscopic biotech is not software-only singularity (and doesn't depend on it being possible), it's a separate example of a system that is not human industry, that seems sufficient to create a rate of scaling completely implausible for the human economy (illustrating the crux with Ege Erdil's position, which system is relevant for timelines). Software-only singularity operates within an unchanging amount of raw compute, while macroscopic biotech takeoff manufactures new compute much faster than human industry could be instructed to.

The "fruit flies" are the material substrate, packed with DNA that can specialize the cells to do their part in a larger organism reassembled from these fast-growing and mobile packages of cells, and so precise individual control over the "fruit flies" is unnecessary, only the assembled large biorobots will need to be controllable. The main issue is that macroscopic biotech probably won't be able to create efficient compute or fusion plants directly, but it can create integrated factories that manufacture these things (or non-biological robots) in more standard ways by reproducing the whole necessary supply chains starting with raw materials (or whatever is most convenient to transport) on site, plausibly extremely quickly. With enough robots, city-sized factories could be assembled overnight, and then reassembled again and again wholesale, as feedback from trying it out in the physical world reveals flaws in design.

59

The case for multi-decade AI timelines [Linkpost]

59

59

59

The case for multi-decade AI timelines [Linkpost]

59

59