I also asked this question on the Effective Altruism Forum. One informative answer on the EA Forum from Nicole Nohemi has already summarized a range of relevant forecasts from Metaculus, as well as relevant predictions or models from Shane Legg and Demis Hassabis from leading AI research company DeepMind, and AI alignment researchers Eliezer Yudkowsky, Paul Christiano and Ajeya Cotra. This question has been cross-posted to LessWrong to broaden the conversation to the rationality community as well.
The proportion of long-termists who are expressing confident convictions that the timeline for smarter-than-human AI is much shorter than has previously been predicted has been increasing at an accelerating rate in the last year. This appears to be a shift in perspective among several hundred long-termists. Among the dozens I've read opinions from, numbers are almost never provided.
Based on the way different people are talking about it, I wouldn't be surprised if they thought the timeline is 10-20 years, or 5-10 years, or even 2-3 years. I've talked to others who are also concerned and open-minded to this or that short AI timeline but haven't done the research themselves yet, or had much opportunity to learn from those who have. We want to understand better but basic information crucial to understanding more like numbers for different models or timelines aren't being presented. We want to know and need to know help better. What are the numbers?
I think the difficulty with answering this question is that many of the disagreements boil down to differences in estimates for how long it will take to operationalize lab-grade capabilities. Say we have intelligences that are narrowly human / superhuman on every task you can think of (which, for what it’s worth, I think will happen within 5-10 years). How long before we have self-replicating factories? Until foom? Until things are dangerously out of our control? Until GDP doubles within one year? In what order do these things happen? Etc. etc.
If I got anything out of the thousands of words of debate on the site in the last couple of months, it’s the answers to these questions that folks seem to disagree about (though I think I only actually have a good sense of Paul’s answers to these). Also curious to see more specific answers / timelines.
The same point was made on the Effective Altruism Forum and it's a considerable one. Yet I expected that.
The problem frustrating me is that the relative number of individuals who have volunteered their own numbers is so low it's an insignificant minority. One person doesn't disagree with their own self unless there is model uncertainty or whatever. Unless individ... (read more)