I’ve heard more arguments in favor of a near-term[1] coordinated AI pause/slow-down[2] than arguments against. I think such questions, including complex flow-through effects, are difficult, and it’s important to really consider both sides. I am not convinced by these arguments, and feel highly uncertain as to when a slow-down would be best, but believe it most likely will be good to slow down or pause AI development at some point.
Here are 10 reasons against a near-term AI slow-down:
A near-term slowdown could prevent the warning shots that would make the public take AI risk more seriously. And techniques for safe near-term AI may be insufficient for advanced AI, so a near-term slow-down could give a false sense of security for handling truly dangerous AI.
Slowing down too early could have an idea inoculation / boy-calls-wolf effect, where the topic becomes more politicized[3] and taken less seriously because people haven’t see anything scary enough to make them see AI as truly dangerous, and then when we really need to go slow later it’s much more difficult to get political will.
Not slowing down means we have less of various types of overhang that could make things more dangerous if/when full-speed research resumes, such as:
Compute overhang (if hardware is not also fully paused) and other technical advances that complement AI
Progress on robotics
More AI researchers joining the field
Overhang of new ideas for training AI more effectively
Progress in biology that lowers the floor for AI capabilities needed to create omnicidal pandemics
Various AI related info risks, such as would-be dictators realizing they could use AI to take over, and entrepreneurs realizing AI could make them rich
Etc. (I haven’t thought about this much but suspect there are more)
Getting near-human level AI systems before slowing down or pausing could give us AI that is much more helpful for making the most of the slow-down, e.g. AI for alignment research, epistemics including forecasting, coordination, policy and strategy research, etc.
Waiting to slow down means that we will have a much better understanding of the kind of AI risks that are most serious to focus on when we do pause or slow-down, including more realistic model organisms to study, as the shape of danger could shift significantly before AI becomes truly dangerous.
It is plausible that if we get advanced AI sooner, people who care about x-risk and the long-term future will be more differentially advantaged by AI use, as they are currently disproportionately focused on AI, but if things slow down this may give others time to catch up and cause regression to the mean with less altruistic or existentially concerned values influencing AI development.
Safety-focused labs are currently more in the lead, a slow-down could lead to loss of lead, and if the slow-down breaks down, safety focused labs may then have less of a buffer for slow-down when they need it (this could be considered a subset of number 6.)
Getting powerful AI sooner means we can save more people currently alive if this brings forward longevity escape velocity and doesn’t increase extinction risk[4]. More generally, advanced AI might help with many near-term issues such as animal welfare, poverty, and climate change.
A coordinated pause/slow-down, if it requires government involvement, would currently involve the Trump administration and Chinese Communist Party. At least half of this equation will likely change in a couple years.
Curious if people have anything they would add to this list? Also would love to hear arguments against each of these points, a red-team of my red-team!
What I mean by “near-term” is far more about when these factors come into play than it is about any wall-clock timeframe, but very roughly I’d estimate “near-term” means something like less than 6 months-2 years.
Note that whether or not a near-term coordinated slow-down is good is a separate question from whether it is good to advocate for or build the machinery for an eventual slow-down in the near-term; these may be necessary significantly in advance in order to spread awareness of the idea, widen the overton window, strategically prepare, and iteratively develop concrete high-quality plans for what the best version of a slow-down would look like. Whether a coordinated slow down would be good is also a separate question from whether individual actors should slow down.
An important consideration for person-affecting views. If not slowing down does increase x-risk, then this seems highly dubious, even from a person-affecting perspective, as we would be risking all lives to save some fraction of lives.
I’ve heard more arguments in favor of a near-term[1] coordinated AI pause/slow-down[2] than arguments against. I think such questions, including complex flow-through effects, are difficult, and it’s important to really consider both sides. I am not convinced by these arguments, and feel highly uncertain as to when a slow-down would be best, but believe it most likely will be good to slow down or pause AI development at some point.
Here are 10 reasons against a near-term AI slow-down:
Curious if people have anything they would add to this list? Also would love to hear arguments against each of these points, a red-team of my red-team!
What I mean by “near-term” is far more about when these factors come into play than it is about any wall-clock timeframe, but very roughly I’d estimate “near-term” means something like less than 6 months-2 years.
Note that whether or not a near-term coordinated slow-down is good is a separate question from whether it is good to advocate for or build the machinery for an eventual slow-down in the near-term; these may be necessary significantly in advance in order to spread awareness of the idea, widen the overton window, strategically prepare, and iteratively develop concrete high-quality plans for what the best version of a slow-down would look like. Whether a coordinated slow down would be good is also a separate question from whether individual actors should slow down.
Slowing down could also cause an economic downturn, increasing chances a premature slow-down would polarize people to be more against safety.
An important consideration for person-affecting views. If not slowing down does increase x-risk, then this seems highly dubious, even from a person-affecting perspective, as we would be risking all lives to save some fraction of lives.