I often hear people say they think we should pause AI at some point, but not yet. Their basis for this seems to be some combination of:
If we pause at the last possible moment, then we will have the most advanced AI possible during the pause, which will be helpful for doing AI safety research during the pause
Implicitly, there is some quantity of ‘pausing credit’, that will buy us a few months of pause say, and if we use them now, we won’t have them to use later, when it is important
If we pause, and then AI doesn’t seem to be at dire risk of destroying the world, maybe the public will backlash against this and it will be harder to do any kind of AI safety (especially if it has major economic consequences)
The models aren’t dangerous yet
This all sounds very questionable to me. I suggest instead that the following are at least as likely to be true:
We can’t pause on a dime at the precise second that ‘we’ decide it is important to—pulling the breaks will take a while, during which time we will continue to rocket into danger.
If we managed to pause now, that would greatly increase the chance we paused again later. At the moment, a major obstacle to people supporting this is their senses that it is impossible. Building the machinery to pause makes it much easier to pause again. Doing something once is in general extremely helpful for doing it later times—if you are taking the most important action in the world, you really don’t want to be trying it for the first time, when it matters.
The public substantially hates AI, but feels incredibly disempowered, because for instance they buy the story that technological ‘progress’ is inexorable. If they saw vividly how much power our institutions have to shape the course of technology, they would become more activated against AI.
Some of the models seem to be some amount dangerous, we can’t tell how dangerous the new ones are, they are improving very fast, and the point where we are confident that they are currently dangerous is very suboptimally late to start this project.
I believe it is likely that a pause would be a complete non-starter without at least a framework for what conditions trigger what actions, unfortunately. Metaphorically, it feels like we're in a bus changing our tires while driving down the freeway here, the driver(s) are saying "Hey, this is getting to be a really bad idea", and the folks who chartered the bus are shouting that they want to go faster, despite not really explaining to any of us where we're going, just that "It's going to be great, trust us!".
I often hear people say they think we should pause AI at some point, but not yet. Their basis for this seems to be some combination of:
If we pause at the last possible moment, then we will have the most advanced AI possible during the pause, which will be helpful for doing AI safety research during the pause
Implicitly, there is some quantity of ‘pausing credit’, that will buy us a few months of pause say, and if we use them now, we won’t have them to use later, when it is important
If we pause, and then AI doesn’t seem to be at dire risk of destroying the world, maybe the public will backlash against this and it will be harder to do any kind of AI safety (especially if it has major economic consequences)
The models aren’t dangerous yet
This all sounds very questionable to me. I suggest instead that the following are at least as likely to be true:
We can’t pause on a dime at the precise second that ‘we’ decide it is important to—pulling the breaks will take a while, during which time we will continue to rocket into danger.
If we managed to pause now, that would greatly increase the chance we paused again later. At the moment, a major obstacle to people supporting this is their senses that it is impossible. Building the machinery to pause makes it much easier to pause again. Doing something once is in general extremely helpful for doing it later times—if you are taking the most important action in the world, you really don’t want to be trying it for the first time, when it matters.
The public substantially hates AI, but feels incredibly disempowered, because for instance they buy the story that technological ‘progress’ is inexorable. If they saw vividly how much power our institutions have to shape the course of technology, they would become more activated against AI.
Some of the models seem to be some amount dangerous, we can’t tell how dangerous the new ones are, they are improving very fast, and the point where we are confident that they are currently dangerous is very suboptimally late to start this project.