I’m pretty interested in conveying the costs of letting AI development proceed unchecked to the general public. In relation to this, I feel like I run into a lot of disagreements with people around me.
Misc very quickly written points on which my own impression has seemed very different from others in recent months:
1. It’s way, way, way more important that we stop dangerous AI soon than that we ensure that we don’t stop AI too much
I have the sense the sophisticated and worldly position here is to be very concerned about ‘over-regulation’ and about not doing exactly the right regulations. But it’s an obvious pragmatic fact that the stakes of overregulation are minuscule compared to the stakes of underregulating here. And the stakes of mis-regulation are very often going to be unintended slowing, which is in the same bucket.
2. If we stopped AI development, the risk of not starting up again for an extremely long time is low
This one just feels kind of crazy to me, like I may be missing something. In combination with the above point, it feels like if I met someone who was on fire, and they didn’t want me to put out the fire, in case they couldn’t figure out how to get warm again, ever. Like, forever is a long time, and if this thing is so great, you’re probably not going to forget about it! It’s like you think the current instantiation of yourself (or society) is so much more agentic than your future selves, that it’s worth huge costs to keep power in the current version’s hands.
3. Getting the public involved is not very dangerous
Some people seem to think of the public as like a kind of wild and crazy god one could awaken, with very high variance results. I think this doesn’t give them nearly enough credit. While I disagree with the public on quite a lot of points, I think disagreements are a lot more salient than agreements. We might disagree a lot on exactly how the FDA proceeds or who gets in how much trouble for doing what with which drugs. But we agree that broadly medical interventions and drugs are dangerous for people, and on wanting more safety around things that kill people, and I bet a lot of the details I don’t notice are better than nothing. And in a specific case where the default is insanely dangerous, bringing more people with power into the room, who are liable to stop things that are dangerous, and couldn’t really at this point make it much worse, seems pretty safe.
4. Public comms is not that rife with downside risks
There is a general sense that it is riskier to donate to groups hoping to talk to the public, because they can have large scale effects and make everyone look bad. So people donate their money to ‘safe bets’ such as e.g. technical safety. I think this is generally very miscalibrated. Talking to the public incompetently isn’t as far as I can tell much more likely to have reputational risks than doing other stuff incompetently, because everyone is in public. I don’t see many examples of this happening at a scale larger than the positive effects.
5. Pausing advanced AI soon greatly increases our chances of surviving
For one thing, I’d say the world’s degree of not giving a shit about safety in this case of advanced AI right now is extremely unique among issues, and if we wait for long enough for people to reckon with what’s happening at all, it is very unlikely to proceed in such wildly reckless way later on.
Also, I just expect us to make a lot of progress in understanding current AI models and organizing ourselves to not do crazy things if we have another decade to work out the kinks.
6. We don’t need a plan before we pause
People seem to think we do. I don’t know what’s up with that.
7. Raising the alarm is very important at the moment
I feel like people around me mostly agree that raising the alarm is good, but I’m surprised by the frequency of ambivalence about it. Partly this is to do with earlier points and related things, such as ‘what if people become too scared about AI and something bad happens’.
8. Working on marginally improving safety of models at the AI companies is not remotely competitive with raising the alarm at the moment
From my current vantage, it seems like people building AI do not mostly expect if AI left our control soon for it to be solidly aligned with a high probability. And they don’t seem to think we are close to that. So for a high chance of success, we really need to avoid that happening and we can make a lot of progress on that on the margin. Whereas if we know we are far from safety right now, adding marginal safety in a scenario where we are about to lose control seems pretty hopeless.
9. Doing advocacy doesn’t seem to have adverse effects on people doing it
I’ve heard a couple of times that people who do advocacy seem to go crazy. I don’t know of anything that seems like solid evidence of this, and weak or circumstantial evidence seems to only involve a handful of people.
And one where I feel like I haven’t heard so much disagreement lately:
10. It matters what the public think / political will is growable
I’ve been interested in talking to the public for a long time, and people have often disagreed with me. My sense is that they have often thought it seemed pointless. Because the public doesn’t have power. But I think the public does have power, and not just because they can ultimately vote. People care what other people think, including elites, and experts and people with power. Nobody is coming to their views in a vacuum. This matters.
I’m pretty interested in conveying the costs of letting AI development proceed unchecked to the general public. In relation to this, I feel like I run into a lot of disagreements with people around me.
Misc very quickly written points on which my own impression has seemed very different from others in recent months:
1. It’s way, way, way more important that we stop dangerous AI soon than that we ensure that we don’t stop AI too much
I have the sense the sophisticated and worldly position here is to be very concerned about ‘over-regulation’ and about not doing exactly the right regulations. But it’s an obvious pragmatic fact that the stakes of overregulation are minuscule compared to the stakes of underregulating here. And the stakes of mis-regulation are very often going to be unintended slowing, which is in the same bucket.
2. If we stopped AI development, the risk of not starting up again for an extremely long time is low
This one just feels kind of crazy to me, like I may be missing something. In combination with the above point, it feels like if I met someone who was on fire, and they didn’t want me to put out the fire, in case they couldn’t figure out how to get warm again, ever. Like, forever is a long time, and if this thing is so great, you’re probably not going to forget about it! It’s like you think the current instantiation of yourself (or society) is so much more agentic than your future selves, that it’s worth huge costs to keep power in the current version’s hands.
3. Getting the public involved is not very dangerous
Some people seem to think of the public as like a kind of wild and crazy god one could awaken, with very high variance results. I think this doesn’t give them nearly enough credit. While I disagree with the public on quite a lot of points, I think disagreements are a lot more salient than agreements. We might disagree a lot on exactly how the FDA proceeds or who gets in how much trouble for doing what with which drugs. But we agree that broadly medical interventions and drugs are dangerous for people, and on wanting more safety around things that kill people, and I bet a lot of the details I don’t notice are better than nothing. And in a specific case where the default is insanely dangerous, bringing more people with power into the room, who are liable to stop things that are dangerous, and couldn’t really at this point make it much worse, seems pretty safe.
4. Public comms is not that rife with downside risks
There is a general sense that it is riskier to donate to groups hoping to talk to the public, because they can have large scale effects and make everyone look bad. So people donate their money to ‘safe bets’ such as e.g. technical safety. I think this is generally very miscalibrated. Talking to the public incompetently isn’t as far as I can tell much more likely to have reputational risks than doing other stuff incompetently, because everyone is in public. I don’t see many examples of this happening at a scale larger than the positive effects.
5. Pausing advanced AI soon greatly increases our chances of surviving
For one thing, I’d say the world’s degree of not giving a shit about safety in this case of advanced AI right now is extremely unique among issues, and if we wait for long enough for people to reckon with what’s happening at all, it is very unlikely to proceed in such wildly reckless way later on.
Also, I just expect us to make a lot of progress in understanding current AI models and organizing ourselves to not do crazy things if we have another decade to work out the kinks.
6. We don’t need a plan before we pause
People seem to think we do. I don’t know what’s up with that.
7. Raising the alarm is very important at the moment
I feel like people around me mostly agree that raising the alarm is good, but I’m surprised by the frequency of ambivalence about it. Partly this is to do with earlier points and related things, such as ‘what if people become too scared about AI and something bad happens’.
8. Working on marginally improving safety of models at the AI companies is not remotely competitive with raising the alarm at the moment
From my current vantage, it seems like people building AI do not mostly expect if AI left our control soon for it to be solidly aligned with a high probability. And they don’t seem to think we are close to that. So for a high chance of success, we really need to avoid that happening and we can make a lot of progress on that on the margin. Whereas if we know we are far from safety right now, adding marginal safety in a scenario where we are about to lose control seems pretty hopeless.
9. Doing advocacy doesn’t seem to have adverse effects on people doing it
I’ve heard a couple of times that people who do advocacy seem to go crazy. I don’t know of anything that seems like solid evidence of this, and weak or circumstantial evidence seems to only involve a handful of people.
And one where I feel like I haven’t heard so much disagreement lately:
10. It matters what the public think / political will is growable
I’ve been interested in talking to the public for a long time, and people have often disagreed with me. My sense is that they have often thought it seemed pointless. Because the public doesn’t have power. But I think the public does have power, and not just because they can ultimately vote. People care what other people think, including elites, and experts and people with power. Nobody is coming to their views in a vacuum. This matters.