I think many AI safety people are not taking the appeal of political violence seriously, and thus addressing it inadequately. The arguments I've seen against political violence are about the ineffectiveness of individual acts of violence. They are not stepping into the shoes of people who really believe that political violence could be the solution; they neglect the natural next question of coordinated, larger-scale acts of violence. I argue that even these larger acts of violence are impractical[1].
I’m thinking about a conversation I had last night, in which my friend claimed (in a motivated way) that it must be impossible to build ASI. He felt that if he believed ASI[2] could actually be achieved, then he would want to kill everyone working on building it. His implicit argument was as follows:
ASI will be built
If ASI is built, this will almost certainly be terrible for humanity.
If something will be terrible for humanity, you should do everything you can to stop it from happening.
Political violence stops things from happening for at least two reasons: ordinary people will take the problem more seriously and people currently working on making the thing happen will become too scared to continue.
So, people who believe ASI will be built should resort to political violence (e.g. killing researchers and bombing data centers).
Underlying his words, I sensed fear. If it is possible to build ASI, then the consequences would be unthinkable; the actions I would feel an impulse to do would be unthinkable. So, I cannot believe that it is possible to build ASI.
* * *
I don't feel an impulse to violence, so my friend's words surprised me. I think many people with strong moral principles, like LessWrongers or AI safety people tend to be, also don't feel an impulse to violence, which is why we don't treat political violence like a serious option. However, I think treating it as a serious option, as evidently some people do, is necessary to respond to it thoroughly. Maybe there are some ragebait-y trolls on the internet, but my friend’s concern is real; my friend’s instinct to violence is real.
Maybe examples would illustrate what I mean when I say we don't treat political violence like a serious option. Zvi begins “Political Violence Is Never Acceptable” with:
Political Violence Is Never Acceptable
Nor is the threat or implication of violence. Period. Ever. No exceptions.
And, in the last section, he says:
I condemn these attacks, and any and all such violence against anyone, in the strongest possible terms. I do this both because it is immoral, and also because it is illegal, and also because it wouldn’t work. Nothing hurts your cause more.
Is it really true that political violence, or even the threat of violence, is never, ever acceptable? I think not. Even in Eliezer's “Only Law Can Prevent Extinction”, Eliezer acknowledges that violence is sometimes necessary, because the law itself rests on violence.
The sort of absolute language and hyperbolic rhetoric that Zvi uses does not actually address a real, substantive disagreement. If I believed that political violence could be effective, I would not feel like my beliefs were adequately addressed after reading his post.
In Eliezer's piece (which I otherwise think is pretty good and discuss below), in order to illustrate how pointless political violence is as an attempt to make AI more safe, he uses the analogy of killing puppies as an attempt to cure your child of cancer:
How certain do you have to be that your child has terminal cancer, before you start killing puppies? 10% sure? 50% sure? 99.9%? The answer is that it doesn't matter how certain you are, killing puppies doesn't cure cancer. You can kill one hundred puppies and still not save your kid. There is no sin so great that it just has to be helpful because of how sinful it is.
This is a poor analogy. It is much more reasonable to think that killing an AI researcher would meaningfully slow AI development (perhaps by instilling fear and halting important work) than it is to believe that killing puppies will cure your child of cancer. We cannot afford poor, inflammatory analogies here. We can argue more generously and we can think more clearly. We can do better.
* * *
Many people have already argued that individual acts of violence, whether against infrastructure (like bombing data centers) or people (like lab CEOs), will not slow down AI development because there is simply too much infrastructure and too many people. For example, Eliezer writes:
And similarly: To impede one executive, one researcher, or one company, does not change where AI is heading.
...
There is no one researcher who holds the secret to your death. They are all looking for pieces of the puzzle to accumulate, for individual rewards of fame and fortune. If somehow the person who was to find the next piece of the puzzle randomly choked on a chicken bone, somebody else would find a different puzzle piece a few months later, and Death would march on. AI researchers tell themselves that even if they gave up their enormous salaries, that wouldn’t help humanity much, because other researchers would just take their place. And the grim fact is that this is true, whether or not you consider it an excuse.
Recalling our conversation yesterday, I think both my friend and I agree that individual violence is ineffective. At best, individual political violence is ineffective and a small-scale tragedy; at worst, it harms the movement and is an eventual large-scale catastrophe.
Taking the argument in favor of political violence seriously though, one might ask: if there are so many people who care about AI safety, why don't you guys coordinate to kill all the researchers and executives, to dismantle all the companies, to bomb all the data centers? Or even to kill a large portion? Ultimately, the death of some millions of researchers should be outweighed by even a 1% chance of human extinction, right?
It’s true that if I woke up and saw that every AI researcher — even one out of every five researchers — was dead, I would be terrified of continuing my work. AI development would be paused for now. There are other questions, of course. Would there be enough impetus to legislate AI development? Would this make people take x-risk seriously? How long would it take new, ambitious, now well-defended entrepreneurs to resume development? But if I had a button that I knew would indefinitely pause AI development at the cost of millions of lives, I can see the argument for clicking the button.
However, this button hypothetical is beside the point. It’s hard for me to imagine the reaction to this counterfactual because I cannot imagine a world in which people manage to coordinate at a scale this large and go undetected. Practically, it seems nearly impossible to achieve the coordination necessary to destroy enough data centers or kill enough researchers to make an impact on AI progress. For one, these actions would have to be carried out over a very short span of time, probably shorter than one night, in order not to be noticed before companies beef up their security. For another, one person can only kill so many people in a day, so you would have to get thousands of people on board with mass murder (or at least mass property destruction) and coordinate with them within this short time-frame. Finally, you’d have to do all this without someone whistleblowing, or something else going awry.
In fact, it seems like large-scale political violence is more difficult to coordinate than a global ASI ban, which Eliezer argues for. Even though a global ASI ban is often treated as an idealistic goal, I argue that it's significantly easier to make happen than any sort of impactful political violence. It would require immense coordination, to be sure, but not nearly as much as the violence would require. Also, the coordination for a global ban wouldn’t have to be secret, as it would be for violence. Furthermore, a global ASI ban is something we can progress to through smaller laws, whereas any sort of effective political violence would have to happen all at once. Really, compared to effective political violence, a global ASI ban looks so doable.
* * *
My argument is not that we should strive for a global ASI ban. I’m not sure I agree with Eliezer on this. My point is just even when we treat political violence as a serious option, it is still impractical. It takes a lot of effort for political violence to have meaningful impact — more effort than would be required by other more certain, more lawful paths, even ones as improbable as a global ASI ban.
This is what I want to tell my friend. Violence may be the instinctual first solution that comes to your mind. However, the most effective solutions are usually not instinctual. Rather, they’re born of thought and careful consideration.
Your desire for action is useful, but only if it’s directed towards useful goals. Even though political violence is not a useful goal, there are so many actually useful goals that we need you for. There are policies to be made, experiments to be run, people to call.
Even though I don’t feel as strong an instinct for violence as you do, I can empathize with the fear that fuels this instinct. It makes total sense to be terrified by the idea of ASI. But this fear should not prevent you from believing that ASI is possible. When I think about AI safety, I sometimes feel small in the face of this massive problem. What can one person do? But I remind myself that there is more than one person who cares enough to work on this problem. And you — you could be one more.
I also think violence is immoral, but that's beside the point. I'm taking the perspective of someone who believes that the immorality of violence is outweighed by other factors, like the scope of existential risk.
When I say ASI, I mean something like Bostrom’s definition: “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.” I find this definition dissatisfying because I don’t know how you would measure it directly, but it suffices for this point I want to make in this post.
I think many AI safety people are not taking the appeal of political violence seriously, and thus addressing it inadequately. The arguments I've seen against political violence are about the ineffectiveness of individual acts of violence. They are not stepping into the shoes of people who really believe that political violence could be the solution; they neglect the natural next question of coordinated, larger-scale acts of violence. I argue that even these larger acts of violence are impractical[1].
I’m thinking about a conversation I had last night, in which my friend claimed (in a motivated way) that it must be impossible to build ASI. He felt that if he believed ASI[2] could actually be achieved, then he would want to kill everyone working on building it. His implicit argument was as follows:
Underlying his words, I sensed fear. If it is possible to build ASI, then the consequences would be unthinkable; the actions I would feel an impulse to do would be unthinkable. So, I cannot believe that it is possible to build ASI.
* * *
I don't feel an impulse to violence, so my friend's words surprised me. I think many people with strong moral principles, like LessWrongers or AI safety people tend to be, also don't feel an impulse to violence, which is why we don't treat political violence like a serious option. However, I think treating it as a serious option, as evidently some people do, is necessary to respond to it thoroughly. Maybe there are some ragebait-y trolls on the internet, but my friend’s concern is real; my friend’s instinct to violence is real.
Maybe examples would illustrate what I mean when I say we don't treat political violence like a serious option. Zvi begins “Political Violence Is Never Acceptable” with:
And, in the last section, he says:
Is it really true that political violence, or even the threat of violence, is never, ever acceptable? I think not. Even in Eliezer's “Only Law Can Prevent Extinction”, Eliezer acknowledges that violence is sometimes necessary, because the law itself rests on violence.
The sort of absolute language and hyperbolic rhetoric that Zvi uses does not actually address a real, substantive disagreement. If I believed that political violence could be effective, I would not feel like my beliefs were adequately addressed after reading his post.
In Eliezer's piece (which I otherwise think is pretty good and discuss below), in order to illustrate how pointless political violence is as an attempt to make AI more safe, he uses the analogy of killing puppies as an attempt to cure your child of cancer:
This is a poor analogy. It is much more reasonable to think that killing an AI researcher would meaningfully slow AI development (perhaps by instilling fear and halting important work) than it is to believe that killing puppies will cure your child of cancer. We cannot afford poor, inflammatory analogies here. We can argue more generously and we can think more clearly. We can do better.
* * *
Many people have already argued that individual acts of violence, whether against infrastructure (like bombing data centers) or people (like lab CEOs), will not slow down AI development because there is simply too much infrastructure and too many people. For example, Eliezer writes:
Recalling our conversation yesterday, I think both my friend and I agree that individual violence is ineffective. At best, individual political violence is ineffective and a small-scale tragedy; at worst, it harms the movement and is an eventual large-scale catastrophe.
Taking the argument in favor of political violence seriously though, one might ask: if there are so many people who care about AI safety, why don't you guys coordinate to kill all the researchers and executives, to dismantle all the companies, to bomb all the data centers? Or even to kill a large portion? Ultimately, the death of some millions of researchers should be outweighed by even a 1% chance of human extinction, right?
It’s true that if I woke up and saw that every AI researcher — even one out of every five researchers — was dead, I would be terrified of continuing my work. AI development would be paused for now. There are other questions, of course. Would there be enough impetus to legislate AI development? Would this make people take x-risk seriously? How long would it take new, ambitious, now well-defended entrepreneurs to resume development? But if I had a button that I knew would indefinitely pause AI development at the cost of millions of lives, I can see the argument for clicking the button.
However, this button hypothetical is beside the point. It’s hard for me to imagine the reaction to this counterfactual because I cannot imagine a world in which people manage to coordinate at a scale this large and go undetected. Practically, it seems nearly impossible to achieve the coordination necessary to destroy enough data centers or kill enough researchers to make an impact on AI progress. For one, these actions would have to be carried out over a very short span of time, probably shorter than one night, in order not to be noticed before companies beef up their security. For another, one person can only kill so many people in a day, so you would have to get thousands of people on board with mass murder (or at least mass property destruction) and coordinate with them within this short time-frame. Finally, you’d have to do all this without someone whistleblowing, or something else going awry.
In fact, it seems like large-scale political violence is more difficult to coordinate than a global ASI ban, which Eliezer argues for. Even though a global ASI ban is often treated as an idealistic goal, I argue that it's significantly easier to make happen than any sort of impactful political violence. It would require immense coordination, to be sure, but not nearly as much as the violence would require. Also, the coordination for a global ban wouldn’t have to be secret, as it would be for violence. Furthermore, a global ASI ban is something we can progress to through smaller laws, whereas any sort of effective political violence would have to happen all at once. Really, compared to effective political violence, a global ASI ban looks so doable.
* * *
My argument is not that we should strive for a global ASI ban. I’m not sure I agree with Eliezer on this. My point is just even when we treat political violence as a serious option, it is still impractical. It takes a lot of effort for political violence to have meaningful impact — more effort than would be required by other more certain, more lawful paths, even ones as improbable as a global ASI ban.
This is what I want to tell my friend. Violence may be the instinctual first solution that comes to your mind. However, the most effective solutions are usually not instinctual. Rather, they’re born of thought and careful consideration.
Your desire for action is useful, but only if it’s directed towards useful goals. Even though political violence is not a useful goal, there are so many actually useful goals that we need you for. There are policies to be made, experiments to be run, people to call.
Even though I don’t feel as strong an instinct for violence as you do, I can empathize with the fear that fuels this instinct. It makes total sense to be terrified by the idea of ASI. But this fear should not prevent you from believing that ASI is possible. When I think about AI safety, I sometimes feel small in the face of this massive problem. What can one person do? But I remind myself that there is more than one person who cares enough to work on this problem. And you — you could be one more.
I also think violence is immoral, but that's beside the point. I'm taking the perspective of someone who believes that the immorality of violence is outweighed by other factors, like the scope of existential risk.
When I say ASI, I mean something like Bostrom’s definition: “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.” I find this definition dissatisfying because I don’t know how you would measure it directly, but it suffices for this point I want to make in this post.