I think there is some value in exploring the philosophical foundations of ethics, and LessWrong culture is often up for that sort of thing. But, it's worth saying explicitly: the taboo against violence is correct, and has strong arguments for it from a wide variety of angles. People who think their case is an exception are nearly always wrong, and nearly always make things worse.
(This does not include things that could be construed as violence but only if you stretch the definition, like supporting regulation through normal legal channels, or aggressive criticism, or lawsuits. I think those things are not taboo and would support some of them.)
Consider the following rhetorical question:
Ethical vegans are annoyed when people suggest their rhetoric hints at violence against factory farms and farmers. But even if ethical vegans don't advocate violence, it does seem like violence is the logical conclusion of their worldview - so why is it a taboo?
Do we expect the answer to this to be any different for vegans than for AI-risk worriers?
You can imagine an argument that goes "Violence against AI labs is justified in spite of the direct harm it does, because it would prevent progress towards AGI." I have only ever heard people say that someone else's views imply this argument, and never actually heard someone actually advance this argument sincerely; nevertheless the hypothetical argument is at least coherent.
Yudkowsky's position is that the argument above is incorrect because he denies the premise that using violence in this way would actually prevent progress towards AGI. See e.g. here and the following dialogue. (I assume he also believes in the normal reasons why clever one-time exceptions to the taboo against violence are unpersuasive.)
I addressed a general question like that in https://www.lesswrong.com/posts/p2Qq4WWQnEokgjimy/respect-chesterton-schelling-fences
Basically, guardrails exist for a reason, and you are generally not smart enough to predict the consequences of removing them. This applies to most suggestions of the form "why don't we just <do some violent thing> to make the world better". There are narrow exceptions where breaking a guardrail has actual rather than imaginary benefits, but finding them requires a lot of careful analysis and modeling.
Umm, because individual non-government-sanctioned violence is horrific, and generally results in severe punishment which prevents longer-term action. Oh, wait, that's why it's not used, not why it's taboo to even disuss.
It's taboo for discussion because serious planning for violence is a direct crime (conspiracy) itself. Don't do that. Open advocacy of violence also signals that, by your rules, it's OK for others to target you for violence if they disagree strongly enough. I don't recommend that, either (especially if you think your opponents are better at violence than you are).
Because it's anti-social (in most cases; things like law enforcement are usually fine), and the only good timelines (by any metric) are pro-social.
Consider if it became like the Irish troubles. Do you think alignment gets solved in this environment? No. What you get is people creating AI war machines. And they don't bother with alignment because they are trying to get an advantage over the enemy, not benefit everyone. Everyone is incentivised to push capabilities as far as they can, except past the singularity threshold. And there's not even a disincentive for going past it, you're just neutral on it. So the dangerous bit isn't even that the AI are war machines, it's that they are unaligned.
It's a general principle that anti-social acts tend to harm utility overall due to second-order effects that wash out the short-sighted first-order effects. Alignment is an explicitly pro-social endeavor!
I think violence helps unaligned AI more than it helps aligned AI.
If the research all goes underground it will slow it down but it will also make it basically guaranteed that there's a competitive, uncoordinated transition to superintelligence.
When Eliezer proposes "turn all the GPUs to Rubik's cubes", this pivotal act I think IS outright violence. Nanotechnology doesn't work that way (something something local forces dominate). What DOES work is having nearly unlimited drones because they were manufactured by robots that made themselves exponentially, making ASI equipped parties have more industrial resources than the entire worlds capacity right now.
Whoever has "nearly unlimited drones" is a State, and is committing State Sponsored Violence which is OK. (By the international law of "whatcha gonna do about it")
So the winners of an AI race with their "aligned" allied superintelligence actually manufactured enough automated weapons to destroy everyone else's AI labs and to place the surviving human population under arrest.
That's how an AI war actually ends. If this is how it goes (and remember this is a future humans "won") this is what happens.
The amount of violence before the outcome depends on the relative resources of the warring sides.
ASI singleton case : nobody has to be killed, billions of drones using advanced technology attack everywhere on the planet at once. Decision makers are bloodlessly placed under arrest, guards are tranquilized, the drones have perfect aim so guns are shot out of hands and engines on military machines hit with small shaped charges. The only violence where humans die is in the assaults on nuclear weapons facilities, since math.
Some nukes may be fired on the territory of the nation hosting the ASI, this kills a few million tops, "depending on the breaks".
Two warring parties case, one party's ASI or industrial resources are significantly weaker : nuclear war and prolonged endless series of battles between drones. Millions or billions of humans killed as collateral damage, battlefields littered with nuclear blast craters and destroyed hardware. "Minor inconvenience" for the winning side since they have exponentially built robotics, the cleanup is rapid.
Free for all, everyone gets ASI, it's not actually all that strong in utility terms : Outcomes range from a world of international treaties similar to now and a stable equilibria or a world war that consumes the earth, most humans don't survive. Again, it's a minor inconvenience for the winners. No digital data is lost, exponentially replicated robotics mean the only long term cost is a few years to clean up.
If someone thinks that violence against AI labs is bad, then they will make it a taboo because they think it is bad, and they don't want violent ideas to spread.
There are a lot of interesting discussions to be had on why one believes this category of violence to be bad, and you can argue against these perspectives in a fairly neutral-sounding, non-stressful way, quite easily, if you know how to phrase yourself well.
A lot (although not all) people are fairly open to this.
If someone thinks that violence against AI labs is good, then they probably really wouldn't want you talking about it on a publicly accessible, fairly well-known website. It's a very bad strategy from most pro-violence perspectives.
I'm going to quite strongly suggest, regardless of anyone's perspectives on this topic, that you probably shouldn't discuss it here - there are very few angles from which this could be imagined to be a good thing for any rationalism-associated person/movement. Or at least that you put a lot of thought into how you talk about it. Optics are a real and valuable thing, as annoying as that is.
Even certain styles of discussing anti-violence can come across as optically weird if you phrase yourself in certain ways.
Perhaps they prefer not to be held responsible when it happens
People like Eliezer are annoyed when people suggest their rhetoric hints at violence against AI labs and researchers. But even if Eliezer & co don't advocate violence, it does seem like violence is the logical conclusion of their worldview - so why is it a taboo?
(By violence I don't necessarily mean physical violence - I mean more general efforts to disrupt AI progress, including e.g. with coordinated cyber attacks.)