I think this post makes a reasonable point and brings in relevant historical evidence, so I upvoted it. But I disagree with it.
The counterfactual isn't for them to do nothing though. IMO, someone working on duct taping together the RSI rocket at a frontier lab, should instead quit and warn the public about how dangerous RSI is, and how the things they were being asked to do at the lab amounted to duct tape that probably would fall off at some point in the future, plausibly in a catastrophic or hidden way. Even just a simple tweet (such as what Dan Selsam and Oleg Murk did) is probably better IMO than whatever they would have accomplished by adding more duct tape.
Perhaps this comes down to a disagreement about how likely this "we race through RSI at approximately maximum speed, but we do prosaic AI alignment which makes our AIs aligned enough for long enough that they can solve the rest of the problem for us" strategy is to work. I think it is unlikely to work.
I also think it is extremely reckless to the point of grave immorality for people to unilaterally impose this strategy on the rest of the world -- like Coxon said, "gambling with our lives." Instead, we should be demanding a better strategy in which AI development proceeds more cautiously and transparently.
1
Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism floating around from the left. The most prominent was a cluster centered around EAs being white men, and therefore suspicious.[1] Another category, in my experience the second-most common, was that in attempting to rectify the problems in the world, EA was masking, whitewashing, and misdirecting efforts away from solving the "real problem", usually capitalism but often other contradictory forces.
These arguments were often not specific to EA, but mainstream arguments against charitable giving in general. On any Hacker News post about a billionaire giving to charity, one of the first few comments would inevitably be this Oscar Wilde quote:
These arguments have a long history. Anticapitalist critiques of charity go back to Engels and Marx, who called charitable givers a bourgeois faction intent on "redressing social grievances in order to secure the continued existence of bourgeois society." In their most explicit form these critiques had the following shape:
While you could argue with these proposals on the object level, I think most people have an intuitive sense that this line of thinking is fraught, independent of whatever political program is mentioned. One general concern you might have is that if you're going to claim that the first-order effects of someone's good work are swamped by some second-order effect, particularly some second-order effect on optics, then the burden of proof is pretty high. And clearly, the people making these arguments do not have a good enough model of political economy to clear that bar. Not only is it not obvious that billionaire charity makes people like billionaires more, it is also not clear that the detractor's preferred intervention (namely, jettisoning capitalism) is effective at improving the world.
Recently, many people on LessWrong have made structurally similar arguments about prosaic AI safety work. Here are two such shortforms with >100 karma presenting this argument, though it has been raised in many other places. It generally goes like this:
The people who make these arguments make them in good faith. Nevertheless, I have both object-level and meta-level objections to their line of reasoning. My first object-level problem with these arguments is that I do think that prosaic AI safety efforts reduce the chance of runaway misaligned AI - even if they don't scale to superintelligence - and that the people making these arguments have confused theories of change. I've tried to articulate that disagreement in more detail here, but in no particular order:
My meta-level objection is that "AI safety of superintelligence" is a pre-paradigmatic field for which there are no feedback loops, and indeed no superintelligence, and so sacrifices to the greater good are even more speculative than when they're made in service of political ideologies. That doesn't mean that every approach is equal, and it certainly doesn't mean that attention is correctly allocated. But it does mean we should be cautious about encouraging others to abandon approaches that work on current models, if the counterfactual is for them to do nothing.
I am lampooning a bit, but not by much.
From "The Soul of Man Under Socialism". When I started writing this post I was under the mistaken impression that this quote was from Frederick Douglass. In My Bondage and My Freedom and other places, Douglass actually repeatedly claimed the opposite. I find it telling that the actual former slave who I thought wrote this quote believed that good treatment of slaves encouraged them to meditate on their own situation, and that the quote was instead written by a white socialist several decades after slavery had been abolished in the U.S. and Britain.
As an unstated corollary, the evil capitalists, such as the guys involved in the BP Oil Spill disaster, are the True Heroes of the revolution, inasmuch as they do not then use their wealth to cement political conditions.
As an unstated corollary, Elon Musk is currently the most effective altruist, because his model is very likely to breach containment in the near future, and he is not currently reducing time-to-AGI except indirectly through competitive pressure, because Grok is not at the frontier.