Are we living in a simulation? No. But it is surreal that effective altruism has become the recent target of the official account for the Office of the Under Secretary of War for Research and Engineering.
It’s probably the first time many have even encountered the term effective altruism. A movement emerged to try to improve the world by donating some excess wealth to improve animal welfare in factory farms, improve global health, with an eye to effectiveness of the institutions money is donated to. Something so trivial nobody should really find themselves opposed towards. Unless they oppose any and all philanthropic activities. So what is going on?
In the race for global AI dominance, economically, militarily, and so forth, some people view any attempts to slow down AI development for ethical reasons, to ensure that large language models, for instance, aren’t racist, or that superintelligence could be dangerous, as problematic in themselves. The background assumption is a winner-takes-all mentality according to which whatever nation gets to superintelligent AI first gains global dominance. Viewed from this mentality, one can make sense of these surreal X posts. If you think it’s all one big race between China and the US that will determine the fate of this planet, one can make some sense of these tweets. But that’s quite a radical position to hold. Then again, politicians and the general public are prone to think of world problems as zero-sum games, where there can only be one winner and everyone else is a loser.
X has seen thousands of tweets over the last week about effective altruism, much of it hostile and focused on its alleged “AI doomerism.” Many effective altruists are worried that AI might have catastrophic effects, all the way from human extinction to milder ones like enabling terrorists to create bioweapons. That is true. But I’ve been puzzled by the repeated assertions by prominent people that effective altruists are an “AI doomer cult.”
I agree with many critics that it’s extremely speculative to try to evaluate the risks of AI. And different people in the AI risk debates, whether or not they are affiliated with the effective altruism movement, will have very different estimates that will appear absurd to others. We face radical uncertainty.
Personally, I do not spend much time at all thinking about the risk of human extinction due to AI, except for how AI might impact animal welfare, especially on farms. I have better things to do and leave that question to the experts. But it’s incredibly puzzling to see public intellectuals make confident assertions about the absurdity of some views when they have virtually no background in the topic. It reminded me a bit of the satirical movie Don’t Look Up.
People who know little to nothing about AI try to smear people as idiots or cultists. If I tried to rationalize this, I think the best answer is something along the lines of: it doesn’t matter if humanity goes extinct, because it’s not worth living in a world where China wins the AI race. It’s reminiscent of the totalist view of the Nazis that it doesn’t matter if all Germans die in a world war, since there is only one world worth living in: a world in which Germans win. And yet, I don’t think this is what we should call the most “pro-Germany policy.”
The zero-sum mindset is a cognitive bias we all share, and it is a dangerous one, for it stifles cooperation, “even in situations where cooperation is a matter of life or death.”1 If you want good political decision-making, you should make sure that actors do not suffer from a zero-sum mindset.
Another distinguishing feature of effective altruists, in their attempts to counter cognitive biases, is a deliberate emphasis on countering our human psychology, which tends to disregard tiny risks or to treat them as almost certain. In general, humans are pretty awful at probabilistic reasoning.2 So it’s good to try and study these risks in whatever way we can to turn uncertainty into manageable risk.
Another criticism of effective altruists is that they’ve gone all in on crazy longtermism, only thinking about the long-term future and AI risk, but neglecting the present. That’s far from the truth.
Luke Moore, Senior Partnerships Manager at Giving What We Can tried to break down the Effective Altruism Funding environment in a post on the Effective Altruism Forum for 2026:
~50% to global health
~30% to global catastrophic risk reduction (up from ~20% in 2024)
~10% to animal welfare
~5% to climate change mitigation
(the rest is unknown or a bit harder to categorise: ~5%)
I, for one, think it’s great that someone is concerned with global catastrophic risks. If it wasn’t for EAs, there would be very little funding available to actually do serious research in this area. But my attitude is really a risk-balancing one here. You might think that longtermism, roughly, the view that we should not discriminate against future humans (or animals) just because they will live in the future, is a crazy philosophical view. If there are billions of humans who could live in the future and we ought to take them into account, our political decisions might look radically different. But humans, as a species, and our institutions along with them, are so ridiculously short-termist that it hardly matters if longtermism as a philosophy is mistaken or ridiculous. They offer a good counterweight.
However, in the AI space, you might think that this is not true. Perhaps, as critics lament, longtermists/effective altruists (the latter does not imply the former) dominate this discourse. I am skeptical. EAs share visible platform, show stransparency, and seek out counter-arguments, which makes them highly visible in the space. But let’s grant they are actually overrepresented. Yet again, effective altruists are generally not AI doomers. They are just trying to estimate what the actual risks are. Likely, many are overconfident in their own assessments. (So is everyone.) Some will no doubt appear like the “crazy” performance Leonardo DiCaprio gave in Don’t Look Up, when others sought to downplay these risks entirely. But so what?
I think some of the estimates given by researchers are blatatently absurd and outrageous in their confidence. But so what? None of that is a reason to ignore low-probability risks. People confuse individual assessments for communities of epistemic peers trying to get at the truth.
Overconfidence in these estimates by individuals is not worse (however ludicrous they may be) than collective ignorance towards unknown risks altogether.
Disagreement is necessary for healthy epistemic communities. And effective altruists disagree all the time. And that’s good! They literally come from all spheres of the political compass, which is why they are also perhaps targetted from all sides:
People often judge movements based on the views of one representative. But EAs are incredibly diverse in their views. They only uniform thing one can really find between them, is that it would be great if everyone in a position to do so donated at least 10% of their income to philanthropic causes, like animal welfare improvements in factory farms, with an eye to actual effectiveness.
Perhaps because we are so used to political tribalism that reduces diversity of views, it’s easy to use the individual assessments of individuals as representative claims for entire movements. But that’s just another failure of reasoning, even confined to the domain of politics. Just like zero-sum thinking and probability neglect it’s just intellectually lazy and a sign of ignorance.
Andrews Fearon, P., & Götz, F. M. (2024). The zero-sum mindset. Journal of Personality and Social Psychology, 127(4), 758–795. https://doi.org/10.1037/pspa0000404
Zero-sum thinking, probability neglect, and other failures of reasoning
crosspost
Are we living in a simulation? No. But it is surreal that effective altruism has become the recent target of the official account for the Office of the Under Secretary of War for Research and Engineering.
It’s probably the first time many have even encountered the term effective altruism. A movement emerged to try to improve the world by donating some excess wealth to improve animal welfare in factory farms, improve global health, with an eye to effectiveness of the institutions money is donated to. Something so trivial nobody should really find themselves opposed towards. Unless they oppose any and all philanthropic activities. So what is going on?
In the race for global AI dominance, economically, militarily, and so forth, some people view any attempts to slow down AI development for ethical reasons, to ensure that large language models, for instance, aren’t racist, or that superintelligence could be dangerous, as problematic in themselves. The background assumption is a winner-takes-all mentality according to which whatever nation gets to superintelligent AI first gains global dominance. Viewed from this mentality, one can make sense of these surreal X posts. If you think it’s all one big race between China and the US that will determine the fate of this planet, one can make some sense of these tweets. But that’s quite a radical position to hold. Then again, politicians and the general public are prone to think of world problems as zero-sum games, where there can only be one winner and everyone else is a loser.
X has seen thousands of tweets over the last week about effective altruism, much of it hostile and focused on its alleged “AI doomerism.” Many effective altruists are worried that AI might have catastrophic effects, all the way from human extinction to milder ones like enabling terrorists to create bioweapons. That is true. But I’ve been puzzled by the repeated assertions by prominent people that effective altruists are an “AI doomer cult.”
I agree with many critics that it’s extremely speculative to try to evaluate the risks of AI. And different people in the AI risk debates, whether or not they are affiliated with the effective altruism movement, will have very different estimates that will appear absurd to others. We face radical uncertainty.
Personally, I do not spend much time at all thinking about the risk of human extinction due to AI, except for how AI might impact animal welfare, especially on farms. I have better things to do and leave that question to the experts. But it’s incredibly puzzling to see public intellectuals make confident assertions about the absurdity of some views when they have virtually no background in the topic. It reminded me a bit of the satirical movie Don’t Look Up.
People who know little to nothing about AI try to smear people as idiots or cultists. If I tried to rationalize this, I think the best answer is something along the lines of: it doesn’t matter if humanity goes extinct, because it’s not worth living in a world where China wins the AI race. It’s reminiscent of the totalist view of the Nazis that it doesn’t matter if all Germans die in a world war, since there is only one world worth living in: a world in which Germans win. And yet, I don’t think this is what we should call the most “pro-Germany policy.”
The zero-sum mindset is a cognitive bias we all share, and it is a dangerous one, for it stifles cooperation, “even in situations where cooperation is a matter of life or death.”1 If you want good political decision-making, you should make sure that actors do not suffer from a zero-sum mindset.
Another distinguishing feature of effective altruists, in their attempts to counter cognitive biases, is a deliberate emphasis on countering our human psychology, which tends to disregard tiny risks or to treat them as almost certain. In general, humans are pretty awful at probabilistic reasoning.2 So it’s good to try and study these risks in whatever way we can to turn uncertainty into manageable risk.
Another criticism of effective altruists is that they’ve gone all in on crazy longtermism, only thinking about the long-term future and AI risk, but neglecting the present. That’s far from the truth.
Luke Moore, Senior Partnerships Manager at Giving What We Can tried to break down the Effective Altruism Funding environment in a post on the Effective Altruism Forum for 2026:
I, for one, think it’s great that someone is concerned with global catastrophic risks. If it wasn’t for EAs, there would be very little funding available to actually do serious research in this area. But my attitude is really a risk-balancing one here. You might think that longtermism, roughly, the view that we should not discriminate against future humans (or animals) just because they will live in the future, is a crazy philosophical view. If there are billions of humans who could live in the future and we ought to take them into account, our political decisions might look radically different. But humans, as a species, and our institutions along with them, are so ridiculously short-termist that it hardly matters if longtermism as a philosophy is mistaken or ridiculous. They offer a good counterweight.
However, in the AI space, you might think that this is not true. Perhaps, as critics lament, longtermists/effective altruists (the latter does not imply the former) dominate this discourse. I am skeptical. EAs share visible platform, show stransparency, and seek out counter-arguments, which makes them highly visible in the space. But let’s grant they are actually overrepresented. Yet again, effective altruists are generally not AI doomers. They are just trying to estimate what the actual risks are. Likely, many are overconfident in their own assessments. (So is everyone.) Some will no doubt appear like the “crazy” performance Leonardo DiCaprio gave in Don’t Look Up, when others sought to downplay these risks entirely. But so what?
I think some of the estimates given by researchers are blatatently absurd and outrageous in their confidence. But so what? None of that is a reason to ignore low-probability risks. People confuse individual assessments for communities of epistemic peers trying to get at the truth.
Overconfidence in these estimates by individuals is not worse (however ludicrous they may be) than collective ignorance towards unknown risks altogether.
Disagreement is necessary for healthy epistemic communities. And effective altruists disagree all the time. And that’s good! They literally come from all spheres of the political compass, which is why they are also perhaps targetted from all sides:
People often judge movements based on the views of one representative. But EAs are incredibly diverse in their views. They only uniform thing one can really find between them, is that it would be great if everyone in a position to do so donated at least 10% of their income to philanthropic causes, like animal welfare improvements in factory farms, with an eye to actual effectiveness.
Perhaps because we are so used to political tribalism that reduces diversity of views, it’s easy to use the individual assessments of individuals as representative claims for entire movements. But that’s just another failure of reasoning, even confined to the domain of politics. Just like zero-sum thinking and probability neglect it’s just intellectually lazy and a sign of ignorance.
--------------
1
Andrews Fearon, P., & Götz, F. M. (2024). The zero-sum mindset. Journal of Personality and Social Psychology, 127(4), 758–795. https://doi.org/10.1037/pspa0000404
2
See Sunstein, C. R. (2002). Probability Neglect: Emotions, Worst Cases, and Law. The Yale Law Journal, 112(1), 61–107. https://doi.org/10.2307/1562234