A response to someone asking about my criticisms of EA (crossposted from twitter):

EA started off with global health and ended up pivoting hard to AI safety, AI governance, etc. You can think of this as “we started with one cause area and we found another using the same core assumptions” but another way to think about it is “the worldview which generated ‘work on global health’ was wrong about some crucial things”, and the ideology hasn’t been adequately refactored to take those things out.

Some of them include:

1. Focusing on helping the worst-off instead of empowering the best
2. Being very scared of political controversy (e.g. not being willing to discuss the real causes of African poverty)
3. Not deeply believing in technological progress (e.g. prioritizing bed nets over vaccines). Related to point 1.
4. Trusting the international order to mostly have its shit together (related to point 2)

You can partially learn these lessons from within the EA framework but it’s very unnatural and you won’t learn them well enough. E.g. now EAs are pivoting to politics but again they’re flinching away from anything remotely controversial and so are basically just propping up existing elites.

On a deeper ideological level a lot of this is downstream of utilitarianism/consequentialism being wrong. Again hard to compress but a few quick points:

1. Utilitarianism primarily treats individuals as the relevant actors. But the whole point of ethics is to allow us to (safely) act as coherent coordinated groups instead.
2. Utilitarianism exacerbates the natural human tendency to flinch away from fears. It’s far easier to soothe your ego by finding a rationalization for having caused good consequences, than to self-deceive about whether you’re a paragon of courage or honesty.
3. On a technical level utilitarianism is extremely non-incentive-compatible. You’re basically asking for people to set up systems to exploit you, as long as they do so with enough levels of indirection that you can fool yourself into thinking you’re doing good (e.g. a dictator keeping a population as poor as possible, in plausibly-deniable ways, so utilitarians will keep sending aid which they can steal).

A lot of utilitarians will say “whether or not our strategy is bad, consequences are still the only thing that ultimately matter”. But this is like a Marxist saying to an economist “whether or not our strategy is bad, liberating the workers is the only thing that ultimately matters“ and then using that as an excuse to not learn economics. There *are* deep principles of (internal and external) cooperation, but utilitarianism is very effective in making people look away from them and towards power-seeking strategies.

A second tweet, in response to someone disagreeing with the Marxism analogy because many utilitarians follow principles too:

I think “maximize expected utility while obeying some constraints” looks very different from actually taking non-consequentialist decision procedures seriously.

In principle the utility-maximizing decision procedure might not involve thinking about “impact” at all.

And this is not even an insane hypothetical, IMO thinking about impact is pretty corrosive to one’s ability to do excellent research for example.

It’s hard to engage too deeply here because I think the notion of a “utility-maximizing decision procedure” is very underdefined on an individual level, and most of the action is in correlating one’s decisions with others. But my meta-level point is that it’s these kinds of complexities which utilitarians tend to brush under the rug by focusing their intellectual energy on criteria of rightness and then adding on some principles almost as an afterthought.