Review
Review
Saving the world sucks
51Rafael Harth
44Vanessa Kosoy
13LawrenceC
9Vanessa Kosoy
2Richard_Kennaway
9Vanessa Kosoy
3Richard_Kennaway
35AnthonyC
22sapphire
6Chris_Leong
5sapphire
5Karthik Tadepalli
21Chris_Leong
1Causal Chain
2Chris_Leong
15simon
10Ape in the coat
5Joe Collman
5quetzal_rainbow
4mako yass
2ProgramCrafter
1Karthik Tadepalli
1yanni kyriacos
1yanni kyriacos
1StartAtTheEnd
3Jiro
1StartAtTheEnd
0[anonymous]
0Mitchell_Porter
New Comment
I can't really argue against this post insofar as it's the description of your mental state, but it certainly doesn't apply to me. I became way happier after trying to save the world, and I very much decided to try to save the world because of ethical considerations rather than because that's what I happened to find fun. (And all this is still true today.)
I agree with the OP that: Utilitarianism is not a good description of most people's values, possibly not even a good description of anyone's values. Effective altruism encourages people to pretend that they are intrinsically utilitarian, which is not healthy or truth-seeking. Intrinsic values are (to 1st approximation) immutable. It's healthy to understand your own values, it's bad to shame people for having "wrong" values.
I agree with critics of the OP that: Cooperation is rational, we should be trying to help each other over and above the (already significant) extent to which we intrinsically care about each other, because this is in our mutual interest. A healthy community rewards prosocial behavior and punishes sufficiently antisocial behavior (there should also be ample room for "neutral" though).
A point insufficiently appreciated by either: The rationalist/EA community doesn't reward prosocial behavior enough. In particular, we need much more in the way of emotional support and mental health resources for community members. I speak from personal experience here: I am very grateful to this community for support in the career/professional sense. However, on the personal/emotional level, I never felt that the community cares about what I'm going through.
2I agree with most of the points you're making here.
The rationalist/EA community doesn't reward prosocial behavior enough.
I think there's a continued debate about whether these groups should behave more like a professional circle or as a social community. (In practice, both spheres are a bit of both.) I think from the lens of EA/rats as a social group, we don't really provide enough emotional support and mental health resources. However, insofar as EA is intended to be a professional circle trying to do hard things, it makes sense why these resources might be deprioritized.
There is tension between the stance that "EA is just a professional circle" and the (common) thesis that EA is a moral ideal. The latter carries the connotation of "things you will be rewarded for doing" (by others sharing the ideal). Likely some will claim that, in their philosophy, there is no such connotation: but it is on them to emphasize this, since this runs contrary to the intuitive perception of morality by most people. People who take up the ideology expecting the implied community aspect might understandably feel disappointed or even betrayed when they find it lacking, which might have happened to the OP.
As I said, cooperation is rational. There are, roughly speaking, two mechanisms to achieve cooperation: the "acausal" way and the "causal" way. The acausal way means doing something out of abstract reasoning that, if many others do the same, it will be in everyone's benefit, and moreover many others follow the same reasoning. This might work even without a community, in principle.
However, the more robust mechanism is causal: tit-for-tat. This requires that other people actually reward you for doing the thing. One way to reward is by money, which EA does to some extent: however, it also encourages members to take pay cuts and/or make donations. Another way to reward is by the things money cannot buy: respect, friendship, emotional support and generally conveying the sense that you're a cherished member of the community. On this front, more could be done IMO.
Even if we accept that EA is nothing more than a professional circle, it is still lacking in the respects I pointed out. In many professional circles, you work in an office with peers, leading naturally to a network of personal connections. On the other hand, AFAICT many EAs work independently/remotedly (I am certainly one of those), which denies the same benefits.
There is tension between the stance that "EA is just a professional circle" and the (common) thesis that EA is a moral ideal.
It is a professional circle founded on a moral ideal. The former to be Effective, the latter to be Altruistic.
The latter carries the connotation of "things you will be rewarded for doing" (by others sharing the ideal).
It is an old saying that virtue is its own reward.
"Virtue is its own reward" is a nice thing to believe in when you feel respected, protected and loved. When you feel tired, lonely and afraid, and nobody cares at all, it's very hard to understand why you should be making big sacrifices for the sake of virtue. But, hey, people are different. Maybe, for you virtue is truly, unconditionally, its own reward, and a sufficient one at that. And maybe EA is a community professional circle only for people who are that stoic and selfless. But, if so, please put the warning in big letters on the lid.
But, if so, please put the warning in big letters on the lid.
I have 700 warnings in big letters here!
The EA movement is rather like a church. (I have in mind the Catholic and Orthodox churches, not the new-fangled outfits that developed after the Reformation.) The prominent philosophers, like Peter Singer, are the prophets. The people who found and run organisations like GiveWell are the clergy. There are the rank and file members, who are the monks toiling on the work of the church. There are lay preachers, such as Scott Alexander. There are ordinary folk who do no more than tithe to the approved charities or turn vegetarian. And caught up in all that there are a few who have argued themselves into an unsustainable religious mania.
I think I agree with a lot of the object level content of the post (and I once fell into a years-long depression due to my own inability to live what I wanted my values to be), but I also would add that there needs to be a lot of initial and ongoing work done, by a lot of people who don't need or necessarily want to be doing it, in order for us to exist in a world where we (and most people) can have the kind of freedom you're talking about, in a meaningful way. You don't have to be the one doing it, but someone does, and we'd better hope they have the kind of ideals which involve valuing other peoples' well-being. Which, of course, we get mainly by encouraging the best, brightest, and most-likely-to-be-influential to have those kinds of values.
See John Adams:
I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and porcelain.
And then what happens in the 4th generation? Someone in each generation needs to be at the John Adams equivalent object level, and manage to become powerful enough to keep the virtuous cycle going.
2I will say I tried extremely hard to be a good EA. It basically drove me insane. Community is extremely unsupportive. I basically decided Im retired. I put in enough years in the misery mines and donated more than enough money (most of what I did ended in failure but I cashed out 7 figs of crypto and donated a lot). I will be a nice friendly generous person by the extremely low standards of my actually existing society. But otherwise I will just do what I want. I cried enough tears and sacrificed enough of the windfalls I was legally entitled to keep.
4For what it's worth, your "small and vulnerable" post is what convinced me that people can really have an unbelievable amount of kindness and compassion in them, a belief that made me much more receptive to EA. Stay out of the misery mines pls!
In Defense of Values
I don't mean to be harsh, but if everyone in this community followed your advice, then the world would likely end. And you can call that the rational outcome if you want, but if that's the outcome, what value is rationality?
I don't like pressuring people, so in my AI Safety Movement Building, I try to only encourage people to do things if it's in line with their values and, while there is some advice I can offer here, I mostly just leave it to people to figure out their own values.
But we need people to choose to be prosocial and for that to happen, a community needs a norm that prosociality is considered better than anti-sociality and pro-social people should be esteemed more than those who are anti-social. A community that loses sight of this is bound to fail. I humbly submit that we should decide NOT to destroy our community.
I can see why you might find it frustrating, and why you might want your choices to be considered the equal of those who are making significant sacrifices to save the world, but unfortunately, according to my values (and I suspect the values of most people here) it's absolutely vital that we do not do that.
For this reason, I down-voted this post. I expect this will be controversial, so I'll explain my reasoning:
While I'm generally against downvoting a post based on values, I feel it's valid in these circumstances. If this post was a piece of analytical philosophy making an argument that given certain assumptions we can conclude a particular thing about morality, then I think the post should be evaluated by how well the conclusions follow from the premises and how plausible the premises are, even if it were arguing for something that I hate.
Whilst this post does gesture at some of these arguments, it is best characterized as more an exhortation than a philosophical argument. It's not trying to make detailed and rigorous arguments, so I can't judge the post on this basis and indeed imposing this standard would be unfair to the author.
This forces me to find another basis on which to judge the post. Given this, I think it's fine to just upvote or downvote based on whether I think we should adopt or oppose this exhortation[1] and I believe this exhortation is negative, so I'm downvoting it. Telling people that they probably really, truly want to be selfish is bad. I'm not saying that we should bully people into being pro-social, but the OP is encouraging people who would be pro-social by default not to be.
I'd encourage other people to consider the post on the same basis and upvote or downvote accordingly. I'll admit that I'm not neutral here: I'm worried that people might disagree with this post, but feel it'd be wrong for them to downvote it, and so not do so. I'm here to say that it's not wrong; but also, if you think that these are the values we should adopt, then you should feel free to upvote it instead.
One point I want to emphasize: you don't need to be able to identify a specific wrong argument in order to downvote a post. The problem with this principle is that then a post could be mostly immune to down-voting by vaguely gesturing to some dubious assumptions instead of explicitly stating them where they might be subject to criticism. And I claim that if you pay attention, you'll notice that something in the implicit framing of this post is off, even if you can't quite put it into words.
- ^
If this post was a comment with separate upvote/downvote and agree vote/disagree vote, then I'd probably just disagree vote instead. However, it isn't, so I've got to work with what I've got.
I interpret upvotes/downvote as
- Do I want other people to read this post
- Do I want to encourage the author and others to write more posts like this.
And I favour this post for both of those reasons.
I agree that this post doesn't make philosophical argument for it's position, but I don't require that for every post. I value it as an observation of how the EA movement has affected this particular person, and as criticism.
A couple of strongly Anti-EA friends of mine became so due to a similar moral burnout, so it's particularly apparent to me how little emphasis is put on mental health.
I agree that this post doesn't make philosophical argument for it's position, but I don't require that for every post. I value it as an observation of how the EA movement has affected this particular person, and as criticism.
Just to make my position really clear: I never said this post needed to make a philosophical argument for its position, rather that if a post wasn't a philosophical argument we shouldn't judge it by the standards we apply to a philosophical argument.
Then I tried to figure out an alternative standard by which to judge this post.
I don’t want to tile the universe with hedonium.
Good! I don't want to be replaced with tiled hedonium either.
Perhaps some of the issue might be with perceptions of what is supposed to be "good" not matching your own values.
It is always painful to see burnt out people who didn't have the wisdom to attempt to save the world with less effort and then overcorrect in the opposite direction. I see it as a failure of applying consequentialism to the naive utilitarian reasoning. Yes, if you just dedicate 100% of your efforts towards saving the world it would be better than dedicating only 50% of your efforts, all things being equal. But things are unlikely to be equal. It's better to leave yourself some slack so that you could keep saving the world after 10 years, instead of deciding that it's all was a mistake and then agitating aginst saving the world, saying that it's only for some special people.
I think the correct balance isn't "Don't save the world, unless you want to do it for non-worldsaving reasons". I think it's more like "Save the world not more than 20% beyond what you are comfortable with. If you are in the situation where you feel constant guilt and stress because you are not yet a god - then you need to decrease the amount of effort you put towards attempts of world saving. If you have never even tried to save the world - then doing 10% EA pledge seems as a sound idea.
Exploring your limits is okay. Constantly pushing yourself beyond them, without any feeling of satisfaction is neither sustainable nor productive. Saving the world mustn't suck. Just keep it at "mildly inconvinient" level at worst.
Some relevant thoughts are in Nate's Replacing Guilt sequence.
I can understand the sentiments expressed here - particularly in terms of dealing with these things at the age of 12.
However, I'd draw a distinction between:
- Making the world better according to values that aren't your own, from a sense of obligation.
- Doing something to prevent the literal end of the world.
And I'd note that (2) does not rest on (1), on altruism (effective or otherwise), or on any particularly narrow moral view. Wanting some kind of non-paperclip-like world to continue existing isn't a niche value.
Nietzsche or Ayn Rand would be among the last people to be guilted into saving the world, but may well do it anyway. This is not because they cared deeply about shrimp! (not to my knowledge, that is)
But of course there's a lot to be said for understanding your values, and following a path you endorse on your own terms.
Nobody linked that, so I probably should: https://www.lesswrong.com/posts/cujpciCqNbawBihhQ/self-integrity-and-the-drowning-child
I also have very little intrinsic desire to maximize aggregate utility, but I do seem to have an intrinsic desire to fight for the community that does. It's tricky, but this is how it always has been everywhere. You have to organize, but the organization and the individual will want different things, because the organization is a superagent trying to pursue a coherent agenda despite being composed of multiple individuals, and then some people pretend to themselves have the aggregate utility function, and they get promoted because they can be trusted to directly pursue the interests of the organization, but most of them are lying about their utility function in order to pursue promotion, and some of them don't even consciously recognize that about themselves, such as your former self, who no one could have helped.
the best you can do is what you think is personally good
As far as you're an ideal optimizing agent with consistent values and full knowledge, otherwise actions based on your thoughts may end up worse than using social heuristics.
That there are no bugs when it comes to values. That you should care about exactly what you want to care about. That if you want to team up and save the world from AI or poverty or mortality, you can, but you don’t have to.
Locally invalid. Values can be terminal (what you care about) and instrumental, and saving the world for most people is actually instrumental.
There’s value in giving explicit permission to confused newcomers to not get trapped in moral chains, because it’s really easy to hurt yourself doing that.
I think that's true since memes can be harmful, but there is also value in reminding that if more people worked to save improve the world on average, it would be better, and often a simpler way is to do that yourself instead of pushing that meme+responsibility out.
Save the world if you want to, but please don’t if you don’t want to.
I'd continue that with "but please don't destroy the world whichever option you choose, since that will interfere with my terminal goals and I'll care about your non-existence".
I've seen a lot of EAs who are earnest. I think they are in for hurt down the line. I am not earnest in that way. I am not committed to tight philosophical justifications of my actions or values. I dont follow arguments to the end of the line. But one day I heard will macaskill describe the drowning child thought experiment, thought "yeah that makes sense to me", and added that to my list of thoughts. When I realized I was on the path to an economics PhD (for my own passions), I figured it was worth looking up this EA stuff and seeing what it had to say. I figured there would be lots of useful things I could do. I think that was the right intuition. I have found myself in a good position where I only need to make minor changes to my path to increase my impact dramatically.
Saving the world only sucks when you sacrifice everything else for it. Saving the world in your free time is great fun.
Probably a good time to leave this here: https://www.clearerthinking.org/tools/the-intrinsic-values-test
Thank you for this post. It says to me that community builders need to be very careful in how they treat young, people. For better and worse, they are more impressionable.
I can see where you're coming from with the dislike of social pressure. Life isn't life without agency (stated as opinion).
I want to "save the world" to the extent that I can transform it into something that I like more than what currently exists. It's like cleaning my house, I merely dislike the look of garbage.
My egoistic desires just happen to align somewhat with that of other people, since I'm a human as well. (Only somewhat. I'm not so naive that I think suffering is inherently bad)
There's two things that I feel are imporant here, though:
A: Other people do what they think is right, but 99% of people are idiots, so they are actually making things worse.
B: If fighting against the natural flow of things is a waste of time like the daoist seem to say, then so be it. But, I'm starting to see some bad disasters on the horizon, and I'm not even talking about AI risks or environmental change. I'm not sure if humanity deserves to survive, but I think it would be a shame to end things so early.
I want to “save the world” to the extent that I can transform it into something that I like more than what currently exists.
The context seems to be saving the world from runaway AI, which can't be nontrivially described that way.
Correct, I should have had that in mind when I wrote my comment.
The alternative to runaway AI is an AI whose values are acceptable to us. I belive that this is a design problem. The perfect system should be something like what's described in The Fun Theory Sequence and posts like When "yang" goes wrong describes two extremes as anarchy and tyranny, and I think it's fair to assume that the perfect world must be a balance between these two extremes.
But this topic is not trivial at all, I will give you that. I also don't think we agree very much about what the ideal future or AI looks like. I think some naive "Minimize suffering" optimization target is the most popular, but if you ask me that's merely due to an old misunderstanding of Buddhism.
You could also say that any AI will become "Runaway", and that we merely get to influence the direction. I'm pushing in my direction of choice, even if the force is tiny.
But I will stop here, as I have a lot of opinions on the subject, which probably don't fit the consensus very well.
Real people will actually die. One can wash one's hands of this and there's nothing I can do to stop this, but real people will actually die if we don't try to help others. It's not a game.
Curated and popular this week