Either scenario clearly implies that these estimates are severely distorted and have to be interpreted as marketing copy designed to control your behavior, not unbiased estimates designed to improve the quality of your decisionmaking process.
While most things have at least some motives to control your behavior, I do think GiveWell outlines a pretty reasonable motivation here that they explained in detail in the exact blogpost that you linked (and I know that you critiqued that reasoning on your blog, though I haven't found the arguments there particularly compelling). Even if their reasoning is wrong, they might still genuinely believe that their reasoning is right, which I do think is very important to distinguish from "marketing copy designed to control your behavior".
I am often wrong and still try to explain to others why I am right. Sometimes this is the cause of bad external incentives, but sometimes it's also just a genuine mistake. Humans are not perfect reasoners and they make mistakes for reasons other than to take advantage of other people (sometimes they are tired, or sometimes they haven't invented physics yet and try to build planes anyway, or sometimes they haven't figured out what good game theory actually looks like and try their best anyways).
For clarity, the claim Givewell-at-the-time made was:
For giving opportunities that are above the benchmark we’ve set but not “must-fund” opportunities, we want to recommend that Good Ventures funds 50%. It’s hard to say what the right long-term split should be between Good Ventures (a major foundation) and a large number of individual donors, and we’ve chosen 50% largely because we don’t want to engineer – or appear to be engineering – the figure around how much we project that individuals will give this year (which would create the problematic incentives associated with “funging” approaches). A figure of 50% seems reasonable for the split between (a) one major, “anchor” donor who has substantial resources and great conviction in a giving opportunity; (b) all other donors combined.
With the two claims I've heard about why the 50% split being:
1. There's still more than $8 billion dollars worth of good to do, and they expect their last dollar to be worth more than current dollars.
(I agree that this is at least somewhat sketchy, esp. when you think about Gates Foundation and others, although I think the case is less strong than Benquo is presenting here)
2. Having a charity ha...
Here's the part of the old series that dealt with this consideration: http://benjaminrosshoffman.com/givewell-case-study-effective-altruism-4/
The problem already exists on multiple levels, and the decision GiveWell made doesn't really alleviate it much. We should expect that GiveWell / Open Philanthropy Project is already distorting its judgment to match its idea of what Good Ventures wants, and the programs it's funding are already distorting their behavior to match their idea of what GiveWell / Open Philanthropy Project wants (since many of the "other" donors aren't actually uncorrelated with GiveWell's recommendations either!).
This line of thinking also seems like pretty much the opposite of the one that suggests that making a large grant to OpenAI in order to influence it would be a good idea, as I pointed out here. The whole arrangement is very much not what someone who was trying to avoid this kind of problem would build, so I don't buy it as an ad hoc justification for this particular decision.
I find this general pattern (providing reasons for things, that if taken seriously would actually recommend a quite different course of action t...
Some further thoughts on that: I agree social-reality-distortions are a big problem, although I don't think the werewolf/villager-distinction is the best frame. (In answer to Wei_Dai's comment elsethread, "am I a werewolf" isn't a very useful question. You almost certainly are at least slightly cognitively-distorted due to social reality, at least some of the time. You almost certainly sometimes employ obfuscatory techniques in order to give yourself room to maneuver, at least sort of, at least some times.)
But I think thinking in terms of villagers and werewolves leads you to ask the question 'who is a werewolf' moreso than 'how do we systematically disincentivize obfuscatory or manipulative behavior', which seems a more useful question.
I bring this all up in this particular subthread because I think it's important that one thing that incentivizes obfuscatory behavior is giving away billions of dollars.
My sense (not backed up by much legible argument) is that a major source of inefficiencies of the Gates Foundation (and OpenPhil to a lesser degree) is that they've created an entire ecosystem, which both attracts people motivate...
It seems like there's this general pattern, that occurs over and over, where people follow a path going:
1. Woah. Drowning child argument!
2. Woah. Lives are cheap!
3. Woah, obviously this is important to take action on and scale up now. Mass media! Get the message out!
4. Oh. This is more complicated.
5. Oh, I see, it's even more complicated. (where complication can include moving from global poverty to x-risk as a major focus, as well as realizing that global poverty isn't as simple to solve)
6. Person has transitioned into a more nuanced and careful thinker, and now is one of the people in charge of some kind of org or at least a local community somewhere. (for one example, see CEA's article on shifting from mass media to higher fidelity methods of transition)
But, the mass media (and generally simplified types of thinking independent of strategy) are more memetically virulent than the more careful thinking, and new people keep getting excited about them in waves that are self-sustaining and hard to clarify (esp. since the original EA infrastructure was created by people at the earlier stages of thinking). So it keeps on being something that a newcomer will bump into most often in EA spaces.
CEA continues to actively make the kinds of claims implied by taking GiveWell's cost per life saved numbers literally, as I pointed out in the post. Exact quote from the page I linked:
If you earn the typical income in the US, and donate 10% of your earnings each year to the Against Malaria Foundation, you will probably save dozens of lives over your lifetime.
Either CEA isn't run by people in stage 6, or ... it is, but keeps making claims like this anyway.
Sure, I just think the right frame here is "detect and counteract obfuscatory behavior" rather than "detect werewolves." I think the "detect werewolves", or even "detect werewolf behavior" frame is more likely to collapse into tribal and unhelpful behavior at scale [edit: and possibly before then]
(This is for very similar reasons to why EA arguments often collapse into "donate all your money to help people". It's not that the nuances position isn't there, it just gets outcompeted by simpler versions of itself)
I'm worried here about the general pattern of people optimizing discourse for "the public"
I do agree that this is a pattern to watch out for. I don't think it applies here, but could be wrong. I think it's very important that people be able to hold themselves to higher standards than what they can easily explain to the public, and it seems like a good reflex to notice when people might be trying to do that and point it out.
But I'm worried here about well-informed people caching ideas wrongly, not about the general public. More to say about this, but first want to note:
also note that the frame you objected to (the villagers vs werewolf frame) contains important epistemic content that the "let's incentivize non-obfuscatory behavior" frame doesn't, as you agreed in your subsequent comment after I pointed it out.
Huh - this just feels like a misinterpretation or reading odd things into what I said.
It had seemed obvious to me that to disincentivize obfuscatory behavior, you need people to be aware of what obfuscatory behavior looks like and what to do about it, and it felt weird that you saw that as something different.
It is fair that...
I'm worried about this, concretely, because after reading Effective Altruism is Self Recommending a while, despite the fact that I thought lots about it, and wrote up detailed responses to it (some of which I posted and some of which I just thought about privately), and I ran a meetup somewhat inspired by taking it seriously...
...despite all that, a year ago when I tried to remember what it was about, all I could remember was "givewell == ponzi scheme == bad", without any context of why the ponzi scheme metaphor mattered or how the principle was supposed to generalize. I'm similarly worried that a year from now, "werewolves == bad, hunt werewolves", is going to be the thing I remember about this.
The five-word-limit isn't just for the uninformed public, it's for serious people trying to coordinate. The public can only coordinate around 5-word things. Serious people trying to be informed still have to ingest lots of information and form detailed models but those models are still going to have major bits that are compressed, out of pieces that end up being about five words. And this is a major part of whymany people are confused about Effective Altruism and how to do it right in the first place.
If that's your outlook, it seems pointless to write anything longer than five words on any topic other than how to fix this problem.
I agree with the general urgency of the problem, although I think the frame of your comment is somewhat off. This problem seems... very information-theoretically-entrenched. I have some sense that you think of it as solvable in a way that it's fundamentally not actually solvable, just improvable, like you're trying to build a perpetual motion machine instead of a more efficient engine. There is only so much information people can process.
(This is based entirely off of reading between the lines of comments you've made, and I'm not confident what your outlook actually is here, and apologies for the armchair psychologizing).
I think you can make progress on it, which would look something like:
0) make sure people are aware of the problem
1) building better infrastructure (social or technological), probably could be grouped into a few goals:
2) meanwhile, as a writer, make sure that the concepts you create for the public discourse are optimized for the right kind of compression. Some ideas com...
Trying to nudge others seems like an attempt to route around the problem rather than solve it. It seems like you tried pretty hard to integrate the substantive points in my "Effective Altruism is self-recommending" post, and even with pretty extensive active engagement, your estimate is that you only retained a very superficial summary. I don't see how any compression tech for communication at scale can compete with what an engaged reader like you should be able to do for themselves while taking that kind of initiative.
We know this problem has been solved in the past in some domains - you can't do a thing like the Apollo project or build working hospitals where cardiovascular surgery is regularly successful based on a series of atomic five-word commands; some sort of recursive general grammar is required, and at least some of the participants need to share detailed models.
One way this could be compatible with your observation is that people have somewhat recently gotten worse at this sort of skill; another is that credit-assignment is an unusually difficult domain to do this in. My recent blog posts have argued that at least the latter is true.
In the former case...
when I tried to remember what it was about, all I could remember [...] I'm similarly worried that a year from now
Make spaced repetition cards?
Holden Karnofsky explicitly disclaimed the "independence via multiple funders" consideration as not one that motivated the partial funding recommendation.
If you give based on mass-marketed high-cost-effectiveness representations, you're buying mass-marketed high-cost-effectiveness representations, not lives saved. Doing a little good is better than buying a symbolic representation of a large amount of good. There's no substitute for developing and acting on your own models of the world.
As far as I can see, this pretty much destroys the generic utilitarian imperative to live like a monk and give all your excess money to the global poor or something even more urgent. Insofar as there's a way to fix these problems as a low-info donor, there's already enough money. Claims to the contrary are either obvious nonsense, or marketing copy by the same people who brought you the obvious nonsense. Spend money on taking care of yourself and your friends and the people around you and your community and trying specific concrete things that might have specific concrete benefits. And try to fix the underlying systems problems that got you so confused in the first place.
Some people on your blog have noted that this doesn't seem true, at least, because GiveDirectly still exists (both literally and as a sort of metaphorical pri...
It seems to me that if attending to the ordinary business of your life, including career and hobbies, amounts to doing nothing, there's something very deeply wrong happening, and people would do well to attend to that problem first. On the other hand, doing nothing is preferable to doing harm, and it's entirely possible that many people are actually causing harm, e.g. by generating misinformation, and it would be better if they just stopped, even if they can't figure out how to do whatever they were pretending to do.
I certainly don't think that someone donating their surplus to GiveDirectly, or living more modestly in order to share more with others, is doing a wrong thing. It's admirable to want to share one's wealth with those who have less.
It seems to me that if attending to the ordinary business of your life, including career and hobbies, amounts to doing nothing, there’s something very deeply wrong happening, and people would do well to attend to that problem first.
I'm tempted to answer this statement by saying that something very deeply wrong is clearly happening, e.g., there's not nearly enough effort in the world to prevent coordination failures that could destroy most of the potential value of the universe, and attending to that problem would involve doing something besides or in addition to attending to the ordinary business of life. I feel like this is probably missing your point though. Do you want to spell out what you mean more, e.g., is there some other "something very deeply wrong happening" you have in mind, and if so what do you think people should do about it?
If people who can pay their own rent are actually doing nothing by default, that implies that our society's credit-allocation system is deeply broken. If so, then we can't reasonably hope to get right answers by applying simplified economic models that assume credit-allocation is approximately right, the way I see EAs doing, until we have a solid theoretical understanding of what kind of world we actually live in.
Here's a simple example: Robin Hanson's written a lot about how it's not clear that health care is beneficial on the margin. This is basically unsurprising if you think there are a lot of bullshit jobs. But 80,000 Hours's medical career advice assumes that the system basically knows what it's doing and that health care delivers health on the margin - the only question is how much.
It seems to me that if an intellectual community isn't resolving these kind of fundamental confusions (and at least one side has to be deeply confused here, or at least badly misinformed), then it should expect to be very deeply confused about philanthropy. Not just in the sense of "what is the optimal strategy," but in the sense of "what does giving away money even do."
[I wrote the 80k medical careers page]
I don't see there as being a 'fundamental confusion' here, and not even that much of a fundamental disagreement.
When I crunched the numbers on 'how much good do doctors do' it was meant to provide a rough handle on a plausible upper bound: even if we beg the question against critics of medicine (of which there are many), and even if we presume any observational marginal response is purely causal (and purely mediated by doctors), the numbers aren't (in EA terms) that exciting in terms of direct impact.
In talks, I generally use the upper 95% confidence bound or central estimate of the doctor coefficient as a rough steer (it isn't a significant predictor, and there's reasonable probability mass on the impact being negative): although I suspect there will be generally unaccounted confounders attenuating 'true' effect rather than colliders masking it, these sort of ecological studies are sufficiently insensitive to either to be no more than indications - alongside the qualitative factors - that the 'best (naive) case' for direct impact as a doctor isn't promising.
There's little ...
Something that nets out to a small or no effect because large benefits and harms cancel out is very different (with different potential for impact) than something like, say, faith healing, where you can’t outperform just by killing fewer patients. A marginalist analysis that assumes that the person making the decision doesn’t know their own intentions & is just another random draw of a ball from an urn totally misses this factor.
helping by default (they pay their landlord, who buys goods, which pays manufacturers, etc.)
The exact opposite - getting paid should imply something. The naive Econ 101 view is that it implies producing something of value. "Production" is generally measured in terms of what people are willing to pay for.
If getting paid has little to do with helping others on net , then our society’s official unit of account isn’t tracking production (Talents), GDP is a measurement of the level of coercion in a society (There Is a War), the bullshit jobs hypothesis is true, we can’t take job descriptions at face value, and CEA’s advice to build career capital just means join a powerful gang.
This undermines enough of the core operating assumptions EAs seem to be using that the right thing to do in that case is try to build better models of what's going on, not act based on what your own models imply is disinformation.
Ok. This makes sense to me. GDP measures a mix of trades that occur due to simple mutual benefit and "trades" that occur because of extortion or manipulation.
If you look at the combined metric, and interpret it to be a measure of only the first kind of trade, you're likely overstating how much value is being created, perhaps by a huge margin, depending on what percentage of trades are based on violence.
But I'm not really clear on why you're talking about GDP at all. It seems like you're taking the claim that "GDP is a bad metric for value creation", and concluding that "interventions like give directly are a misguided."
Rereading this thread, I come to
If people who can pay their own rent are actually doing nothing by default, that implies that our society's credit-allocation system is deeply broken. If so, then we can't reasonably hope to get right answers by applying simplified economic models that assume credit-allocation is approximately right, the way I see EAs doing, until we have a solid theoretical understanding of what kind of world we actually live in.
Is the argument something like...
Ok. Great.
credit-allocation methods have to be presumed broken until established otherwise, and no adequate audit has entered common knowledge.
That does not seem obvious to me. It certainly does not seem to follow from merely the fact the GDP is not a good measure of national welfare. (In large part, because my impression is that economists say all the time that GDP is not a good measure of national welfare.)
Presumably you believe that point 2 holds, not just because of the GDP example, but because you've seen many, many examples (like health care, which you mention above). Or maybe because you have an analytical argument that the sort of thing that happens with GDP has to generalize to other credit allocation systems?
Is that right? Can you say more about why you expect this to be a general problem?
. . .
I have a much higher credence that give Directly Exists and is doing basically what it says it is doing than you do.
If I do a stack trace on why I think that...
Somewhat confused by the coca-cola example. I don't buy coke very often, but it seems usually worth it to me when I do buy it (in small amounts, since I do think it tastes pretty good). Is the claim that they are not providing any value some kind of assumption about my coherent extrapolated volition?
I mean, I agree that Coca Cola engages in marketing practices that try to fabricate associations that are not particularly truth-oriented, but that's very different from the thing with Theranos.
I model Coca Cola mostly as damaging for my health, and model its short-term positive performance effects to be basically fully mediated via caffeine, but I still think it's providing me value above and beyond those those benefits, and outweighing the costs in certain situations.
Theranos seems highly disanalogous, since I think almost no one who knew the actual extend of Theranos' capabilities, and had accurate beliefs about its technologies, would give money to them. I have pretty confident bounds on the effects of coca-cola, and still decide to sometimes give them my money, and would be really highly surprised if there turns out to be a fact about coke that its internal executives are aware of (even subconsciously) that would drastically change that assessment for me, and it doesn't seem like that's what you are arguing for.
Maybe more specifically an ingroup that takes over a potentially real, profitable social niche, squeezes out everyone else, and uses the niche’s leverage to maximize rent extraction, is a gang.
Many people find that thinking about effectiveness rapidly makes local giving seem a less attractive option.
The thought processes I can see that might lead someone to give locally in pursuit of effectiveness are quite complex ones:
Those are both certainly possible but I think they take more than a "small amount of thinking". Of course there are other ways to end up prioritizing local causes, but I think those go in the "without reflecting much" category. It seems to me that a modest amount of (serious) thinking about effectiveness makes local giving very hard to justify for its effectiveness, unless you happen to have a really exceptional local cause on your doorstep.
I haven't read your entire series of posts on Givewell and effective altruism. So I'm basing this comment mostly off of just this post. It seems like it is jumping all over the place.
You say:
Either charities like the Gates Foundation and Good Ventures are hoarding money at the price of millions of preventable deaths, or the low cost-per-life-saved numbers are wildly exaggerated. My former employer GiveWell in particular stands out as a problem here, since it publishes such cost-per-life-saved numbers, and yet recommended to Good Ventures that it not fully fund GiveWell's top charities; they were worried that this would be an unfair way to save lives.
This sets up a false dichotomy. Both the Gates Foundation and Good Ventures are focused on areas in addition to funding interventions in the developing world. Obviously, they both believe those other areas, e.g., in Good Ventures' case, existential risk reduction, present them with the opportunity to prevent just as many, if not more, deaths than interventions in the developing world. Of course, a lot of people disagree with the idea something like AI alignment, which Good Ventures funds, is in any way comparable ...
So, EA largely isn't about actually doing altruism effectively (which requires having correct information about what things actually work, e.g. estimates of cost per life saved, and not adding noise to conversations about these), it's an aesthetic identity movement around GiveWell as a central node, similar to e.g. most popular environmentalism (which, for example, opposes nuclear power despite it being good for the environment, because nuclear power is discordant with the environmentalism identity/aesthetics, and Greenpeace is against it), which is also claiming credit for, literally, evaluating and acting towards the moral good (as environmentalism claims credit for evaluating and acting towards the health of the planet). This makes sense as an explanation of the sociological phenomenon, and also implies that, according to the stated values of EA, EA-as-it-is ought to be replaced with something very, very different.
[EDIT: noting that what you said in another comment also agrees with the aesthetic identity movement view: "I imagine the CEA does it through some combo of revering Singer, thinking it’s good for optics, and not thinking the level of precision at which there error is taking place is so grievous as to be objectionable in the context they’re presented in."]
I agree with your analysis of the situation, but I wonder whether it’s possible to replace EA with anything that won’t turn into exactly the same thing. After all, the EA movement is the result of some people noticing that much of existing charity is like this, and saying “we should replace that with something very, very different”…
And EA did better than the previous things, along some important dimensions! And people attempting to do the next thing will have EA as an example to learn from, which will (hopefully) prompt them to read and understand sociology, game theory, etc. The question of "why do so many things turn into aesthetic identity movements" is an interesting and important one, and, through study of this (and related) questions, it seems quite tractable to have a much better shot at creating something that produces long-term value, than by not studying those questions.
Success is nowhere near guaranteed, and total success is quite unlikely, but, trying again (after a lot of study and reflection) seems like a better plan than just continuing to keep the current thing running.
The question of “why do so many things turn into aesthetic identity movements” is an interesting and important one, and, through study of this (and related) questions, it seems quite tractable to have a much better shot at creating something that produces long-term value, than by not studying those questions.
I agree that studying this is quite important. (If, of course, such an endeavor is entered into with the understanding that everyone around the investigators, and indeed the investigators themselves, have an interest in subverting the investigation. The level of epistemic vigilance required for the task is very unusually high.)
It is not obvious to me that further attempts at successfully building the object-level structure (or even defining the object-level structure) are warranted, prior to having substantially advanced our knowledge on the topic of the above question. (It seems like you may already agree with me, on this; I am not sure if I’m interpreting your comment correctly.)
So, rationality largely isn't actually about doing thinking clearly [...] it's an aesthetic identity movement around HPMoR as a central node [...] This makes sense as an explanation of the sociological phenomenon, and also implies that, according to the stated value of rationality, rationality-as-it-is ought to be replaced with something very, very different.
This just seems obviously correct to me, and I think my failure to properly integrate this perspective until very recently has been extremely bad for my sanity and emotional well-being.
Specifically: if you fail to make a hard mental disinction between "rationality"-the-æsthetic-identity-movement and rationality-the-true-art-of-systematically-correct-reasoning, then finding yourself in a persistent disagreement with so-called "rationalists" about something sufficiently basic-seeming creates an enormous amount of cognitive dissonance ("Am I crazy? Are they crazy? What's going on?? Auuuuuugh") in a way that disagreeing with, say, secular humanists or arbitrary University of Chicago graduates, doesn't.
But ... it shouldn't. Sure, self-identification with the "rationalist" brand name is a signal that someone knows some things abou
I haven't been following as closely how Ben construes 'bad faith', and I haven't taken the opportunity to discover, if he were willing to relay it what his model of bad faith is.
I think the most relevant post by Ben here is "Bad Intent Is a Disposition, Not a Feeling". (Highly recommended!)
Recently I've often found myself wishing for better (widely-understood) terminology for phenomena that it's otherwise tempting to call "bad faith", "intellectual dishonesty", &c. I think it's pretty rare for people to be consciously, deliberately lying, but motivated bad reasoning is horrifyingly ubiquitous and exhibits a lot of the same structural problems as deliberate dishonesty, in a way that's worth distinguishing from "innocent" mistakes because of the way it responds to incentives. (As Upton Sinclair wrote, "It is difficult to get a man to understand something when his salary depends upon his not understanding it.")
If our discourse norms require us to "assume good faith", but there's an important sense in which that assumption isn't true (because motivated misunderstandings resist correction in a way that simple mistakes don't), but we can't talk about the ways it isn't true without
The point I was trying to make is that much of the rationality community has nothing to do with the community’s stated values.
Yes, this is true, and also implies that the rationality community should be replaced with something very different, according to its stated goals. (Did you think I didn't think that?)
Geeks, Mops, Sociopaths happened to the rationality community, not just EA.
So, in stating as though a fact about EA your personal impression of it based on Sarah’s blog post as if that means something unique about EA that isn’t true about other human communities, you’ve argued for too much.
I don't think it's unique! I think it's extremely, extremely common for things to become aesthetic identity movements! This makes the phenomenon matter more, not less!
I have about as many beefs with the rationality movement as I do with the EA movement. I am commenting on this post because Ben already wrote it and I had things to add.
It's possible that I should feel more moral pressure than I currently do to actively (not just, as a comment on other people's posts) say what's wrong about the current state of the rationality community publicly. I've already been saying things privately. (This is an invitation to try morally pressuring me, using arguments, if you think it would actually be good for me to do this)
a) GiveWell does publish cost-effectiveness estimates. I found them in a few clicks. So Ben's claim is neither dishonest nor false.
While I agree that this is a sufficient rebuttal of Ray's "dishonest and/or false" charge (Ben said that GiveWell publishes such numbers, and GiveWell does, in fact, publish such numbers), it seems worth acknowleding Ray's point about context and reduced visibility: it's not misleading to publish potentially-untrustworthy (but arguably better than nothing) numbers surrounded by appropriate caveats and qualifiers, even when it would be misleading to loudly trumpet the numbers as if they were fully trustworthy.
That said, however, Ray's "GiveWell goes to great lengths to hide those numbers" claim seems false to me in light of an email I received from GiveWell today (the occasion of my posting this belated comment), which reads, in part:
GiveWell has made a lot of progress since your last recorded gift in 2015. Our current top charities continue to avert deaths and improve lives each day, and are the best giving opportunities we're aware of today. To illustrate, right now we estimate that for every $2,400 donated to Malaria Consortium for its seasonal malaria chemoprevention program, the death of a child will be averted.
(Bolding mine.)
Further update on this. Givewell has since posted this blogpost. I haven't yet reviewed this enough have a strong opinion on it, but I think it at least explains some of the difference in epistemic state I had at the time of this discussion.
Relevant bit:
Although we don’t advise taking our cost-effectiveness estimates literally, we do think they are one of the best ways we can communicate about the rough magnitude of expected impact of donations to our recommended charities.
A few years ago, we decided not to feature our cost-effectiveness estimates prominently on our website. We had seen people using our estimates to make claims about the precise cost to save a life that lost the nuances of our analysis; it seemed they were understandably misinterpreting concrete numbers as conveying more certainty than we have. After seeing this happen repeatedly, we chose to deemphasize these figures. We continued to publish them but did not feature them prominently.
Over the past few years, we have incorporated more factors into our cost-effectiveness model and increased the amount of weight we place on its outputs in our reviews (see the contrast between our 2014 cost-effectiveness model ve...
I agree with a lot of this (maybe not to the same degree) but I'm not sure what's causing it. You link to something about villagers vs werewolves. Does that mean you think GiveWell has been effectively taken over by werewolves or was run by werewolves from the beginning?
Assuming some version of "yes", I'm pretty sure the people running GiveWell do not think of themselves as werewolves. How can I rule out the possibility that I myself am a werewolf and not aware of it?
ETA: How much would it help to actually play the game? I got a copy of The Resistance a few days ago which is supposed to be the same kind of game as Werewolf, but I don't have a ready group of people to play with. Would it be worth the extra effort to find/make a group just to get this experience?
I found that it made a big subjective difference in my threat assessment for this kind of thing, when I'd had the subjective experience of figuring out how to play successfully as a werewolf. YMMV.
I don't think many people have a self-image as a "werewolf" trying to sabotage building of shared maps. I don't think anyone in GiveWell sees themselves that way.
I do think that many people are much more motivated to avoiding being blamed for things, than to create clarity about credit-assignment, and that this is sufficient to produce the "werewolf" pattern. If I ask people what their actual anticipations are about how they and others are likely to behave, in a way that is grounded and concrete, it usually seems like they agree with this assessment. I've had several conversations in which one person has said both of the following:
As far as I can tell, the "werewolf" thing is how large parts of normal, polite society work by default, and most people trying to do something that requires accurate collective credit-assignment in high stakes situations just haven't reflected on how far a departure that would require from normal behavior.
As far as I can tell, the "werewolf" thing is how large parts of normal, polite society work by default
This is true, and important. Except "werewolf" is a misleading analogy for it - they're not intentionally colluding with other secret werewolves, and it's not a permanent attribute of the participants. It's more that misdirection and obfuscation are key strategies for some social-competitive games, and these games are part of almost all humans motivation sets, both explicitly (wanting to have a good job, be liked, etc.) and implicitly (trying to win every status game, whether it has any impact on their life or not).
The ones who are best at it (most visibly successful) firmly believe that the truth is aligned with their winning the games. They're werewolf-ing for the greater good, because they happen to be convincing the villagers to do the right things, not because they're eating villagers. And as such, calling it "werewolf behavior" is rejected.
I do think that many people are much more motivated to avoiding being blamed for things, than to create clarity about credit-assignment, and that this is sufficient to produce the “werewolf” pattern.
I think I myself am often more motivated to avoiding being blamed for things, than to create clarity about credit-assignment. I feel like overall I'm still doing more good than harm (and therefore "it would be better if they just stopped", as you put it in another comment, doesn't apply to me). How can I tell if I'm wrong about this?
As far as I can tell, the “werewolf” thing is how large parts of normal, polite society work by default, and most people trying to do something that requires accurate collective credit-assignment in high stakes situations just haven’t reflected on how far a departure that would require from normal behavior.
How optimistic are you that "accurate collective credit-assignment in high stakes situations" can be greatly improved from what people are currently doing? If you're optimistic, can you give some evidence or arguments for this, aside from the fact that villagers in Werewolf can win if they know what to do?
I'm more confident that we can't solve AI alignment without fixing this, than I am that we can fix it.
Accounts of late-20th-Century business practices seem like they report much more Werewolfing than accounts of late-19th-Century business practices - advice on how to get ahead has changed a lot, as have incidental accounts of how things work. If something's changed recently, we should have at least some hope of changing it back, though obviously we need to understand the reasons. Taking a longer view, new civilizations have emerged from time to time, and it looks to me like often rising civilizations have superior incentive-alignment and information-processing than the ones they displace or conquer. This suggests that at worst people get lucky from time to time.
Pragmatically, a higher than historically usual freedom of speech and liberalism more generally seem like they ought to both make it easier to think collectively about political problems than it's been in the past, and make it more obviously appealing, since public reason seemed to do really well at improving a lot of people's lives pretty recently.
I’m more confident that we can’t solve AI alignment without fixing this, than I am that we can fix it.
Can you give some examples of technical AI alignment efforts going wrong as a result of bad credit assignment (assuming that's what you mean)? To me it seems that to the extent things in that field aren't headed in the right direction, it's more a result of people underestimating the philosophical difficulty, or being too certain about some philosophical assumptions, or being too optimistic in general, that kind of thing.
Accounts of late-20th-Century business practices seem like they report much more Werewolfing than accounts of late-19th-Century business practices—advice on how to get ahead has changed a lot, as have incidental accounts of how things work.
This seems easily explainable by the fact that businesses have gotten a lot bigger to take advantage of economies of scale offered by new technologies, so coordination / principal-agent problems have gotten a lot worse as a result.
Taking a longer view, new civilizations have emerged from time to time, and it looks to me like often rising civilizations have superior incentive-alignment and information-processing than the
If orgs like OpenAI and Open Philanthropy Project are sincerely trying to promote technical AI alignment efforts, then they’re obviously confused about the fundamental concept of differential intellectual progress.
I think I have more uncertainty than you do about whether OpenAI/OpenPhil is doing the right thing, but conditional on them not doing the right thing, and also not just being cynical, I don't think being confused about the fundamental concept of differential intellectual progress is the best explanation of why they're not doing the right thing. It seems more likely that they're wrong about how much of a broad base of ML expertise/capability is needed internally in an organization to make progress in AI safety, or about what is the best strategy to cause differential intellectual progress or bring about an aligned AGI or prevent AI risk.
If, on the other hand, we think they’re just being cynical and collecting social credit for labeling things AI safety rather than making a technical error, then the honest AI safety community seems to have failed to create clarity about this fact among their supporters.
I personally assign less than 20% probability that "they’re just
But in that case I probably have similar biases and I don't see a strong reason to think I'm less affected by them than OpenAI/OpenPhil, so it doesn't seem right to accuse them of that when I'm trying to argue for my own positions.
This seems backwards to me. Surely, if you're likely to make error X which you don't want to make, it would be helpful to build shared models of the incidence of error X and help establish a norm of pointing it out when it occurs in others, so that others will be willing and able to correct you in the analogous situation.
It doesn't make any sense to avoid trying to help someone by pointing out their mistake because you might need the same kind of help in the future, at least for nonrivalrous goods like criticism. If you don't think of correcting this kind of error as help, then you're actually just declaring intent to commit fraud. And if you'd find it helpful but expect others to think of it as unwanted interference, then we've found an asymmetric weapon that helps with honesty but not with dishonesty.
Accusing others of bias also seems less effective in terms of changing minds (i.e., it seems likely...
This seems backwards to me. Surely, if you’re likely to make error X which you don’t want to make, it would be helpful to build shared models of the incidence of error X and help establish a norm of pointing it out when it occurs in others, so that others will be willing and able to correct you in the analogous situation.
I think that would make sense if I had a clear sense of how exactly biases related to social credit is causing someone to make a technical error, but usually it's more like "someone disagrees with me on a technical issue and we can't resolve the disagreement, it seems pretty likely that one or both of us is affected by some sort of bias that's related to social credit and that's the root cause of the disagreement, but it could also be something else like being naturally optimistic vs pessimistic, or different past experiences/backgrounds". How am I supposed to "create clarity" in that case?
That’s a reason to be more clear, not less clear, about what’s going on—as long as people who understand what’s going on obscure the issue to be polite, this strategy will continue to work.
As I mentioned before, I don't entirely understand what is going on, in other words
In a competitive attention market without active policing of the behavior pattern I'm describing, it seems wrong to expect participants getting lots of favorable attention and resources to be honest, as that's not what's being selected for.
There's a weird thing going on when, if I try to discuss this, I either get replies like Raemon's claim elsewhere that the problem seems intractable at scale (and it seems like you're saying a similar thing at times), or replies to the effect that there are lots of other good reasons why people might be making mistakes, and it's likely to hurt people's feelings if we overtly assign substantial probability to dishonesty, which will make it harder to persuade them of the truth. The obvious thing that's missing is the intermediate stance of "this is probably a big pervasive problem, and we should try at all to fix it by the obvious means before giving up."
It doesn't seem very surprising to me that a serious problem has already been addressed to the extent that it's true that both 1) it's very hard to make any further progress on the problem and 2) the remaining cost from not fully solving the problem can be lived with.
The obvious thing that’s missing is the intermediate stance of “this is probably a big pervasive problem, and we should try at all to fix it by the obvious means before giving up.”
It seems to me that people like political scientists, business leaders, and economists have been attacking the problem for a while, so it doesn't seem that likely there's a lot of low hanging fruit to be found by "obvious means". I have some more hope that the situation with AI alignment is different enough from what people thought about in the past (e.g., a lot of people involved are at least partly motivated by altruism compared to the kinds of people described in Moral Mazes) that you can make progress on credit assigning as applied to AI alignment, but you still seem to be too optimistic.
But in that case I probably have similar biases and I don't see a strong reason to think I'm less affected by them than OpenAI/OpenPhil, so it doesn't seem right to accuse them of that when I'm trying to argue for my own positions.
You're not, as far as I know, promoting an AI safety org raising the kind of funds, or attracting the kind of attention, that OpenAI is. Likewise you're not claiming mainstream media attention or attracting a large donor base the way GiveWell / Open Philanthropy Project is. So there's a pretty strong reason to expect that you haven't been selected for cognitive distortions that make you better at those things anywhere near as strongly as people in those orgs have.
I'm confused about the argument you're trying to make here (I also disagree with some things, but I want to understand the post properly before engaging with that). The main claims seem to be
There are simply not enough excess deaths for these claims to be plausible.
and, after telling us how many preventable deaths there could be,
Either charities like the Gates Foundation and Good Ventures are hoarding money at the price of millions of preventable deaths, or the low cost-per-life-saved numbers are wildly exaggerated.
But I don't understand how these claims interconnect. If there were more people dying from preventable diseases, how would that dissolve the dilemma that the second claim poses?
Also, you say that $125 billion is well within the reach of the GF, but their website says that their present endowment is only $50.7 billion. Is this a mistake, or do you mean something else with "within reach"?
I still have no idea of how the total amount of dying people is relevant, but my best reading of your argument is:
According to this post, OpenPhil intends to spend rougly 10% of their money on "straightforward charity" (rather than their other cause areas). That would be about $1B (though I can't find the exact numbers right now), which is a lot, but hardly unlimited. Their worries about displacing other donors, coupled with the possibility of learning about better opportunities in the future, seems sufficient to justify partial funding to me.
That leaves the Gates Foundation (at least among the foundations that you mentioned, of course there's a lot more). I don't have a good model of when really big foundations does and doesn't grant money, but I think Carl Shulman makes some interesting points in this old thread.
In some sense, EA is arbitrage between the price of life in rich and poor countries, and such price will eventually become more equal.
Another point is that saving life locally sometimes is possible almost for free, if you happen to be in right place and thus have unique information. For example, calling 911 if you see a drawing child may be very effective and cost you almost nothing. There were several cases in my life when I had to put attention of a taxi driver to a pedestrian ahead - not sure if I actually saved life. But to save life locally one needs to pay attention on what is going around him and know how to react effectively.
Given the kinds of information gaps and political problems I describe here, it seems to me that while in expectation they’re a good thing to do with your surplus, the expected utility multiplier should be far less than the very large one implied by a straightforward “diminishing marginal utility of money” calculation.
If we assume that all of this is treatable at current cost per life saved numbers - the most generous possible assumption for the claim that there's a funding gap - then at $5,000 per life saved (substantially higher than GiveWell's current estimates), that would cost about $50 Billion to avert.
Of course, that’s an annual number, not a total number. But if we think that there is a present, rather than a future, funding gap of that size, that would have to mean that it’s within the power of the Gates Foundation alone to wipe out all fatal communic...
Hmm, what's your source for that? They have a total net-wealth of something like $160 billion, so it can't be more than a factor of 3. And it seems quite likely to me that both of them have at least some values that are not easily captured by "save as many lives as possible" such that I don't expect all of that $160 billion to go towards that goal (e.g. I expect a significant fraction of that money to go things like education, scientific achievement and other things that don't have the direct aim of saving lives but are pursuing other more nebulous goals).
I have one or more comments I'd like to make, but I'd like to know what sorts of comments you consider to be either 'annoying' or 'counterproductive' before I make them. I agree with some aspects of this article, but I disagree with others. I've checked, and I think my disagreements will be greater in both number and degree than other comments here. I wouldn't expect you to find critical engagement based on some strong disagreement to "be annoying or counterproductive", but I'd like to get a sense if y...
Did you know that if a comment gets deleted, the author of it is notified via PM and given a copy of the deleted comment? So if you don't mind the experience of having a comment deleted, you can just post your comment here, and repost it as an article later if it does get deleted.
I didn't know that, but neither do I mind the experience of having a comment deleted. I would mind:
Were...
This does seem like something we should telegraph better than we currently do, although I'm not sure how.
In general, I'd very much like a permanent neat-things-to-know-about-LW post or page, which receives edits when there's a significant update (do tell me if there's already something like this). For example, I remember trying to find information about the mapping between karma and voting power a few months ago, and it was very difficult. I think I eventually found an announcement post that had the answer, but I can't know for sure, since there might have been a change since that announcement was made. More recently, I saw that there were footnotes in the sequences, and failed to find any reference whatsoever on how to create footnotes. I didn't learn how to do this until a month or so later, when the footnotes came to the EA forum and aaron wrote a post about it.
Okay, thanks. Sorry for the paranoia. I just haven't commented on any LW posts with the 'reign of terror' commenting guidelines before, so I didn't know what to expect. That gives me enough context to feel confident my comment won't be like that one you deleted.
I picked Reign of Terror because I wasn’t sure I wanted to commit to the higher deletion thresholds (I think the comment I deleted technically doesn’t meet them), so I wanted to avoid making a false promise.
I do want to hold myself to the standard of welcoming criticism & only deleting stuff that seems like it’s destroying importance-weighted information on net. I don’t want to be held to the standard of having to pretend that everyone being conventionally polite and superficially relevant is really trying.
I have updated (partly in this thread, although it retroactively fits into past observations that were model-less at the time), that's it's probably best to have a moderation setting that clearly communicates what you've described here.
Either charities like the Gates Foundation and Good Ventures are hoarding money at the price of millions of preventable deaths
My assumption before reading this has been that this is the case. Given that, does a reason remain to update away from the position that the GiveWell claim is basically correct?
For the rest of this post, let's suppose the true amount of money needed to save a life through GiveWell's top charities is 50.000$. I don't think anything about Singer's main point changes.
For one, it's my understanding that decreas...
Was your posting this inspired by another criticism of Givewell recently published by another former staffer at Givewell?
So, first of all, when you write this:
Either charities like the Gates Foundation and Good Ventures are hoarding money at the price of millions of preventable deaths, or the low cost-per-life-saved numbers are wildly exaggerated.
It seems like what you're trying to accomplish for rhetorical effect, but not irrationally, is to demonstrate that the only alternative to "wildly exaggerated" cost-effectiveness estimates is that foundations like these are doing something even worse, that they are hoarding money. There are a few problems with this.
Singer's argument is that
1) We have a moral obligation to try to do the most net good we can.
2) Your obligation to do so holds regardless of distance or the neglect of others.
3) This creates an unconventionally rigorous and demanding moral standard.
Benquo's is that
1) Even the best charity impact analysis is too opaque to be believable.
2) The rich have already pledged enough to solve the big problems.
3) Therefore, spend on yourself, spend locally, and on "specific concrete things that might have specific concrete benefits;" also, try to ...
Doesn't this only hold if you abdicate all moral judgment to Gates/Good Ventures, such that if Gates Foundation/Good Ventures pass up an opportunity to save lives, it follows necessarily that the offer was fraudulent?
I would not, in fact, save a drowning child.
Or rather, I'd save a central example of a drowning child, but I wouldn't save a drowning child under literally all circumstances, and I think most people wouldn't either. If a child was drowning in a scenario similar to one that Singer uses it as an analogy for, it would be something like a scenario where there is an endless series of drowning children in front of me with an individually small but cumulatively large cost to saving them. Under those circumstances, I would not save every drowning child, or even try to maximize the number of drowning children I do save.
There's 2 reasons people far away may not try to save these lives.
1) They don't know if others are donating, maybe they don't need to.
2) They can't see the foundation acting on their part, maybe they will fail rescuing them, or maybe they are only cashing in.
There's a large feedback issue here.
Stories such as Peter Singer's "drowning child" hypothetical frequently imply that there is a major funding gap for health interventions in poor countries, such that there is a moral imperative for people in rich-countries to give a large portion of their income to charity. There are simply not enough excess deaths for these claims to be plausible.
Much of this is a restatement of part of my series on GiveWell and the problem of partial funding, so if you read that carefully and in detail, this may not be new to you, but it's important enough to have its own concise post. This post has been edited after its initial publication for clarity and tone.
In his 1997 essay The Drowning Child and the Expanding Circle, Peter Singer laid out the basic argument for a moral obligation to give much more than most to, for the good of poor foreigners:
To challenge my students to think about the ethics of what we owe to people in need, I ask them to imagine that their route to the university takes them past a shallow pond. One morning, I say to them, you notice a child has fallen in and appears to be drowning. To wade in and pull the child out would be easy but it will mean that you get your clothes wet and muddy, and by the time you go home and change you will have missed your first class.
I then ask the students: do you have any obligation to rescue the child? Unanimously, the students say they do. The importance of saving a child so far outweighs the cost of getting one’s clothes muddy and missing a class, that they refuse to consider it any kind of excuse for not saving the child. Does it make a difference, I ask, that there are other people walking past the pond who would equally be able to rescue the child but are not doing so? No, the students reply, the fact that others are not doing what they ought to do is no reason why I should not do what I ought to do.
Once we are all clear about our obligations to rescue the drowning child in front of us, I ask: would it make any difference if the child were far away, in another country perhaps, but similarly in danger of death, and equally within your means to save, at no great cost – and absolutely no danger – to yourself? Virtually all agree that distance and nationality make no moral difference to the situation. I then point out that we are all in that situation of the person passing the shallow pond: we can all save lives of people, both children and adults, who would otherwise die, and we can do so at a very small cost to us: the cost of a new CD, a shirt or a night out at a restaurant or concert, can mean the difference between life and death to more than one person somewhere in the world – and overseas aid agencies like Oxfam overcome the problem of acting at a distance.
Singer no longer consistently endorses cost-effectiveness estimates that are so low, but still endorses the basic argument. Nor is this limited to him. As of 2019, GiveWell claims that its top charities can avert a death for a few thousand dollars, and the Center for Effective Altruism claims that someone with a typical American income can save dozens of lives over their lifetime by donating 10% of their income to the Against Malaria Foundation, which points to GiveWell's analysis for support. (This despite GiveWell's long-standing disclaimer that you shouldn't take its expected value calculations literally). The 2014 Slate Star Codex post Infinite Debt describes the Giving What We Can pledge as effectively a negotiated compromise between the perceived moral imperative to give literally everything you can to alleviate Bottomless Pits of Suffering, and the understandable desire to still have some nice things.
According to the 2017 Global Burden of Disease report, around 10 million people die per year, globally, of "Communicable, maternal, neonatal, and nutritional diseases.”* This is roughly the category that the low cost-per-life-saved interventions target. If we assume that all of this is treatable at current cost per life saved numbers - the most generous possible assumption for the claim that there's a funding gap - then at $5,000 per life saved (substantially higher than GiveWell's current estimates), that would cost about $50 Billion to avert.
This is already well within the capacity of funds available to the Gates Foundation alone, and the Open Philanthropy Project / GiveWell is the main advisor of another multi-billion-dollar foundation, Good Ventures. The true number is almost certainly much smaller because many communicable, maternal, neonatal, and nutritional diseases do not admit of the kinds of cheap mass-administered cures that justify current cost-effectiveness numbers.
Of course, that’s an annual number, not a total number. But if we think that there is a present, rather than a future, funding gap of that size, that would have to mean that it’s within the power of the Gates Foundation alone to wipe out all fatalities due to communicable diseases immediately, a couple times over - in which case the progress really would be permanent, or at least quite lasting. And infections are the major target of current mass-market donor recommendations.
Even if we assume no long-run direct effects (no reduction in infection rates the next year, no flow-through effects, the people whose lives are saved just sit around not contributing to their communities), a large funding gap implies opportunities to demonstrate impact empirically with existing funds. Take the example of malaria alone (the target of the intervention specifically mentioned by CEA in its "dozens of lives" claim). The GBD report estimates 619,800 annual deaths - a reduction by half at $5k per life saved would only cost $3 billion per year, an annual outlay that the Gates Foundation alone could sustain for over a decade, and Good Ventures could certainly maintain for a couple of years on its own.
GiveWell's stated reason for not bothering to monitor statistical data on outcomes (such as e.g. malaria incidence and mortality, in the case of AMF) is that the data are too noisy. A reduction like that ought to be very noticeable, and therefore ought to make filling the next year's funding gap much more appealing to other potential donors. (And if the intervention doesn't do what we thought, then potential donors are less motivated to step in - but that's good, because it doesn't work!)
Imagine the world in which funds already allocated are enough to bring deaths due to communicable, maternal, neonatal, and nutritional diseases to zero or nearly zero even for one year. What else would be possible? And if you think that people's revealed preferences correctly assume that this is far from possible, what specifically does that imply about the cost per life saved?
If the low cost-per-life-saved numbers are meaningful and accurate, then charities like the Gates Foundation and Good Ventures are hoarding money at the price of millions of preventable deaths. If the Gates Foundation and Good Ventures are behaving properly because they know better, then the opportunity to save additional lives cheaply has been greatly exaggerated. My former employer GiveWell in particular stands out, since it publishes such cost-per-life-saved numbers, and yet recommended to Good Ventures that it not fully fund GiveWell's top charities; they were worried that Good Ventures would be saving more than their "fair share" of lives.
In either case, we're not getting these estimates from a source that behaves as though it both cared about and believed them. The process that promoted them to your attention is more like advertising than like science or business accounting. Basic epistemic self-defense requires us to interpret them as marketing copy designed to control your behavior, not unbiased estimates designed to improve the quality of your decisionmaking process.
We should be more skeptical, not less, of vague claims by the same parties to even more spectacular returns on investment for speculative, hard to evaluate interventions, especially ones that promise to do the opposite of what the argument justifying the intervention recommends.
If you give based on mass-marketed high-cost-effectiveness representations, you're buying mass-marketed high-cost-effectiveness representations, not lives saved. Doing a little good is better than buying a symbolic representation of a large amount of good. There's no substitute for developing and acting on your own models of the world.
As far as I can see, this pretty much destroys the generic utilitarian imperative to live like a monk while maximizing earnings, and give all your excess money to the global poor or something even more urgent. Insofar as there's a way to fix these problems as a low-info donor, there's already enough money. Claims to the contrary are either obvious nonsense, or marketing copy by the same people who brought you the obvious nonsense. Spend money on taking care of yourself and your friends and the people around you and your community and trying specific concrete things that might have specific concrete benefits. And try to fix the underlying systems problems that got you so confused in the first place.
* A previous version of this post erroneously read a decadal rate of decline as an annual rate of decline, which implied a stronger conclusion than is warranted. Thanks to Alexander Gordon-Brown to pointing out the error.