Pay charities for results?

by BenGilbert 3 min read8th Feb 201318 comments



Two problems with charity:


1) You usually don't know what your donation achieves. At best, you might know what your money is spent on. You don't know how effective this is at producing the outcomes you care about. Even Givewell, who seem to me to have done more careful work on cost-effectiveness than anyone else, regard their cost-effectiveness estimates as very rough and no more than an indicative starting point for evaluating charities.


2) Charities have low or no financial incentives to be as effective as they can, not least because usually no-one knows how effective they are.



Potential solution:


Instead of donating to charities, pay them for results achieved.


Ideally, you would pay for the final outcomes which you care about, eg paying a certain amount per unit reduction in child mortality, reduced disease prevalence, improved test scores, etc. If this is too difficult, then you could pay for intermediate results, eg number of children vaccinated, number of people protected by bednets, etc. Results could be measured against a control group, some baseline, outcomes in parts of the country where the charity doesn't operate etc. (Comparison with a well-constructed control group would probably be best in most cases).


This isn't really something that an individual donor can do, since it relies on accurate, independent measurement of results and will be most effective when charities know that their funding depends on the results they achieve. To work best, it would have to happen in a co-ordinated way and at a large enough scale that proper measurement is affordable.





1) You only pay if a charity is effective at doing what you want it to do. You have less need to try to understand what a charity does; you can offer the money for the results and leave it to them to find how to produce them.


2) Charities will have financial incentives to be as effective as possible, including finding out how effective they already are and what they could do to be more effective.


3) Effective charities will get more money and be able to expand, ineffective charities will get less money and may have to close down.


4) Lots of information would be produced about what is and is not effective, which could help donors, aid agencies, charities etc make better decisions in future, whether or not they are paying / being paid for results.



Disadvantages relative to the current system of donating:


1) Measuring the results costs money; it might be better just to donate this money.


2) It might be possible to manipulate or falsify the results.


3) Where you cannot measure what you really care about (eg the long-term improvement in someone's life from their receiving education), focusing on intermediate results might make a charity less effective (eg teaching to the test, leading to less learning of things that will help later).


4) Charities may be able to produce better results in ways that produce negative, unmeasured side-effects. They may be more likely to do this when they have financial incentives to improve measured results.



Discussion of disadvantages


1) As well as potentially improving incentives, measurement could create very valuable information, especially since the evidence base for most charitable activity is very weak. It is likely to be money well spent.


2) Obtaining accurate results is a methodological issue, with technical solutions, though cost is a constraint. Falsification is certainly a danger. On the other hand, charities found to be falsifying results would face huge reputational costs; they would have strong incentives not to try. Also, charities with integrity might be unlikely to try, whilst the charities that would be willing to try might well be those that are already misspending money donated to them in the ordinary way; with an attempt at measurement, there is at least a chance of exposing this.


3) Targeting measures which become progressively worse guides to the actual good being done is a danger. Choosing a good measure – one which is hard to achieve without achieving the outcomes you ultimately care about – could make this a smaller problem. Even so, whenever the ultimate outcome isn't targeted directly, there is likely to be some diversion of resources from what a charity thinks does the most good, to what it thinks will improve measured results. However, this may be a gain, not a loss, if activities which improve measured results in fact do more good than the things which the charity believes are effective; and, if it is a loss, it has to be weighed against the gains from improved transparency, feedback etc.


As a separate issue, when the ultimate outcome cannot be targeted directly, a theory of why the intermediate targets are effective ways of achieving that outcome is needed, but this is no less true of charitable projects now. Also, there may be no measure with a close enough relation to a charity's ultimate goal that can usefully be measured; for such charities, paying them for results will not be appropriate.


4) If a charity's activities do more harm than good, then donations and payments for results would both be bad. Payment for results might be particularly bad if it leads to more money going to charities that do more net harm, or changes how charities function such that the net good they do goes down, because they pursue results in more harmful ways. Weighing up how big a problem this is would require some understanding of how the charities involved function (so, in fact, it's not quite so simple as 'offer money for results and leave it to them to find how to produce them'). The seriousness of the problem could be very different for different areas, regions or organisations; in any case, I think it's worth bearing in mind.




Bottom line for me: the advantages seem to outweigh the disadvantages, and I would be more likely to give to charity if I could do it this way. At the very least, I would like to see it tried.


(This approach is actually beginning to be taken, though only in a tiny minority of projects, and not in ways to which an individual donor can contribute, as far as I can see – subject for another post, perhaps.)