You may want to read up on the burgeoning area of charity success or social impact bonds: http://www.nytimes.com/2011/02/09/business/economy/09leonhardt.html http://en.wikipedia.org/wiki/Social_impact_bond
Note that the metrics have to be carefully designed to avoid the obvious principal-agent problems/Goodhart's law/Lucas critique.
Yes, it's thinking about social impact bonds and the like that started me thinking about paying charities for results. What I would note, though, is that, at the moment, individual donors/investors can invest in social impact bonds. What they cannot do is offer money for the payouts to investors in social impact bonds. So, you can put money into a bond which pays out if charities are effective at, eg, reducing the number of children who go into care homes in the UK. (this is a real example). But you'd only want to do that if you were confident that the charity was effective, otherwise you'll both lose out financially and have achieved no good. I'd be more interested in adding money to the fund which pays out for results I care about, thus giving investors who might have a better idea of what is and isn't effective incentives to put money into the charities they think will produce good results.
Getting a large and statistically unbiased control group would seem to be practically impossible in many cases, yet having one would be utterly essential.
An old scam: "Send us $50, and we will use our connections to get your child admitted at the college of their choice! Guaranteed, with a full refund given in the event of a rejection!" Then the scammers do nothing, refund the portion of the money sent in by applicants who were later rejected, and keep the portion of the money sent in by applicants who were later accepted.
People are usually wary enough to avoid distributing their money this way, but for votes and sociological beliefs it seems to be a more tried-and-true approach. "Having control of the most prominent political office when a business cycle goes boom" and "having control when a business cycle goes bust" seem to be reliable ways for political parties to gain and lose mindshare even when it is very unlikely that such a rapid and strong connection exists between that office and that economic status. Similarly, every time crime rates go significantly up or down, everyone is eager to blame contemporaneous policy X or credit contemporaneous policy Y, usually despite prominent examples of neighboring polities which didn't implement X/Y but saw the same type of crime rate change at the same time.
I agree that having a good control group would be very important. I also agree that for many charitable activities this would be impossible. However, in many other activities, it would be possible. A good number of randomized controlled trials are being run to measure the effectiveness of health, educational and even political/governance activities, so they can be done. They might be costly, but that's a separate question, and, as I said, there could be very large benefits in improved knowledge from doing them.
I think your scam analogy is somewhat off-target. A closer analogy would be that, for donations at the moment, they ask you to send $50 and don't offer any refund if your child is rejected. You just have to trust them, or examine them closely enough, to be confident they are doing something effective. A payment for results arrangement with a badly constructed control group might be like your analogy, but even this might be some sort of improvement, because there is now at least an incentive to reduce the number of rejections, even if some of the money is going for things that would have happened anyway. And, with a good control group, you would only be paying for things you could be confident were the results of the charity's activity.
An old scam: "Send us $50, and we will use our connections to get your child admitted at the college of their choice! Guaranteed, with a full refund given in the event of a rejection!" Then the scammers do nothing, refund the portion of the money sent in by applicants who were later rejected, and keep the portion of the money sent in by applicants who were later accepted.
I saw that used as a joke on Facebook (though the actual example was “I will use sorcery to predict your expected baby's sex for $1000, and I will refund you if I'm wrong”), but I didn't think anyone had used that in real life. (But I really oughtn't be surprised this much.)
Another disadvantage that you haven't mentioned:
5) Starting a new charity suddenly requires an investment and carries a risk. If you fail to live up to expectations, then you are suddenly in debt.
I'm not sure if this is a disadvantage or an advantage. I think it might be a good thing if people starting charities thought more about what confidence they had in how effective their activities would be, rather than simply having good intentions and an idea of how they might be able to fulfil them. I'd be happier giving my money to a charity whose founders stood to lose something themselves if their work turned out to be ineffective, and welcomed the chance to prove their effectiveness one way or the other. However, I agree that people are often too risk-averse and that there might well be a need for charities to be able to share some at least of the risk with lenders who had social as well as financial goals.
The point is that this requires making founders residual claimants of their "charities", otherwise no-one is going to want to start one because the risk is all on the downside. Which means that you actually have a social enterprise, rather than a charity.
If we had an excellent sense of what charities were effective at given tasks, we would just skip the middle step of paying for results and donate directly - in effect, paying for results that already happened. The hard problem in funding charities remains determining if they're effective, not compensating them.
I see where you're coming from, but I see 3 advantages to paying for results. (1) This approach involves facing head-on the challenge of determining whether a charity is effective, which may be hard but is surely vitally important to answer. (2) It creates incentives for charities which are already effective to become even more so. (3) It could help to foster a system of charitable funding in which money goes to effective charities, not because experts have examined how they function and concluded that they work well, but just through the feedback processes which reward effective and punish ineffective charities. When charities operate in complex environments where the consequences of their activities are not always easy to predict, this sort of system might do better than expert evaluation.
I'm not sure how you intend payment to work. Is the idea to do something like 'If you, charity X, can show me clear proof that you have vaccinated Y kids, I agree to pay you Z dollars'?
There seems to be a number of problems with this system, not least that they need your money in order to achieve the results in the first place. Nevertheless there does seem to be a (limited) space for X-prize type rewards, to spur interest and funding.
However if the system is something like 'Charity X can show clear proof that they have been able to consistently do good work, so I will donate to them' then I think that's already similar to how a lot of effective altruists donate already.
There seems to be a number of problems with this system, not least that they need your money in order to achieve the results in the first place.
One could even imagine an entirely for-profit firm that gives free vaccines to poor third-world kids. Initial funding could come from the firm's investors or loans (i.e. the same ways all for-profit firms get funding). A charitable person inks a contract for "delivery" of 10,000 vaccinated kids for a given amount of money. The firm has an incentive to deliver this efficiently and quickly, so they can pocket the difference. Competition between firms then keeps the profit margins reasonable.
The main disadvantage is that non-profit groups would have huge tax advantages over for-profit firms.
Yes, the first idea is what I had in mind. The problem with getting financing in order to achieve results has a few possible solutions, I mention some below in the response to Kawoomba. Which other problems did you have in mind?
The advantages seem to me to work at various levels – improved incentives, much improved knowledge, and the possibility of a system emerging where you can leave it to the financial incentives and the feedback processes they create to bring about increased effectiveness and innovation.
Ideally, you would pay for the final outcomes which you care about, eg paying a certain amount per unit reduction in child mortality, reduced disease prevalence, improved test scores, etc. If this is too difficult, then you could pay for intermediate results, eg number of children vaccinated, number of people protected by bednets, etc.
A concern, as I see it, is the first part (paying for outcomes) is too difficult to successfully do on a large scale (though social impact bonds do seem promising from my first glances), and the second part (paying for outputs, like number of bed nets) already happens.
So the charity would ... borrow the money, at market prices? Rather, try to borrow, what with typically having little securities and all.
They can use equity financing of course. The money you've promised becomes the profits, to be distributed to shareholders in the event of success.
I definitely agree that financing charities is an issue, but there are various possibilities. You could finance them through donations for a few years while declaring that, from x years time onwards, your funding would depend on the results obtained using the money you donated; from that point on, the system would be self-sustaining for effective charities, and not for ineffective ones (which would be, I think, a good thing). Or they could borrow money from lenders who want to support charitable causes and are willing to lend at lower interest rates, or through a social impact bond structure where lenders provide money in return for some or all of the future results-based payments should the charity be effective. Or, finally, through borrowing at market rates, though I agree that this is likely to be uneconomical, at least until there is more evidence for judging which charities are likely to be able to produce good results and repay the money they borrow.
Two problems with charity:
1) You usually don't know what your donation achieves. At best, you might know what your money is spent on. You don't know how effective this is at producing the outcomes you care about. Even Givewell, who seem to me to have done more careful work on cost-effectiveness than anyone else, regard their cost-effectiveness estimates as very rough and no more than an indicative starting point for evaluating charities.
2) Charities have low or no financial incentives to be as effective as they can, not least because usually no-one knows how effective they are.
Potential solution:
Instead of donating to charities, pay them for results achieved.
Ideally, you would pay for the final outcomes which you care about, eg paying a certain amount per unit reduction in child mortality, reduced disease prevalence, improved test scores, etc. If this is too difficult, then you could pay for intermediate results, eg number of children vaccinated, number of people protected by bednets, etc. Results could be measured against a control group, some baseline, outcomes in parts of the country where the charity doesn't operate etc. (Comparison with a well-constructed control group would probably be best in most cases).
This isn't really something that an individual donor can do, since it relies on accurate, independent measurement of results and will be most effective when charities know that their funding depends on the results they achieve. To work best, it would have to happen in a co-ordinated way and at a large enough scale that proper measurement is affordable.
Advantages:
1) You only pay if a charity is effective at doing what you want it to do. You have less need to try to understand what a charity does; you can offer the money for the results and leave it to them to find how to produce them.
2) Charities will have financial incentives to be as effective as possible, including finding out how effective they already are and what they could do to be more effective.
3) Effective charities will get more money and be able to expand, ineffective charities will get less money and may have to close down.
4) Lots of information would be produced about what is and is not effective, which could help donors, aid agencies, charities etc make better decisions in future, whether or not they are paying / being paid for results.
Disadvantages relative to the current system of donating:
1) Measuring the results costs money; it might be better just to donate this money.
2) It might be possible to manipulate or falsify the results.
3) Where you cannot measure what you really care about (eg the long-term improvement in someone's life from their receiving education), focusing on intermediate results might make a charity less effective (eg teaching to the test, leading to less learning of things that will help later).
4) Charities may be able to produce better results in ways that produce negative, unmeasured side-effects. They may be more likely to do this when they have financial incentives to improve measured results.
Discussion of disadvantages
1) As well as potentially improving incentives, measurement could create very valuable information, especially since the evidence base for most charitable activity is very weak. It is likely to be money well spent.
2) Obtaining accurate results is a methodological issue, with technical solutions, though cost is a constraint. Falsification is certainly a danger. On the other hand, charities found to be falsifying results would face huge reputational costs; they would have strong incentives not to try. Also, charities with integrity might be unlikely to try, whilst the charities that would be willing to try might well be those that are already misspending money donated to them in the ordinary way; with an attempt at measurement, there is at least a chance of exposing this.
3) Targeting measures which become progressively worse guides to the actual good being done is a danger. Choosing a good measure – one which is hard to achieve without achieving the outcomes you ultimately care about – could make this a smaller problem. Even so, whenever the ultimate outcome isn't targeted directly, there is likely to be some diversion of resources from what a charity thinks does the most good, to what it thinks will improve measured results. However, this may be a gain, not a loss, if activities which improve measured results in fact do more good than the things which the charity believes are effective; and, if it is a loss, it has to be weighed against the gains from improved transparency, feedback etc.
As a separate issue, when the ultimate outcome cannot be targeted directly, a theory of why the intermediate targets are effective ways of achieving that outcome is needed, but this is no less true of charitable projects now. Also, there may be no measure with a close enough relation to a charity's ultimate goal that can usefully be measured; for such charities, paying them for results will not be appropriate.
4) If a charity's activities do more harm than good, then donations and payments for results would both be bad. Payment for results might be particularly bad if it leads to more money going to charities that do more net harm, or changes how charities function such that the net good they do goes down, because they pursue results in more harmful ways. Weighing up how big a problem this is would require some understanding of how the charities involved function (so, in fact, it's not quite so simple as 'offer money for results and leave it to them to find how to produce them'). The seriousness of the problem could be very different for different areas, regions or organisations; in any case, I think it's worth bearing in mind.
*****
Bottom line for me: the advantages seem to outweigh the disadvantages, and I would be more likely to give to charity if I could do it this way. At the very least, I would like to see it tried.
(This approach is actually beginning to be taken, though only in a tiny minority of projects, and not in ways to which an individual donor can contribute, as far as I can see – subject for another post, perhaps.)
Comments?