The Schistosomiasis Control Initiative (SCI) told us earlier this year that it had received a donation of praziquantel (the more expensive of the drugs it uses for deworming) from World Vision. Since it already has the drugs, donations can pay just for delivery costs. Thus, SCI observed, your gift is “leveraged” - $1 in donations buys more than $1 in health programming, since the drugs are essentially “free.”

The Against Malaria Foundation (AMF) in some ways achieves leverage through the opposite strategy. It doesn’t pay for the costs of distributing nets or even shipping nets - it only pays for the nets themselves. Partners are charged with raising funding elsewhere for shipping and distribution (though they are required to do these things; nets aren’t granted without a distribution plan in place). By adhering to this policy, AMF puts other funders - including some government agencies - in a similar situation to SCI’s donors: the nets are essentially “free,” so $1 from these agencies buys far more than $1 in health programming. Because of this dynamic, these other funders (AMF argues, correctly from what we can see) are happy to provide this funding - so the original donor’s gift is “leveraged” too (by buying just a net, the donor gets “free” shipping/distribution costs).

How far could this logic be extended? What if a charity first raised money for travel expenses, then went to another donor and raised money for field staff arguing that “travel expenses are essentially free,” then went to another donor and raised money for supplies arguing that “field staff and travel expenses are essentially free,” etc.? What if a charity went to 50 million separate donors, asking each to give $1 contingent on each of the other 49,999,999 giving as well, arguing that this created 50,000,000:1 leverage for each of the $1 donors?

We think that much of our discussion of donation matching applies here. If you take claims of “leverage” literally, you’re allowing other funders - the ones who gave pills, or nets, or covered distribution costs - to influence your giving merely through the structure of their gift. That in turn creates incentives for them to take gifts they would have made anyway, and structure them in a way that gets you to give more to the program of their choice. In fact, these other donors may even be taking advantage of you to pay for costs they would have been happy to cover themselves. Perhaps World Vision would pay for distribution, but knows that by giving “only pills” it can get others to pay for distribution. Perhaps AMF would pay for distribution if it had to, and doesn’t only because it believes it can get others to.

Understanding the true nature of leverage - who is the “leverager” and who is the “leveragee” - is difficult. When someone else is giving contingent on your giving, and this fact in turn influences your giving … sometimes this means that you’re getting them to give more (or differently) than they would have otherwise (in which case you’re the “leverager”) and sometimes it means the reverse (in which case you’re the “leveragee”). We don’t know of any easy/reliable way to tell which situation you’re looking at, and when.

With this in mind, our general principles for considering leverage are:

• You can’t take “leverage factors” at face value, for largely the same reasons that you can’t take donation matches at face value.
• It’s probably a good thing when the charity you support is engaging in some sort of “leveraging.” This means it’s putting thought into getting your funds to influence others’ funds. If you believe that you’re supporting the most impactful charity possible, this means that others’ funds may be moving from less impactful activities to more impactful activities. (Though the situation isn’t this simple: we believe that governments and other large funders often have a much better array of options for impactful giving than the individual donors we serve.)
• The exact relationship between the claimed “leverage” and your actual impact is very non-straightforward. It may seem that you have more “leverage” when your $1 causes someone else to spend $9, as opposed to when it gets someone else to spend 10c. However, in the former situation, there’s a much higher probability that you are in fact the “leveragee” - that the other funder would cover the other 10% themselves if it weren’t for you, and you’re just saving them money. In general, it seems to us that the higher your claimed “leverage,” the greater the probability that someone else is in fact leveraging you.

  If anything, we would guess that your true impact is higher when the “leverage” is well under 100% (i.e., the other funder is giving less than $1 for each $1 you give), implying that the other party is likely providing funding because of the “leverage” they attain rather than because they would want to fund the project on their own. Once you have another major funder covering a major chunk of the costs, it becomes more important to ask whether they would be willing to cover all of them if the “leverage” from you and similar donors weren’t available.

When we do cost-effectiveness estimates (e.g., “cost per life saved”) we consider all expenses from all sources, not just funding provided by GiveWell donors. For SCI, we count both drug and delivery costs, even when drugs are donated. (Generally, we try to count all donated goods and services at market value, i.e., the price the donor could have sold them for instead of donating them.) For AMF, we count net costs and distribution costs, even though AMF pays only for the former. In the case of VillageReach, we even count government costs of delivering vaccines, even though VillageReach works exclusively to improve the efficiency of the delivery system.

We consider this approach the simplest approach to dealing with the issues discussed here, and given our limited understanding of how “leverage” works, we believe that this approach minimizes the error in our estimates that might come from misreading the “leverage” situation. As our understanding of “leverage” improves, we may approach our cost-effectiveness estimates differently.

We know that donors love donation matching.* We know that if we could offer donation matching on gifts to our top charities this giving season, our money moved would rise. And we know that we could offer donation matching if we thought it was the right thing to do: there are donors planning six-figure gifts to our top charities this year who would almost certainly be willing to structure their gifts as “matches” if we asked. (It might not be possible to “match” all of our money moved, but we could almost certainly provide “matching” for a short period, which would motivate people to give during that period and would also provide us with some data on the impact of matching on our audience.)

But we’ve decided not to do this because we would feel dishonest. We’d be advertising that you can “double your gift,” but the truth would be that we just restructured a gift from a six-figure donor that was going to happen anyway. We’ve discussed whether we might be able to provide “true” donation matching - finding a donor who would give to our top charities only on condition that others did - but not surprisingly, everyone we could think of who would be open to making a large gift to our top charities would be open to this whether or not we could match them up with smaller donors. Ultimately, the only match we can offer is illusory matching.

I don’t deny that non-illusory matching may exist in some other circumstances. A couple possibilities:

• Coordination matching. A charity needs to raise a specific amount for a specific purpose. A large funder (the “matcher”) is happy to contribute part of the amount needed as long as the specific purpose is achieved; therefore, the matcher makes the gift conditional on other gifts.
• Influence matching. The matcher wishes both to support a particular charity and to encourage others to give to that charity. Therefore, the matcher makes a legitimate commitment to give only if others do, in an attempt to influence their giving.

In both of these cases, it may seem at first glance that a one-to-one match really does “double” your donation, but I don’t think it’s quite that simple.

Regarding coordination matching - I would guess it’s relatively common for a funder to say privately, for example, “I’ll give $100,000 if you can raise the remaining needed $900,000.” But there are a couple of problems when it comes to advertising this situation as a “match.” First, saying “every $9 you give will be matched with $1 from a major donor” wouldn’t be very psychologically compelling - matches rarely go below the 1:1 threshold. Second, even if the funder were providing enough for a 1:1 match, it still wouldn’t be quite true that each $1 was matched with another $1: the match would occur only in the case that the total amount needed was raised. So while “coordination matching” is a possibility, we would guess that it rarely explains the “each $1 you give will be matched by $1″ campaigns commonly used in fundraising.

Influence matching is something I think impact-maximizing donors ought to be concerned about. In the short run, influence matching makes it true that your $1 donation results in $2 donated to the charity in question. But it also means that you’ve let the matching funder influence your giving - perhaps pulling you away from the most impactful charity (in your judgment) to a less impactful one - just by the way they structured their gift. By giving, you are rewarding this behavior by the matching funder, and you may be encouraging them to take future unconditional gifts and turn them into conditional gifts, because of the ability to sway other donors.

Perhaps, rather than giving your $1 to the charity the matching funder is pushing, you should fight back by structuring your own influence matching - making a conditional commitment to the highest-impact charity you can find, in order to pull other dollars in toward it.

For the average donation match, it’s unclear to what extent the match represents illusory matching vs. coordination matching vs. influence matching. My guess is that coordination matching is by far the least common (since it requires such a specific set of circumstances to hold) and that illusory matching is the norm (since this is generally the easiest to offer, and since donors don’t tend to distinguish between the different types when they decide where and how much to give).

Corporate matching programs sometimes match only gifts to specific charities; in this case I think it’s best to think of them as “influence matching.” If the company offered matching to any charity (as some companies do) and/or simply made gifts to the charities of its choice, it would no longer be pushing its employees to support specific charities. If you are employed at a company offering matching only on specific charities, I recommend pushing for a change in policy (to unconditional gifts to charities and/or unconditional matching for employees, as other companies do) rather than perpetuating a dynamic where your company’s corporate philanthropy team decides where you give.

In general, I advise donors seeking to maximize their impact to simply support the most impactful charity possible, and not to factor in the presence or absence of donation matching either way. If you support a less impactful charity due to the presence of a match, you may be having more total impact, but you also may be having substantially less (in the case of illusory matching) and/or contributing a dynamic that leads to less effective giving broadly (a risk both for influence matching and illusory matching).

GiveWell may offer donation matching sometime in the future. If so, we will be explicit about whether it is influence matching or coordination matching (we wouldn’t be comfortable offering illusory matching, except perhaps as a joke - i.e., “If you’re thinking of giving to another charity just because of a donation match, let us know and we’ll get your donation to a top-rated charity matched”). If we do implement influence matching, it will be to (a) fully neutralize the effect of other matches on impact-oriented donors, further encouraging them to support the most impactful charities possible; (b) raise money from non-impact-oriented donors who are happy to have their donation “matched” despite the logic above.

*“Donation matching” refers to when a large funder offers to give $X to a particular charity for every $Y other people give - for example, “For every $1 you give to this charity, a large funder will contribute another $1, doubling your impact!” For more, see the 2007 study on donation matching by Dean Karlan.

Normally, we focus on identifying outstanding charities, and minimize the time spent on opaque or otherwise lackluster ones. But lately, we’ve gone into a bit more detail about our take on several of the best-known and most appealing charities out there.

What all of the charities below have in common is that (a) we have major questions and concerns about their activities; (b) the information necessary to see how serious these concerns are does not seem to be available. (In most cases our assessment is based on significant back-and-forth with the charities themselves, though in some cases we are going off their website.)

• Kiva: commonly perceived as a way to (a) make interest-free loans to (b) entrepreneurs you personally select. In fact, it is neither. (See GiveWell Board member Tim Ogden’s summary of the recent discussion/controversy.)

  We ultimately see a Kiva “loan” as a gift to a microfinance institution, and we have major concerns about supporting any but the most outstanding microfinance institutions as well as concerns about Kiva’s due diligence process specifically.

  Full set of blog posts on Kiva

• Grameen Foundation and other large U.S. microfinance charities: in addition to our general concerns about microfinance charities (and the fact that these charities have not provided the information that would address them), we find that these organizations are rarely even clear about what their value-added is. Also see the disappointing (and eye-opening) evaluation of the Village Phone program.

  Full set of blog posts on Grameen Foundation

• Heifer International: commonly perceived as a way to “give a cow to a poor family as a gift,” but this is in fact a donor illusion -donations support Heifer International’s general “agricultural assistance” activities. We have concerns about this general area and concerns about giving livestock specifically. Neither Heifer’s website nor its grant application have provided the kind of information needed to address these concerns.

  Full set of blog posts on Heifer International

• Smile Train: Prof. Steven Levitt of Freakonomics sees it as using an innovative model to deliver cleft surgeries for $250 apiece. Our analysis suggests that it has “more money than doctors” and that it is therefore out of room for more funding. Smile Train’s recent removal of the charts we questioned and its own explanation of why it is giving grants to other major charities appear to support our analysis. In addition, we have major concerns about surgery charities that Smile Train has not provided enough information to address.

  Full set of blog posts on Smile Train

• UNICEF offers donors the chance to “buy” vaccines and other items. We suspect that this offer is a donor illusion, but the larger issue is that UNICEF provides almost no information at all about its huge variety of activities and how they are going. Most giant international aid charities (Save the Children, World Vision, etc.) are similarly complex and opaque, with CARE coming the closest to meaningful transparency.

  Full set of blog posts on UNICEF

• The Acumen Fund stresses its metrics-driven approach to “patient capital,” investing in businesses that aim for long-term sustainability (i.e., profitability) and social impact. As of now, we see no substantive available information about its track record.

  Full discussion

• The Robin Hood Foundation also stresses its emphasis on metrics, quantification, and analytical thinking. But we have found that it is unwilling to share any information about the results of its grantmaking. (Note that it also pushes the “0% overhead” donor illusion.)

  Full set of blog posts on The Robin Hood Foundation

• The Millennium Villages project states that it is “demonstrating that it’s possible to escape … extreme poverty” (front page of its website). As far as we can tell, very few results have been demonstrated to date, with a few boosts in crop yields as its main evidence of impact - evidence we feel is highly questionable.

  Full discussion

• The Worldwide Fistula Fund has been praised by Nicholas Kristof as “giving new life to the pariahs.” Our assessment is that the Fund is pushing a debatable agenda whose effectiveness has yet to be shown.

  Full discussion

• The Carter Center is, in our view, by far the “best bet” (for an individual donor, our focus) of the charities listed in this post. It has achieved substantial, demonstrable impact in the past through its work on guinea worm eradication, and unlike all of the charities listed above, it publicly discloses substantive technical information on the impact of its past activities.

  However, there is a crucial piece of information missing: how it allocates its budget between programs (in particular, how much it spends on its stronger and better-documented vs. more questionable programs). It has declined to share this (basic, in our view) information in repeated exchanges.

  Full set of blog posts on The Carter Center

We think the above charities are fairly representative of “average” charities in international aid. Some tell better stories than others and some have more disclosure than others. But in almost all cases, international aid charities are (a) carrying out complex projects that can fail to do good (or even do harm) in a variety of ways, and (b) not systematically sharing the information that would make it possible to assess how their work is going.

GiveWell is devoted to finding charities in which we can have more confidence. We’ll be discussing our two top-rated charities working internationally in forthcoming posts.

For an organization focused on financial metrics, the American Institute of Philanthropy can be very interesting. I can’t do justice to this excellent article on Smile Train with an excerpt, and I urge you to read it all.

It thoroughly debunks an alleged claim by Smile Train that “100% of your donation goes toward programs — 0% goes toward overhead.” Smile Train currently seems to have backed off this claim at least somewhat, although Steven Levitt of Freakonomics appears to have been sold this story (and to have bought it) in 2006.

If identifying effectiveness with “low overhead” is silly, the idea of “0% overhead” simply seems absurd. As the article shows, it doesn’t (and can’t) mean that there are no operating costs affecting the total costs of the program. Rather, it’s another case of zooming in on “your” money, rather than discussing the true total costs of the program you’re supporting the existence of. It makes no sense in an analytical framework; it’s a feel-good gimmick.

That’s why we were surprised when we first saw this gimmick prominently displayed by a group that many consider to be the epitome of hard-nosed, analytical giving: Robin Hood.

Robin Hood’s financials make the situation look similar to Smile Train’s (minus the questionable reclassification of funds that AIP attributes to the latter). About 11% of Robin Hood’s total expenses are “Administration salaries and overhead” or “Fundraising and Public Information,” but because Board member donations are earmarked to those expenses, everyone else can be told their donations are “overhead-free.”

If your goal were to minimize overhead, the fact that Robin Hood tags funds this way shouldn’t be very relevant to you. Robin Hood could allocate more of those Board donations to programs if it spent less on overhead. If you gave to another organization, you could be scaling up an overall lower-overhead operation.

Bottom line: The “0% overhead” claim is promoting the wrong metric (low overhead) and offering a false way to accomplish it.

Peter Singer has explained the problem with the “net = life” equation, and any other serious analysis we’ve seen of insecticide-treated nets agrees.

Why does this matter? Because Nothing But Nets also prominently states that the total cost of each net is $10. For donors looking to maximize “bang for the buck,” $10 per life saved would probably be the best option available - if it were a real option.

As it happens, distributing bednets is one of the most cost-effective programs we know of (at least when it works) - it’s just nowhere near the $10 figure.

There are probably many reasons that impact-focused giving is so rare, but we think that at least one factor is that when a donor does go looking for information, so many of the “numbers” and “facts” they run into are exaggerations and illusions. It’s a frustrating experience that leaves some jaded about the whole endeavor, and thus missing out on real chances to have an enormous impact.

Two days ago, I critiqued the message sent by charts on Smile Train’s “Financial Information” page, particularly the idea that a surgery was being provided for every $250 in donations.

My recollection is that the page had been up in this form since at least the summer of 2006 (when I first started investigating surgery charities), and Internet Archive appears to confirm that it’s been up at least since early 2007.

We have not heard/seen a response from Smile Train, but as of last night, that page appears to have been taken down entirely.

Smile Train is still claiming $250 as the cost per surgery, which (I argued) seems contradicted by the now-taken-down charts.

Details

Two pages appear to have changed: the “charts” page (the page I critiqued) is now gone, and the “financials” page now longer links to the “charts” page.

“Charts” page:

• 11/25/09 Google cache (this link won’t be good long-term) and screenshots of Google cache: top half, bottom half
• The URL now goes to a “not found” page
• Full page as it originally appeared

“Financials” page:

• 11/27/09 Google cache (won’t be available long-term) and screenshot of Google Cache. Includes a “Click to see our latest financial information” link.
• The current version of the page does not include a “Click to see our latest financial information” link.