The GiveWell Blog

An essential question that no one is asking charities

(updated on: ) | by Holden

If a charity demonstrates that its core program has changed lives in the past, is likely to change lives in the future, and gets great “bang for your buck,” is this enough reason to donate to it? We say no.

The missing piece: Will more funding lead to more of the good program(s)? We generally call this the “room for more funding” question, and we’ve seen next to no helpful discussion of the issue within academia, within the nonprofit sector, or anywhere else.

Often, when I raise this issue, the response I get is “But is that a real problem? Are there charities that have great programs they can’t or won’t expand with more funding?” The answer is yes. Examples:

  • Our analysis of Smile Train strongly suggests that its core program of directly funding doctors has “more money than doctors.” Thus, over 50% of Smile Train’s funds go to activities far from what their fundraising focuses on, including grants to other organizations, research, and “provid[ing] materials on cleft lip and palate for free to anyone interested in this birth defect.” Perhaps these activities have value, but it would be a mistake to donate to Smile Train just based on their headline program.
  • The Aravind Eye Care System is one of the more impressive humanitarian organizations we have seen, performing vision-restoring surgery extremely cost-effectively. They have been so successful, in fact, that their core program doesn’t need donations – as they have explicitly told us. Revenue from for-pay surgeries subsidizes free surgeries, and donations are used on entirely and substantially different programs such as distribution of spectacles and free food.
  • Today’s Aid Watch post gives an excellent picture of why it’s so important to be wary:

    according to Fred Martin, Communications Director at CHF, “In fact our Food Pak program is a small portion of what we do. We highlight it because it is our flagship program that we’ve seen work very well in building relationships with the poor so that deeper needs can be uncovered and responded to.” I learned from Fred they also provide beds in eastern Europe and medicines in Asia …

    As long as charities can get away with it, their incentive is to advertise the best program they have, even well beyond the point where that’s the program that needs more money.

There can be many bottlenecks to expanding a program besides money (skilled labor, environments that are conducive to the program, etc.) If you want to fund great programs, you have to ask not just “What have you done and has it worked?” but “What will you do with more funds than you’re currently expecting?”

We haven’t identified any easy answer or simple formula for this question. We believe that “restricting” your donation to the program you favor is generally a futile endeavor (more on this in a future post).

We have developed some relevant ideas. In addition to some rules of thumb for avoiding the most tangled cases, we ask the strongest charities for documents that speak to the “room for more funding” issue directly, such as examples of un-funded but strong project proposals and financial “scenario analysis” (details to come in future posts).

However, we have found that requesting such documents is an uphill battle because the request is generally so foreign. You won’t see financial scenario analysis on any standard list of “documents a charity should be sharing” (from the IRS or anyone else).

Foundations arguably don’t need to deal with the challenge discussed here, because they can give money in large enough chunks to dictate which projects get carried out. (The extent to which this practice is wise is another question). This may be why no one else seems to be asking for information on “room for more funding.” Whatever the reason, it’s an issue that needs much more attention than it’s getting.

Charity isn’t about helping?

(updated on: ) | by Holden

One person who’s more critical of charity than we are or than David Hunter is is the economist Robin Hanson. He has stated that “charity isn’t about helping” and spelled out this view somewhat in a post about the founder of Rite Aid:

    when folks like Alex spend their later years trying to “do good” with the millions they were paid for actually doing good, they usually end up pissing it away. We already have too much medicine and academia, because such things are mainly wasteful signals. We didn’t need and shouldn’t be thankful for more hospital wings or lecture halls. Imagine how much more good could have been done instead via millions spent trying to make more innovative products or organizations.
    Of course most innovations attempts fail, and that wouldn’t have looked so good for Mr. Grass. I’m sure those hospital wings and lecture halls came with grand ceremonies attended by folks in his social circle, saying what a great guy he was. And I expect people in his social circle are more likely than most to actually use those hospital wings and lecture halls; he was showing loyalty to his clan by buying such things.
    But when I think of all the good that could be done by philanthropists who actually wanted more to do good than to look good, it makes me sad. At it doesn’t make me sympathetic toward the tax deductions and other social support our society offers for these wasteful signals.

Prof. Hanson tends to imply that charitable giving should be essentially ignored in favor of pro-poor causes like allowing more immigration.

What response can the nonprofit sector marshal to arguments like this? I must say that, in fact, much of the nonprofit sector fits incredibly better into Prof. Hanson’s view of charity as “wasteful signaling” than into the traditional view of charity as helping.

Perhaps ironically, if you want a good response to Prof. Hanson’s view, I can’t think of a better place to turn than GiveWell’s top-rated charities. We have done the legwork to identify charities that can convincingly demonstrate positive impact. No matter what one thinks of the sector as a whole, they can’t argue that there are no good charitable options – charities that really will use your money to help people – except by engaging with the specifics of these charities’ strong evidence.

Valid observations that the sector is broken – or not designed around helping people – are no longer an excuse not to give.

Because our Bayesian prior is so skeptical, we end up with charities that you can be confident in, almost no matter where you’re coming from.

Comment on Barron’s “25 Best Givers” list

| by Holden

In concept, I like the idea of showering praise on people based on their philanthropic impact, not merely dollars given (or dollars made).

But I am skeptical as to whether Barron’s did the research necessary to base its piece on facts as opposed to guesses.

Taking a look at this list, what jumps out at me is that #8 on the list has been recognized for founding the Robin Hood Foundation, and I simply cannot imagine what information this could be based on.

Estimating the cost-effectiveness of microfinance charity

(updated on: ) | by Holden

Note: I’ve responded to the most recent batch of comments.

A lot of work has been put into estimating the “bang for your buck” in health initiatives. In the area of microfinance, though, things appear very murky.

Microfinance advocates say things like “As our clients repay the loans, the money is loaned again and again to help many more entrepreneurs. It’s giving that keeps going.” Skeptics reply that much of the cost of lending is in operating institutions, not simply loan capital. We should be able to agree that the cost-effectiveness of microlending is not literally infinite, but what’s the right ballpark? Does the impact per dollar dwarf that of health?

We can take a very rough – and very generous to microfinance – cut by looking at some global estimates by CGAP. Notes before we get to the numbers:

  • We are trying to get a number that we can put alongside existing estimates of health cost-effectiveness, just to see whether the microfinance sector as a whole has a clear and large advantage in cost-effectiveness. The estimate will be extremely rough and will not apply to any given microfinance charity, but rather to the area of microfinance as a whole.
  • Our estimate is essentially a “best-case scenario” for what microfinance cost-effectiveness would be if (a) there were a direct link between donations and people served (b) microfinance could reach an enormous “target population” at the same level of donation funding that’s being provided now.

CGAP looks at both dollars invested in microfinance (PDF) and people served. According to these links,

  • $11.7 billion of funding went to microfinance in 2008, of which 19% – or ~$2.2 billion – was grants (not loans, not investments, not guarantees).
  • There are currently between 130 and 190 million microfinance borrowers worldwide.
  • CGAP implies a “target number” of borrowers: “Given that almost 3bn people live on less than two dollars a day, clearly the battle to bring financial access to as many people as possible is a very long way from being won.” I have major issues with this target – for one thing, I’m not sure that people living under $2/day should all be targets, or are the only targets, of MFIs.

A couple of ways to look at the “costs per MFI client”:

  • A lot of money is spent on microfinance. $2 billion in grants is about 10% as much as the total official development aid of the U.S. government (according to the 2008 Index of Global Philanthropy (PDF)).
  • We’re currently spending $12-$17 in grants alone for every MFI borrower. Of course, the grants could be paying for a lot more than borrowing (including savings), and could be made with the aim of expanding future services rather than maintaining existing ones.
  • If you believe that microfinance will eventually reach the entire CGAP “target population” (or a population that size, which would be around half the population of the world) and that the current level of grants will be maintained (say, growing only at the rate that the size of the target population grows), then at the point where microfinance is reaching its entire “target population,” the grants per person reached will be about $0.75. While this figure could be overstating the costs per person served if grants eventually create self-sustaining institutions and become unnecessary, I think it is far more likely that it understates the cost because (a) those who can most practically be reached in a profitable/sustainable way are likely to be those already reached, and the hardest people to reach are more likely to require continued subsidies; (b) there is a huge amount of other investment in microfinance, and we have very little sense of the role that grants play in enabling the expansion of services; (c) 3 billion clients is an extremely ambitious goal – around 20x the number of people actually being reached today, and around half the world’s current population.

A couple of ways to think about the comparison with health:

I would answer both of these questions mostly with a shrug. Certainly, under this extremely generous estimate of what microfinance could cost, it is “competitive” with the best health programs.

But this is assuming that all of that money going to microfinance is going to eventually succeed in reaching half the world, and also making the even bigger assumption that grants are the key factor. We think it’s very possible that much of microfinance’s reach has very little, or even literally nothing, to do with charitable support. (The less generous cost-effectiveness estimate of $12-$17 is fairly clearly not competitive with the best health programs: compare 12-17 person-years of financial services vs. 1 life saved, or 1 person-year of financial services vs. 3-5 person-years of extra school attendance due to improved health.)

Bottom line: we don’t see cost-effectiveness or “multiplying the impact of your dollar” as a strong argument for funding microfinance over health, on a general sector-level basis. This is the case even under the most generous model of the microfinance figures we’ve come up with.

Global Giving’s spot check and why it should worry you

(updated on: ) | by Holden

Aid Watch:

    “Local people may be the experts, but for outsiders deciding where their donations can do the most good, getting access to local knowledge and acting on it appropriately requires real-time feedback loops that most aid projects lack.
    Over a little more than a year, GlobalGiving combined staff visits, formal evaluation, third-party observer reports called visitor postcards, and internet feedback from local community members to create a nuanced, evolving picture of a community-based youth organization in Western Kenya that had received $8,019 from 193 individual donors through the GlobalGiving website.
    Initially, youth in Kisumu were happy with the organization. Among other things, the founder used the money to fund travel and equipment for the local youth soccer team. But the first tip-off that something was going wrong came when a former soccer player complained through GlobalGiving’s online feedback form that “currently the co-ordinator is evil minded and corrupt.” The view that the founder had begun stealing donations and was stifling dissent among his members was expanded upon by other community members, visitors to the project, and a professional evaluator.
    In the end, a splinter group broke off and started a new sports organization, and the community shifted their support to the new group. Reflecting the local consensus, GlobalGiving removed the discredited organization from its website.” (Emphasis mine)

Aid Watch stresses the “new way to evaluate a project” angle on this story, and we think it’s a good angle. But another angle is that most aid projects don’t receive this level of scrutiny, and the project that was put under this particular microscope – more as a way of testing the microscope than because there were concerns about the project – turned out discredited.

This is a story that I feel should affect your default assumption about whether an aid project is working.

The comments on Aid Watch’s post are also worth reading. One problem with the “funding a project at a time from many different organizations” approach is that it isn’t clear what one does with evaluation and feedback, when it’s available. Knowing how a project went is certainly better than not knowing, but the ultimate goal is to translate knowledge of what’s working into improved performance.

That’s an argument for focusing on the organization rather than the project level. Organizations can be given incentives to learn from their mistakes and improve their projects. Funding tiny organizations for one-off projects, it isn’t clear how to impose any kind of accountability.

LAPO (Kiva partner) and financial vs. social success

(updated on: ) | by Elie

We recently looked at Kiva’s largest partner MFI, LAPO (Lift Above Poverty Organization), as part of our evaluation process for an economic empowerment grant in sub-Saharan Africa.

In brief, we found two surprising pieces of information:

  • LAPO is very profitable.
  • There’s good reason to be concerned about LAPO’s social impact.

As Natalie recently described on our research list, we’ve contacted a handful of individual microfinance institutions in Sub-Saharan Africa to assess whether one might be able to answer the key questions we ask to evaluate a microfinance organization.

One of the steps we took was to look at Kiva’s largest MFI partners. Because Kiva partners are both (a) relatively well-known (due to its presence on Kiva) and (b) underwent Kiva’s due diligence process, we guessed that they might be a reasonable place to begin our search.

When we looked closely at LAPO, we found the following, all of which concerned us (Note: we haven’t yet contacted LAPO as our aim, at this point, was to identify the most promising organizations, not confidently dismiss any particular organization. Because our brief review of LAPO opened several relatively large questions, we chose to move on, as we often do).

  • In the last 3 years (2006-2008), LAPO had significant profit margins (23-28%).
  • In its Mix Market Social Performance Report (xls), LAPO reported a 49% dropout rate. As Holden wrote in our post on evaluating a microfinance charity, dropping out of a program may indicate participants “voting with their feet” and choosing to leave a program that they don’t find beneficial. It is also possible that “drop outs” instead consist of those who “graduate” from the program, i.e., improve their incomes/credit to the point where they can access credit from elsewhere (or no longer want/need credit). However, my instinct is that it’s unlikely that close to 50% of participants are quickly moving up to access more formal sources of credit.
  • LAPO’s Client Exit Study report (doc) reports that individuals need manager approvals to withdraw savings, and that managers investigate the reason for withdrawal before approving (Pg 3). This seems to undermine many of the benefits of saving, which presumably aims to help people deal with risk and unexpected situations.

Does LAPO sound like an institution that needs (or should receive) Kiva’s interest-free funding?

Its appears highly profitable, but its social impact is much less clear given the high drop-out rate, significant hurdles for depositors to withdraw savings. These facts paint a slightly worrying picture of LAPO as an organization that may be earning significant profits through relatively restrictive regulations for clients while getting interest-free funding through Kiva. Perhaps there is a special arrangement here as with Xac Bank, but it certainly raises a concern about charity-minded capital funding profits.