The GiveWell Blog

By default, assume aid projects aren’t reaching the poorest

(updated on: ) | by Holden

If you don’t have evidence one way or the other, should you assume an aid’s projects benefits are reaching the poorest?

We think it’s fair to assume the people with the most need are the people with the least power. We’d also guess that, in general, the people with the most power are best positioned to get anything valuable (training, materials, loans, or whatever else) a charity is subsidizing.

There are ways a charity can get around this dynamic, such as:

  • Working in an area where everyone is in need. We believe that such areas exist, but just because an area has low average incomes, high disease rates, etc. doesn’t mean it has no relatively privileged and powerful members.
  • Running programs that are only appealing to those in need. Some health programs work this way (it’s hard to be treated for tuberculosis unless you have tuberculosis). Microfinance may work this way when interest rates are competitive with (not highly subsidized relative to) alternative sources of credit.
  • Carefully targeting those in need. We should expect this to be very difficult. Charity is inherently about coming into a community from the outside; targeting those in need will generally mean trying to outmaneuver at least some locals. The more locals genuinely share the charity’s mission, the better, but how is the charity to know which it’s dealing with?

Unfortunately (as usual), there isn’t much information out there about how often charities actually succeed in targeting those in need. In conducting our grant application process, we’ve found that systematic assessment of this question is relatively rare. But there is certainly room for concern, as shown by a World Bank review that we’ve quoted before:

The frequent tendency for participatory projects to be dominated if not captured by local elites is highlighted by several case studies. Katz and Sara (1997), in a global review of water projects, find numerous cases of project benefits being appropriated by community leaders and little attempt to include households at any stage … even well trained staff are not always effective in overcoming entrenched norms of exclusion. In a study of community forestry projects in India and Nepal that worked reasonably well, Agarwal (2001) reports that women were systematically excluded from the participatory process because of their weak bargaining power. Rao and Ibanez (2003) find that in the participatory projects in their Jamaican case study, wealthier and better networked individuals dominated decision making. In a similar case-based evaluation of social funds in Jamaica, Malawi, Nicaragua, and Zambia, the World Bank (2002) Operations Evaluation Department concludes that the process was dominated by “prime movers.”

Abraham and Platteau (2004) present evidence on community participation processes in Sub-Saharan Africa based largely on anecdotal evidence from their work in community-based development and on secondary sources. They argue that rural African communities are often dominated by dictatorial leaders who can shape the participation process to benefit themselves because of the poor flow of information. (40-41)

Why are we always criticizing charities?

(updated on: ) | by Elie

Recently, we’ve criticized (in one way or another) many well-known, presumably well-intentioned charities (Smile Train, Acumen Fund, UNICEF, Kiva), which might lead some to ask: should GiveWell focus on the bad (which may discourage donors from giving) as opposed to the good (which would encourage them to give more)? Why so much negativity and not more optimism?

The fact is, we are very optimistic about what a donor can accomplish with their charity. Donors can have huge impact — save a life, improve equality of opportunity, or improve education. Our research process, and our main website, are (and always have been) built around identifying outstanding charities.

GiveWell hasn’t set a bar that no charity can meet. Six international charities have met and passed the bar. Where most charities fall short, they succeed.

The problem is: because the nonprofit sector is saturated with unsubstantiated claims of impact and cost-effectiveness, it’s easy to ignore me when I tell you (for example), “Give $1,000 to the Stop Tuberculosis Partnership, and you’ll likely save someone’s life (perhaps 2 or 3 lives).” It’s easy to respond, “You’re just a cheerleader” or “Why give there when Charity X makes an [illusory] promise of even better impact?”

We don’t report on Smile Train, Kiva, Acumen Fund, UNICEF, or any others for the sake of the criticism; we write about them to show you how much more you can accomplish with your gift if you’re willing to reconsider where you’re giving this year.

Unless you have strong reason to believe otherwise, I’d recommend you choose a great charity as opposed to one that’s merely better-than average. If you only have a fixed amount to give, why not support the very best?

Smile Train removes charts from website; still claims $250 per surgery

(updated on: ) | by Holden

Two days ago, I critiqued the message sent by charts on Smile Train’s “Financial Information” page, particularly the idea that a surgery was being provided for every $250 in donations.

My recollection is that the page had been up in this form since at least the summer of 2006 (when I first started investigating surgery charities), and Internet Archive appears to confirm that it’s been up at least since early 2007.

We have not heard/seen a response from Smile Train, but as of last night, that page appears to have been taken down entirely.

Smile Train is still claiming $250 as the cost per surgery, which (I argued) seems contradicted by the now-taken-down charts.

Details

Two pages appear to have changed: the “charts” page (the page I critiqued) is now gone, and the “financials” page now longer links to the “charts” page.

“Charts” page:

“Financials” page:

Kudos to Ken Berger

(updated on: ) | by Holden

Charity Navigator’s CEO writes:

We MUST get past the notion of doing the “good work” with no accountability. We MUST get past the idea that nonprofits are too complex or unique to be measured. I have seen it close up for years and it is not a pretty picture. The nonprofit sector must get its act together and make sure it is really helping provide meaningful change in communities and peoples lives. It is life or death for many of those we serve whether we are effective or not. So let’s work together to measure, manage and deliver what is really important to make our world a better place. (Emphasis in original)

I respectfully disagree with those who call this “just talk.” Rather, I’m with Sean, who says: “Ken has put his chips down big and made a huge bet that he’s going to pull this all off.”

For years, Charity Navigator has dominated news stories about how to give effectively, even while the limitations of its methodology have been widely recognized within the nonprofit sector. Now, Ken is not just talking about but committing to doing more. His statement is so public and so clear that there’s no credible way to turn back, for him or his organization. If Charity Navigator doesn’t want to have its own quotes used against it for the rest of its existence, it will have to see its commitment to real, meaningful evaluation through.

We don’t yet know all the details of how Charity Navigator is going to go about this, and if we have issues with the methodology it ends up adopting, you’ll hear about them. But Ken absolutely deserves a “standing ovation” now. He is the issuer of our press release who had the most to lose, and he’s chosen to fight for what he knows is right rather than stand pat.

The worst way to pick a charity

(updated on: ) | by Holden

Today, the most common way that donors evaluate charities – when they evaluate them at all – is by asking questions about financials, such as “How much of my donation goes to programs vs. salaries?” This approach makes no sense.

We’ve argued this point before at length. Picking charities based on the “overhead ratio” is like picking your doctor by the percentage of revenue spent on medicine (more absurdity-highlighting analogies here). The actual reported “overhead ratio” is vaguely defined and generally up to the charities reporting it; the concept of “minimal money on overhead” discourages a lot of good and necessary spending, and is ultimately irrelevant to the question of whether a charity is changing lives.

What today’s press release (PDF) makes clear is just how much agreement there is on these points. Today’s two most prominent sources of financial metrics – GuideStar and Charity Navigator – agree that there needs to be more to the picture, and are working toward more meaningful metrics.

It’s scary to think about how commonly this metric is touted, when it’s recognized as a red herring. It seems to us that people have responded to the extreme scarcity of information on charities by using whatever information is available, whether or not it’s meaningful. Fortunately, alternative ways of evaluating charities are emerging.

GiveWell, Great Nonprofits and Philanthropedia are all new, small organizations. None of us have perfected a methodology yet – no one has – and all of us would benefit from more input. If more donors took a look at these resources, we would finally see the conversation around “Where should I give?” move beyond the distracting and unhelpful question of “Who has the lowest ‘overhead?’”

That would be a huge step toward a world in which donors reward charities for effectiveness in improving lives.

When donations and profits meet, beware

(updated on: ) | by Holden

David Roodman raises the concern that Kiva capital could be effectively “padding profits” at a profitable microfinance institution. He concludes,

If social investors provide capital at prices below commercial rates to enterprises with “double bottom lines” (profit and social benefit), how do the investors assure that their cheap capital isn’t being used to boost just one of those bottom lines?

We feel that this is a major concern, and one that also applies to larger-scale social enterprise investment (see last week’s discussion of Acumen Fund, particularly the part about VisionSpring).

With a for-profit, everyone is looking to get their investment back. With a nonprofit, everyone is (ideally) looking to achieve social impact. But an organization that has both charitable and for-profit investors can get caught in the middle: taking donations, but measuring its success in terms of profits instead of lives changed.

The situation is particularly dangerous when profits are viewed as a “proxy” for social impact, and thus become the only measure looked at. Imagine an enterprise that could sell food below market prices, thanks to support from donations. It could be distorting the local economy (by outcompeting local farmers), failing to reach those truly in need, and ultimately failing to accomplish anything besides selling food for less than it costs to people who turn it around for a profit. And yet, all of this could be consistent with a good bottom line, which would make both the investors and the donors happy.

One way to avoid this problem is to refuse to use profits as a proxy for social impact – to insist on rigorous assessment of whether an enterprise is changing lives, rather than settling for the logic of “if it’s selling it must be helping.” But such rigorous assessment costs money, and exacerbates the already great challenge of turning a profit. Jim Fruchterman’s recent comment illustrates the low likelihood that you’ll see much of this approach working in practice.

Another possible approach is to get very specific about how donations are and aren’t being used. Any sort of “hybrid” organization ought to be able to show a history of using donations to absorb risk, but ultimately creating ventures whose profitability and sustainability does not depend in any way on continued subsidies. Our basic feeling is that demonstrating such a thing would be harder than it sounds, and that we have not yet seen an organization that seems capable of doing so.

Bottom line: it’s difficult to hold an organization accountable unless all its investors are on the same page about what it’s accountable for. Blended value makes perfect sense in theory, but in practice, it seems like a huge challenge that nobody is clearly up to. In the meantime, it may make more sense for businesses to be businesses and charities to be charities.