The GiveWell Blog

The worst way to pick a charity

(updated on: ) | by Holden

Today, the most common way that donors evaluate charities – when they evaluate them at all – is by asking questions about financials, such as “How much of my donation goes to programs vs. salaries?” This approach makes no sense.

We’ve argued this point before at length. Picking charities based on the “overhead ratio” is like picking your doctor by the percentage of revenue spent on medicine (more absurdity-highlighting analogies here). The actual reported “overhead ratio” is vaguely defined and generally up to the charities reporting it; the concept of “minimal money on overhead” discourages a lot of good and necessary spending, and is ultimately irrelevant to the question of whether a charity is changing lives.

What today’s press release (PDF) makes clear is just how much agreement there is on these points. Today’s two most prominent sources of financial metrics – GuideStar and Charity Navigator – agree that there needs to be more to the picture, and are working toward more meaningful metrics.

It’s scary to think about how commonly this metric is touted, when it’s recognized as a red herring. It seems to us that people have responded to the extreme scarcity of information on charities by using whatever information is available, whether or not it’s meaningful. Fortunately, alternative ways of evaluating charities are emerging.

GiveWell, Great Nonprofits and Philanthropedia are all new, small organizations. None of us have perfected a methodology yet – no one has – and all of us would benefit from more input. If more donors took a look at these resources, we would finally see the conversation around “Where should I give?” move beyond the distracting and unhelpful question of “Who has the lowest ‘overhead?’”

That would be a huge step toward a world in which donors reward charities for effectiveness in improving lives.

When donations and profits meet, beware

(updated on: ) | by Holden

David Roodman raises the concern that Kiva capital could be effectively “padding profits” at a profitable microfinance institution. He concludes,

If social investors provide capital at prices below commercial rates to enterprises with “double bottom lines” (profit and social benefit), how do the investors assure that their cheap capital isn’t being used to boost just one of those bottom lines?

We feel that this is a major concern, and one that also applies to larger-scale social enterprise investment (see last week’s discussion of Acumen Fund, particularly the part about VisionSpring).

With a for-profit, everyone is looking to get their investment back. With a nonprofit, everyone is (ideally) looking to achieve social impact. But an organization that has both charitable and for-profit investors can get caught in the middle: taking donations, but measuring its success in terms of profits instead of lives changed.

The situation is particularly dangerous when profits are viewed as a “proxy” for social impact, and thus become the only measure looked at. Imagine an enterprise that could sell food below market prices, thanks to support from donations. It could be distorting the local economy (by outcompeting local farmers), failing to reach those truly in need, and ultimately failing to accomplish anything besides selling food for less than it costs to people who turn it around for a profit. And yet, all of this could be consistent with a good bottom line, which would make both the investors and the donors happy.

One way to avoid this problem is to refuse to use profits as a proxy for social impact – to insist on rigorous assessment of whether an enterprise is changing lives, rather than settling for the logic of “if it’s selling it must be helping.” But such rigorous assessment costs money, and exacerbates the already great challenge of turning a profit. Jim Fruchterman’s recent comment illustrates the low likelihood that you’ll see much of this approach working in practice.

Another possible approach is to get very specific about how donations are and aren’t being used. Any sort of “hybrid” organization ought to be able to show a history of using donations to absorb risk, but ultimately creating ventures whose profitability and sustainability does not depend in any way on continued subsidies. Our basic feeling is that demonstrating such a thing would be harder than it sounds, and that we have not yet seen an organization that seems capable of doing so.

Bottom line: it’s difficult to hold an organization accountable unless all its investors are on the same page about what it’s accountable for. Blended value makes perfect sense in theory, but in practice, it seems like a huge challenge that nobody is clearly up to. In the meantime, it may make more sense for businesses to be businesses and charities to be charities.

Smile Train

(updated on: ) | by Holden

These charts from Smile Train imply an appealing story:

(a) Smile Train performs surgeries for $250 apiece.
(b) Smile Train’s main use of donations is to fund $250 surgeries.
(c) A donation to Smile Train funds more surgeries than would the same donation to another cleft palate organization.
(d) If Smile Train had much more money, it would fund many more $250 surgeries.

But after some basic investigation of their website and financials (and conversation with the organization), we believe that:

(a) Smile Train’s total expense per surgery is well over $250.
(b) Directly funding surgeries (the program focused on in the charts above) accounts for under half of Smile Train’s expenses.
(c) Smile Train makes direct grants to other cleft palate organizations, some of which use the same model as the “mission groups” that its charts imply are overly costly.
(d) If Smile Train had more money, it could mean more grants to “mission groups,” general education about cleft palate conditions, and a host of other activities, and would probably not mean more directly funded surgeries.

We feel that, as a result, many of Smile Train’s donors – including Prof. Steven Levitt of Freakonomics and Superfreakonomics – have a highly inaccurate picture of how their donations are used. Details follow.

(a) Smile Train’s total expense per surgery is well over $250.

The same Smile Train page linked above claims 90,000-95,000 surgeries provided in 2008. If this number is entirely correct (not exaggerated or estimated) and we divide it into the roughly $100 million of “money raised” shown above, we obtain a cost-per surgery of over $1000.

In fact, the situation is a bit more complex, because money raised doesn’t equal money spent. To get the latter, we can look at Smile Train’s most recent IRS Form 990, which puts total expenses around $85 million for an implied cost-per-surgery closer to $900.

It’s possible that some surgeries are performed for $250 apiece or less. But are these the surgeries you’re paying for when you donate? That’s where the next few points become relevant.

(b) Directly funding surgeries (the program focused on in the charts above) accounts for under half of Smile Train’s expenses.

Here’s the big picture on Smile Train’s expenses, as far as we can understand them from the most recent IRS Form 990:

“Treatment partnerships” appears to be the core program of funding developing-world doctors to perform surgeries.

The “No information available on website” item refers to $23.6 million spent on “provid[ing] materials on cleft lip and palate for free to anyone interested in this birth defect” (according to the Form 990). We have no other information about this activity.

“Education grants” appear to refer to training developing-world surgeons. “Treatment grants” appear to correspond to grants to other organizations.

(c) Smile Train makes direct grants to other cleft palate organizations, some using the same model as the “mission groups” that its charts imply are overly costly.

Prof. Steven Levitt of Freakonomics and Superfreakonomics writes,

The typical model for cleft repair in developing countries has been to convince U.S. doctors and nurses to volunteer a week’s time, fly to a country, and do 80 surgeries … Smile Train instead partners with and trains local doctors to do the surgeries, which turns out to be far more efficient.

In fact, Smile Train makes grants to a variety of other organizations, including some that focus on the “mission” model (flying U.S. doctors and nurses overseas). The most recent list of grants starts on page 18 of the Form 990, and includes:

  • $1.4 million to GiveWell standout Interplast, which funds both missions and work with local doctors.
  • $70,000 to Surgical Volunteers International and $25,000 to the Smiles International Foundation, both of which appear to be missions only.
  • A wide variety of other grants, including for organizations focused on research, organizations that work primarily in the U.S., and more.

(d) If Smile Train had more money, it could mean more grants to “mission groups,” general education about cleft palate conditions, and a host of other activities, and would probably not mean more directly funded surgeries.

I first encountered some of these oddities back in 2006, and wondered why Smile Train wasn’t simply putting all available funds toward its most cost-effective surgeries (presumably, the ones that cost $250 each). The answer I got at that time was that there was a “doctor shortage” – Smile Train was paying $250 per surgery as much as it could, but had more money than it could use that way, so it engaged in other activities and made grants to other organizations.

Given this situation, I think Smile Train’s approach makes sense. How many charities are raising more cash than they can productively use, and sitting on it instead of giving it away to other organizations?

In terms of spending money it has appropriately, Smile Train may be doing well. But how it brings in that money is another question. When Steve Levitt doesn’t seem aware of how funds are being used, it seems to me that donors are getting the wrong impression.

Prof. Levitt notes that “One thing they don’t do, but maybe they should do, is literally link each $250 donation to a particular child and send before and after pictures.” Smile Train can’t take this suggestion, because it’s not providing a surgery for every $250 raised.

Poor in the U.S. = rich

(updated on: ) | by Holden

A single-parent family of three in New York, making $8000 per year, makes under half the income level of the Federal poverty line and qualifies for food stamps, TANF (direct cash benefits) and Medicaid. (Details at our guide to U.S. public assistance)

And yet, at $2,667 per person per year, this family is wealthier than 70% of the people in the world. (See the Global Rich List calculator as well as the Giving What We Can version, which may be using more up-to-date data.)

In the poorest parts of the world, fewer than half have access to a latrine or toilet; only 17% own a television; and 19-45% lack access to a reliable source of clean water. In the U.S., practically everyone has all three. (Details.) As we wrote yesterday, the two areas have completely different concepts of “hunger.” And finally, while anyone in the U.S. can ultimately be served in an emergency room, people across the world die or suffer from health conditions for which proven solutions exist.

In the U.S., helping the less fortunate usually means tackling a thorny “equality of opportunity” problem such as improving education or helping struggling adults to find and retain jobs. These problems have a long history of failure and few proven approaches to them. By contrast, helping people overseas can mean delivering something as proven and life-changing as a bednet or tuberculosis treatment.

Hopefully, these observations give some context on why a charitable dollar goes so much further overseas. If you are looking to use your wealth to help those less fortunate, we believe it’s hard to argue the case for U.S. as opposed to international charity, unless you believe that American lives are orders of magnitude more valuable.

Hunger here vs. hunger there

(updated on: ) | by Holden

There has been a fair amount of buzz lately (examples here, here, here, here) about “food insecurity” in the U.S. According to the Reuters headline, one in seven Americans is short of food. In looking into the data, what has surprised us is how different the meaning of “hunger” is when we’re talking about the U.S. vs. the developing world.

Developing-world hunger: 30% of children underweight

Developing-world hunger is usually discussed in terms of incredibly severe indicators of hunger. For example, a 2008 Lancet study estimates that

  • 32% of under-5 children in developing countries are “stunted” (height-for-age severely below normal, such that only 2.3% of children should be “stunted” in a normal distribution)
  • 20% are “underweight” (weight-for-age severely below normal, such that only 2.3% of children should be “underweight” in a normal distribution)

  • 3.5% are “wasted” (weight-for-age even more severely below normal, such that only 0.13% of children should be “wasted” in a normal distribution).

These are indicators of severe, long-term consequences of constant undernutrition for young children.

Food insecurity in the U.S. means anything other than complete and constant food security

The recent USDA release that has formed the basis of the recent discussion of U.S. hunger states:

Eighty-five percent of American households were food secure throughout the entire year in 2008, meaning that they had access at all times to enough food for an active, healthy life for all household members. The remaining households (14.6 percent) were food insecure at least some time during the year, including 5.7 percent with very low food security—meaning that the food intake of one or more household members was reduced and their eating patterns were disrupted at times during the year because the household lacked money and other resources for food. (Emphasis mine)

The report’s summary specifies that food insecurity is usually temporary as opposed to chronic (pg 9) and that children are usually protected from food insecurity even in food-insecure households (pg 6-7).

The “food insecurity” categories are derived from people’s answers to questions like “We worried about whether our food would run out before we got money to buy more” and “We couldn’t afford to eat balanced meals” (full list on pg 3). The details of the answers are found on page 45:


These data imply that anything approaching the sort of hunger measured in the developing world is practically nonexistent in the U.S.

Note in particular the difference regarding children. In the developing world, as shown above, severe child hunger is rampant. In the U.S., even in “food insecure” families, it’s extraordinarily rare for children to go hungry even temporarily. And indeed, World Bank data estimates that 1.3% of U.S. children under 5 are “underweight” – less than the 2.3% that would be expected in a fully normal distribution.

Also note that the USDA report estimates costs for different levels of “food plans” (pg 55), and that its “Thrifty” plan – the cheapest – ranges from $21-$40 per week ($3-$6 per day) depending on age. In the developing world, meanwhile, over 2.5 billion people are estimated to live on less than US$2.50 a day total.

Bottom line

We have no intention of trivializing the situation of those in poverty in the U.S. But for a donor making choices, it can be stunning to see what a different meaning “hunger” takes on when applied at home vs. abroad. Do you value the lives of Americans so much more that you’d rather help people with the second kind of hunger than people with the first?

Acumen Fund and social enterprise investment

(updated on: ) | by Holden

Seth Godin makes an appealing case for “social enterprise investment” along the lines of The Acumen Fund:

When two people trade, both win. No one buys a bar a soap unless the money they’re spending for the soap is worth less to them than the soap itself.

When someone in poverty buys a device that improves productivity, the device pays for itself (if it didn’t, they wouldn’t buy it.) So a drip irrigation system, for example, may pay off by creating two or three harvests a year instead of one.

…

How does Acumen Fund create these markets? The answer is patient capital. The companies that are selling solar lamps to replace kerosene or water purification systems in tiny villages, or housing projects for peasants in Pakistan or even ambulance services in Mumbai fully intend to make a profit, but the venture capitalists on Sand Hill Road aren’t in a hurry to invest in them. The investments are a little too risky, take a little too long or a little too unproven to take a chance on.

So Acumen finds these entrepreneurs on site in the developing world, funds them, teaches them and pushes them to build really big organizations. A to Z has literally thousands of people in their modern factory creating malaria bed nets in Tanzania. And so it grows.

There are reasonable arguments that this sort of larger-scale financing is far more promising than microfinance, which typically focuses on tiny microenterprises that are unlikely to achieve large scale and create jobs. If we could find a strong charity in this area, we’d be excited to recommend it. But as with other areas, we try to go beyond the story and ask critical questions, and so far we have found no organization – including the Acumen Fund – that appears to answer them.

Our main question in this area pertains to an investor’s track record. We’d expect large-scale investing to have a success rate well under 50%, but we’d want to see at least an example or two of real success in the past, consisting of any of the following:

  • An investment that has been shown to improve the lives of customers (for example, tracking bednet use in areas served by A to Z).
  • An investment that has been shown to create jobs, i.e., employ people with previously low incomes.
  • A financial return. (Note that it isn’t enough to point to a “portfolio” member and observe that it’s a successful, self-sustaining business. Anyone can make a loan or grant to an already-successful business; this doesn’t mean they’re having an impact.)

Given Acumen Fund’s reputation for (a) general excellence and (b) focus on evaluation (see the Acumen Fund discussing these issues here and here), we would have expected them to produce such examples. But after evaluating them for our economic empowerment grant, we have concluded that:

  • They do not directly track social impact of the organizations they invest in.
  • They do not have information on the effects of their investments on jobs.
  • They have declined to share information on the performance of their loans, and have not yet exited any equity investments.

A couple of other concerns:

  • At least one of their investments, VisionSpring, has a major charitable solicitation component (for example, they have applied for funding from us). Acumen’s loan could end up getting repaid by donations, which wouldn’t go well with the idea of “patient capital to create self-sustaining institutions.”
  • Their documents are largely at too general and vague a level to give a concrete sense of how their investments are going.

Given this situation, it’s hard to see the organization as more than an experiment at this point. No matter what its team’s credentials, there is the enormous question of how developed-world business experience translates to the goal of making good and socially impactful investments in the developing world.

As with other unproven charities, it seems to us that Acumen should be primarily funded by those who are very close to the relevant people and issues, until it is able to demonstrate not just a good story but some real results. Other similar organizations seem to be similar cases.