The GiveWell Blog

Can a donor give “new life for the pariahs” through the World Fistula Fund?

(updated on: ) | by Holden

Nicholas Kristof’s excellent column describes the problem of obstetric fistula and the World Fistula Foundation.

We strongly share Mr. Kristof’s view that obstetric fistula is a pressing problem, and have written before that it’s one of the problems we’d most like to help donors address.

However, we still have not identified a fistula charity that we can be highly confident in. Here’s what we do know about the Worldwide Fistula Fund (WFF), the charity discussed in the column:

We’re confident that WFF is a legitimate operation working hard to address the problem of obstetric fistula. We’ve had significant back-and-forth with Dr. Lewis Wall, the founder and President of the Worldwide Fistula Fund.

Obstetric fistula is correctable, but the surgery is complex and may have a high failure rate. More at our discussion of the condition of obstetric fisula.

WFF’s current initiative is building specialized fistula facilities. The proposed plan (PDF) states that “cases such as fistula repair are chronically “bumped” from the surgical schedule by a continuous string of emergencies,” and thus that “without dedicated
facilities for fistula surgery, these operations will never get done.”

Is it the case that regular hospitals are unwilling or unable to treat fistulas reliably? Or might this approach represent an unnecessary diversion of resources from other pressing health problems (for example, pneumonia)? Especially if WFF’s focus is on lobbying the government to reallocate funding rather than using donations to increase total aid flows? At this point we don’t have the information to say.

We don’t know:

  • The skill level of the surgeons that will be working at WFF’s new facilities, or the success/complication rate of their surgeries. Hopefully this information will become available in the future.
  • The full story of what happens to the women who get corrective surgery. We have some information about the medical benefits of surgery, but very little about the social implications. To what extent can a “pariah” be re-accepted into her community?

Pneumonia Day is another General Health Day

| by Holden

On World Pneumonia Day, it’s natural for a donor to ask, “Where can I give to fight pneumonia?”

But the actual plan to fight pneumonia consists of the following: breastfeeding promotion, vaccination, and treatment in communities, health centers, and hospitals.

All of these are general health initiatives that touch on a wide variety of health problems. So it’s not surprising to see that almost none of the World Pneumonia Day Coalition members are organizations devoted to “pneumonia”; they are vaccine organizations, general charities, and even malaria and tuberculosis organizations.

If you are a “traditional” donor, we hope World Pneumonia Day succeeds in making you support these extremely worthwhile initiatives, by playing to your need for breaking news, grand announcements, and shocking facts that you didn’t know/remember last month.

If you are a results-oriented donor, you don’t need World Pneumonia Day. Just support great global health charities.

Why can’t you make the sale?

(updated on: ) | by Holden

I recently attended a seminar with the fascinating Seth Godin and heard an interesting anecdote about VisionSpring:

I could see that every single person who came to this meeting had enough money [$3] … to buy a pair of new reading glasses. And I could tell from how old they were that they were qualified and I knew what they did for a living so I knew that this would pay for itself in two weeks, three weeks, certainly in 20 years it’s going have a huge return on investment. So they get the demonstration, they take the eye test, they see that they need glasses, they take the sample glasses off, they walk over to the table where there are 8 kinds of glasses to choose from … all carefully wrapped in plastic … and 40% of the people bought a pair of glasses. 60% of the people left. And I’ve thought about this about a million times … If they knew how great the glasses would be, if they could overcome the momentum they had and the desire to keep the money … there’s no question they would have bought a pair of glasses. Maybe for $6 or $9 or $12, they could afford it, they needed it, they were in the right place at the right time, and yet the transfer didn’t occur.

To Mr. Godin, it seemed obvious that the customers should be buying glasses, and that they were being held back by “momentum.” He proposed a change:

Instead of giving the person the eye test, taking off the glasses and having them go over … and now make the decision do you care enough about yourself to buy those, instead, give the guy the eye test, and say those are your glasses, you owe me $3. Now the person has to make a new decision, which is better, giving up the $3 and keeping on what I have or going to the trouble of taking them off and reminding myself that I don’t deserve to see? And when you do that it turns out that the close rate goes up 30%.

To be sure, it sounds like an impressive improvement for a simple change. And yet a 30% improvement on a 40% close rate still leaves about half the people not spending the glasses – now under circumstances that arguably make it pretty difficult to turn them down.

Is it possible that Mr. Godin – and the charity he was observing – have simply overestimated the demand for what they’re doing? Underestimated the extent to which imperfect substitutes may be available? It appears that Village Phone did exactly this in at least one case.

It’s easy to be sure that your product is great and that it’s needed. Yet if you’re wrong, and you have donors subsidize it, you may essentially be giving out cash in a less efficient, less empowering way.

Mr. Godin analogized the situation to fundraising:

You’re sitting in class and the person next to you … is coughing … and you take out a container of Fisherman’s Friend menthol lozenges and you offer it to her and she puts it in her mouth and the coughing goes away and everyone is happy … and yet you’ve been on those calls to raise money from a donor, who apparently has money to donate … and you hand them the equivalent of a Fisherman’s Friend, “look at this program, we’ve been working really hard on it, it’s really important, we need your money” and they say “I need to get back to you” and you know what that means …

To me, there’s a major problem with the assumption that a charity’s program is at effective at solving complex social problems as a Fisherman’s Friend is at soothing throats.

Confidence in your product is great, but it can be misplaced. When charities let assumptions like “Everyone here needs glasses” and “Our program is reliable as a cough drop” go unexamined, they’re going to be left scratching their heads at results-oriented donors.

The Gates Foundation’s agriculture program: Experimenting or floundering?

(updated on: ) | by Holden

Here’s what we know about the Gates Foundation’s agriculture program:

  • Gates believes it’s suggestive that “apart from a few states and small, oil-rich countries, no country has managed a rapid rise from poverty without increasing agricultural productivity. In the poorest countries, agriculture employs a majority of the people.”
  • This isn’t a new argument or an undisputed one. See Peter Timmer on Green Revolution “optimists” vs. “pessimists”.
  • Gates’s approach is “comprehensive,” targets “no single, simple solution”, and includes farmer training/support, irrigation initiatives, market access initiatives, and funding of agricultural research with a focus on gender empowerment.
  • This isn’t a new approach or a historically successful one. The World Bank has focused on essentially the same set of interventions recently, with unclear results, and the previous “holistic” approach of “Integrated Rural Development” is widely considered to have failed. Details at our overview of agriculture aid.

In other words, the Gates Foundation approach – as described – appears to be neither a continuation of things that have worked before nor a fundamentally new approach to the problem. So what might be different this time around?

Lots of things. Better technology could make all the difference; so could a greater degree of commitment. And one way in which the Gates Foundation could really distinguish itself from past efforts would be by doing a superior job learning about what works and what doesn’t – past initiatives have suffered from poor evaluation and very little accessible information about how things have really worked out.

The Gates Foundation’s progress reports so far are extremely preliminary, looking at “inputs” such as “number of farmers organized into groups”. We find these measures wholly appropriate given how early it is in the initiative; and yet, we’ve seen too many programs that still haven’t moved beyond these measures even after claiming “success” and asking individuals to donate and help them scale up.

If the Gates Foundation moves to more rigorous and outcome-focused evaluation over time, we might learn more about what works and what doesn’t, and the “impatient optimism” could turn out to be justified. If not, the Gates Foundation will be making a very expensive gamble with very little information about its odds.

Gates Foundation on agriculture funding: Where are the facts?

(updated on: ) | by Holden

The Gates Foundation states that “funders have sharply cut their international aid to agricultural development over the past few decades.” It is implied that this is a major reason for the failure to see a “Green Revolution in Africa.”

We have been unable to locate support for this claim.

Using data from OECD – the most reliable source of official aid flows as far as we know – we graphed the proportion of (disbursed) aid to Africa that was classified in the “agriculture” sector since the 1970s. It’s possible that the numbers are artificially depressed for early years due to less standardized reporting, but we see no trend of the type the Gates Foundation describes.

In addition, we previously looked at funding on some of the main vehicles credited with the original “Green Revolution” and found no substantial drop since the 1970s.

It’s frustrating that the Gates Foundation doesn’t provide a source for its claim. In general, the material on its website is at a very broad level and does not make it possible for people curious about its underlying reasoning to drill deeper. That means that any criticism or examination of its work has to be either very superficial or done offline (i.e., in direct communication with the Foundation, an opportunity few people seem to get easily or often).

Added 10/30/09: here’s our source data

Village Phone: Another great story under the microscope

(updated on: ) | by Holden

Ever since I heard about the Grameen Foundation’s Village Phone program, I’ve been optimistic. The program involves helping people in remote villages run pay-for-use cellphone services: they get their cellphone, and a loan to buy it, via Grameen, then charge other villagers to use it. It’s an approach to fighting poverty that’s (a) relatively new; (b) using a product that hasn’t been available for a long time but seems clearly useful to anyone doing business in remote areas; (c) utilizing a “franchise model” where people in the villages take a stake in the product.

It was near the top of my “Probably helping people, even though we don’t yet have systematic evaluation yet” list. Now Chris Blattman points to a discouraging evaluation that found “absolutely no impact of the phones on trading activity or availability of goods in local markets” and very small (non-significant) impacts on profits and measures of well-being (school enrollment, consumption of meat, etc.)

This bottom-line result does not, by itself, mean the program “doesn’t work.” It could work very differently in different contexts (discussed below), and there are some possible issues with the paper (which is very recent, and is not a randomized controlled trial). But one thing I like about the study is that it doesn’t just discuss impact – it examines many aspects of the program, and exposes assumptions that may otherwise have gone unquestioned.

Assumption 1: the phones are in high demand and operators easily cover their costs. In fact, usage of the phones was around 4 hours a month, or 8 minutes a day (pg 19). As a result, profits from the phones were not enough to keep up with loan payments (pg 19). Grameen reportedly has responded by changing loan and franchise terms (pg 30). Tuvugane (pg 5), a less sophisticated phone product that was already common in the villages, may have been good enough for most purposes.

Assumption 2: farmers who use the phones benefit from better pricing power. Even though farmers with access to the phones became much more likely to arrange their own transport to market, there was no apparent effect on the prices they received for their goods, possibly due to established relationships with buyers (pg 16).

Assumption 3: if someone chooses to become a cellphone operator, they’re going to benefit from it. In fact, there was a very strange pattern in the businesses of people who became phone operators. Their hours worked rose significantly both for their new phone business and for their already-existing businesses, but their profits and wages paid did not rise (pgs 17-18). A possible explanation is that operators wanted to be available for cellphone users and so stayed at their workplaces longer, but that the extra hours didn’t translate into extra profits. In any case, it’s a pattern that doesn’t seem encouraging, and seems to deserve further investigation.

Bottom line: a product that was supposed to be helpful and in high demand arguably ended up as a bad investment for the franchise operators. This doesn’t mean it shouldn’t have been tried, or that it shouldn’t be tried in the future. But it points to the importance of testing assumptions empirically, rather than scaling up a program as widely as possible based on an appealing story.