The GiveWell Blog

Aid’s track record

While Elie’s been investigating the Carter Center, I’ve been scanning literature (mostly academic) on general questions about aid: what has worked in the past? What’s promising for the future? etc.

Rather than trying to come to an independent conclusion on each debate, I’ve been trying to establish which beliefs are supported by evidence that is largely undisputed among scholars (and where there is no agreement, get a sense of what each side believes and what its most frequently pointed to evidence is).

At this point I’ve looked in a lot of places (though I’m still far from done) and I’m going to start sharing where we stand (and what we’re still missing) on various questions. First I’ll discuss what I’ve read about the track record of aid to developing-world areas.

The vast majority of the large-scale “success stories” I’ve seen come from health initiatives (particularly “vertical” health initiatives, i.e., large-scale campaigns against particular diseases).

The main non-health “success stories” I’m aware of:

I don’t believe that any of the sources cited above are fully comprehensive (or necessarily fully reliable) as lists of large-scale success stories. We’re still looking for more.

Aid has also had major failures – but insufficient monitoring and evaluation means that few are thoroughly documented or discussed.

  • Dissatisfaction with the accomplishments of aid to date is fairly widespread. The latest report on the Millennium Development Goals shows mostly inadequate/less-than-hoped-for progress (and no progress in many cases). William Easterly argues that

    systematic testing would not just count the alleged ‘success stories’ of aid, but also the larger number that got the same amount of aid as the ‘success stories’ and failed: Guinea-Bissau, Somalia, The Gambia, Mali, Rwanda, Nicaragua, Burundi, Guyana, Zambia, the Central African Republic, Senegal, Suriname, Chad, Niger, Togo, Haiti, and so on. Further testing shows that these outcomes were not an artifact of selection bias or reverse causality.

  • Yet while project-level evaluations provide scattered analysis of projects gone wrong (one example of a failed World Bank project here), we have not found well-documented “failure stories” along the lines of the “success stories” above – examining major humanitarian (as opposed to political) initiatives that had little or negative impact.
  • Part of the reason may be the general lack of evaluation and documentation in international aid. The sentiments of this Center for Global Development paper, arguing that “very few programs benefit from studies that could determine whether or not they actually made a difference. This absence of evidence is an urgent problem,” are common; the Paris Declaration on Aid Effectiveness, an agreement between major aid agencies, includes an expressed commitment to more monitoring and evaluation.

I have the sense that failure stories in aid are common, but I continue to look for more concrete examples of what has gone wrong and how.

The relationship between aid and growth at the macro/country level has been extensively studied, but is not well established one way or the other.

Some believe in a moderate positive relationship, often with the caveat that aid works better where existing institutions are stronger (more below) or that aid has diminishing returns. Others believe that there is no relationship or that there is insufficient evidence. Two particularly accessible summaries: A Primer on Foreign Aid (from the optimistic side) and Macro Aid Effectiveness Research: a Guide for the Perplexed (from the skeptical side).

There are no established broad patterns in the sorts of environments where aid has worked, though many believe that aid works better where existing institutions are stronger.

A Primer on Foreign Aid states,

the view that aid works better (or in a stronger version, aid works only) in countries with good policies and institutions has become the conventional wisdom among donors, partly based on [empirical] research and partly due to development practitioners that believe this to be the case based on their own experience.

However, as this paper acknowledges, the empirical research has been questioned repeatedly (as in this paper, which claims that the statistical work underlying most such claims is excessively fragile). In addition, Millions Saved (the success stories compilation referred to above) gives examples that “Success is possible even in the world’s most underdeveloped and remote regions, in the face of grinding poverty and weak health systems” (quote available at this page).

One caveat to keep in mind about all of this analysis is that most academic literature focuses on official aid flows – aid from developed-world governments (or multilateral institutions such as the World Bank), which usually goes through developing-world governments. This sort of aid is different in many ways from private donations going through nonprofits.

Comments

  • Gernot Wagner on February 18, 2009 at 10:03 pm said:

    Holden,

    Interesting thoughts. It is, in fact, scary how little we know about the effectiveness on which aid policies work and have a reasonable chance of success. I applaud your efforts to shine light on this issue and perhaps even try to come up with criteria for making aid work more effectively.

    One issue that strikes me, though, is the focus of your efforts. You seem to be asking the question: if we have $100 for aid, where should this money be spent? That’s the cost-effectiveness question. A broader approach would extend this analysis to ask the efficiency question: how much aid should we be giving in the first place? That’s a much broader question with potentially larger consequences. For reasonable estimates of the statistical value of a human life, we would almost inevitably argue for more aid to, say, health causes in sub-Saharan Africa. Of course, effectiveness matters too, but it needs to be placed in context.

    A second set of issues surrounds aid versus private sector initiatives. Health provides a clear case for direct aid — up to a point. Climate change might be on the other end of the spectrum, where using private funds from the beginning is going to be inevitable. Direct government aid towards mitigation efforts hovers around $2 billion per year. UNFCCC estimates that we would need upwards of $100 billion per year by 2020 to guide the world economy onto a low-carbon pathway. Aid alone will not be able to fill that gap. That’s where carbon markets and the private sector comes in. $100 billion might be a multiple of Official Development Assistance, but it is a fraction of Foreign Direct Investment.

    Best,
    Gernot

  • Kim Dionne on February 19, 2009 at 1:38 pm said:

    did you catch this recent post on Easterly’s blog? The article isn’t yet posted to the journal’s web site, otherwise I’d comment on it. Judging from his summary on the blog post, it looks to have a health slant.

  • Alanna Shaikh on February 19, 2009 at 2:24 pm said:

    As an initial though, I want to agree with Gernot on the importance of differentiating between types of aid. Private charity can be every different from official development assistance, and different forms of ODA – projectizing vs budget supplements, for example, will have very different impacts.

  • Thanks for the comments.

    Gernot and Alanna: I agree that it’s important to differentiate between different types of aid. I mentioned the issue briefly in this post but will be discussing nonprofit programs vs. official aid in a future post.

    Gernot: we’ve chosen for now to focus on international aid. The goal is to understand both the relative strength of different options in this cause, and to have a sense of how beneficial aid is in this area as a whole. Future research will cover other areas, and get to the point where we can compare them.

    Kim: I found the paper referenced in that post to be a good overview. It is referred to twice in this post.

  • Brian Slesinsky on February 21, 2009 at 4:07 pm said:

    Given that you say that it’s not well-studied, I wonder whether it’s possible to give money to people who study the cost effectiveness of aid to the developing world? Of course, funding GiveWell does that at the meta-level, but I mean the people who actually travel places and study individual programs.

  • Brian – I’ve looked around a lot for these sorts of organizations and the two that stand out to me – by far – are the Jameel Poverty Action Lab (J-PAL) and Innovators for Poverty Action (IPA). Both are discussed in this post. I should note that one of our Board members, Lindy Miller, is a former employee of J-PAL.

    You can donate to J-PAL here, IPA here.

  • Peter Singer on March 6, 2009 at 2:09 am said:

    Holden,
    Have you sought data from the Millennium Villages Project? That’s surely a good test – or will be a good test – of whether aid can help rural Africa escape the poverty trap. Last year, when I was writing “The Life You Can Save” I checked what was available, and early results were reported to be promising, but there was no proper assessment yet. I’m not sure if there is more now, but presumably there will be over the next year or two.

  • Holden on March 6, 2009 at 2:06 pm said:

    The Millennium Villages Project is high on our list as a charity to investigate, and should be in our final report. From what I’ve seen, I don’t think there’s enough there yet to test the “sustainably end poverty” idea, but there may be enough to test whether MVP can improve lives significantly and relatively cheaply.

Comments are closed.