More on the ranking of our top charities

We previously wrote that we think cash transfers are likely to be significantly less cost-effective (in terms of “good accomplished per dollar donated”) than deworming; yet we rank GiveDirectly higher than Schistosomiasis Control Initiative. We gave some basic indications of our reasoning in the strengths/weaknesses table of our announcement post. Since then, we’ve had further conversations and sought to better express and formalize our thinking, and we’ve realized that there is a potential source of major confusion here:

  • This year, we selected our top charities based on the criteria we’ve used for years, but we ranked them based on where we would personally give in order to maximize our impact.
  • In past years, the questions of “Where would we give in order to maximize our impact?” and “Which charities meet our criteria?” have been essentially identical, but this year, they have started to diverge. Among other things, the goal of giving to learn has come to carry more and more weight for us.
  • Staff are divided on whether SCI or GiveDirectly better exemplifies our formal “proven, cost-effective, scalable” criteria, but we are in agreement that GiveDirectly is stronger on other dimensions including the opportunity to “give to learn,” and as such, we are unanimous in preferring to support GiveDirectly.
  • We also feel that “giving to learn” is the primary impact-based justification we’re aware of for supporting more than one of our top charities. For donors who don’t put weight on this benefit, we feel the way to maximize impact is to support AMF exclusively.

Below, we elaborate on these points. The upshot of them is that

  • Donors who are place high value on helping GiveWell by giving should either give to all three charities (with our recommended allocation as a reference point) or should make a gift to GiveWell for regranting at our discretion.
  • Donors who are seeking simply to do the most direct good – excluding benefits to GiveWell – should exclusively support Against Malaria Foundation, the charity that performs best by our criteria and the charity that we would support if we could only support one. Of course, donors who disagree with our recommendation of AMF may wish to consider both GiveDirectly and SCI as alternatives.
  • While we don’t feel our communications have been ideal around these issues, we plan to keep our top charities page as is. For donors seeking to support one charity, our recommendation is to support AMF; for donors looking to support multiple charities, but looking to do so based on our recommendation rather than their own review of our work, our recommendation is to use our target allocation; for donors looking to use more of their personal judgment, we make information available for doing so. We are interested in others’ perspectives on whether our top-level communications are appropriate.

The easiest way to address the difference (and where it came from) between our current criteria for identifying top charities and how we answer the question of “Where do I give?” is to briefly recount the evolution of GiveWell’s approach over time. We do so, then outline how we selected and ranked our top charities (and how the two differ), then discuss the implications for donors.

The history of our criteria

GiveWell was founded in order to answer the question, “Where should I give?” (See our story.) To the extent that we’ve formalized and publicized criteria for charities, these have come from formalizing what we were looking for in a giving opportunity. The fact that we start with “Where should I give?” and derive and adapt criteria from there – rather than starting with a set of criteria and applying them formulaically – has long been a distinguishing feature that has led GiveWell to investigate different questions (and generally investigate charities in more depth) compared to other charity evaluators.

When we first started, we recognized that we were extremely new to the world of giving. Accordingly, we wanted to start with the “easiest” giving opportunities for non-experts to assess: opportunities that involved directly paying for delivery of an already-proven intervention with at least somewhat quantifiable positive impacts. We’ve long recognized that there may be better giving opportunities that take a different form (higher risk, higher potential reward), but we haven’t felt that we have the expertise and context needed to assess these opportunities. So we’ve sought the easiest charities to be confident in.

Over time, we’ve gained experience and seen interest from larger donors, leading us to want to broaden our criteria generally. At the same time, we’ve become very interested in the idea of “giving to learn” – gaining information from an organization that is much easier to obtain as a “supporter” (someone who has helped get funding to an organization in the past) than simply as an evaluator (someone who might help get funding to an organization in the future).

  • This idea first started to appeal to us in 2010, when we felt that the significant funds we had directed to VillageReach improved our access (both in terms of VillageReach’s interest in investing time in GiveWell and in terms of GiveWell’s comfort with asking for such investment). We conducted a multiple day field visit and published intensive updates on VillageReach’s work, leading to substantial revisions in our views.
  • Many of the major funders we’ve interacted with have stressed the value of “giving to learn,” outlining a similar dynamic to the one described above: by supporting an organization, one gains the ability to investigate it more deeply.

At this point, it seems to us that one of the best uses of funds might be to support organizations from which – for whatever reason – we believe we can learn a lot, even if we’re not highly confident in such organizations and don’t see the dollars given to them as directly accomplishing much good. We still see enormous room for improvement in our knowledge base, and we anticipate substantial future growth in money moved; thus, we believe that most of the direct impact of the gifts we recommend over GiveWell’s lifetime is likely to be concentrated in the future, and knowledge gained now could have big returns if it improves our future recommendations.

However, as we’ve laid out some of the shifts we’ve been going through this year, many in our audience have said that they want us to continue to focus on charities that meet our traditional criteria – the criteria that have become fairly strongly identified with us by this point.

Therefore, for giving season 2012, we sought to ensure that we were doing our best to highlight all the charities we could find that meet the criteria many have come to associate with us. Our top charities are the current set of charities for which (a) we have conducted extensive due diligence, thoroughly pursued a large number of critical questions for, and remain highly confident in; (b) we believe that more dollars will likely lead to more delivery of programs that are highly cost-effective and backed by strong evidence. (Note that, as has been true every year, there are charities we consider promising and worth investigating, that we simply have not yet done enough due diligence on to place in this category.)

However, when it came time to rank these charities, we reverted to the question of “Where would we like to see funds go in order to accomplish as much good – all things considered – as possible?”

GiveDirectly vs. SCI

Against Malaria Foundation performs very strongly on all of our criteria – more strongly than any other charity we’ve found – and it is the single charity we would most like to see funds go to in order to accomplish good more broadly (including both direct impacts and learning opportunities). This is important because, as we wrote a few weeks ago, we see a fairly strong case for giving exclusively to one organization in order to maximize impact; to the extent that one gives to multiple organizations, we feel that this should be justified by room for more funding considerations or by the goal of “giving to learn.”

Nonetheless, many donors have asked us about the comparison between GiveDirectly and SCI, which is less straightforward because they have quite different strengths and weaknesses.

SCI is, we believe, working on an intervention with greater direct cost-effectiveness and roughly comparable evidence of effectiveness to GiveDirectly. However, GiveDirectly presents a much clearer picture when it comes to room for more funding. Our understanding of SCI is that marginal dollars are used to (a) fill gaps in programs funded by larger donors (such as the UK government’s Department for International Development) and (b) attempt to catalyze the creation of new programs; while SCI has a track record of implementing large programs fully funded by major donors like the US government or the Gates Foundation, we see little to go on in assessing its ability to carry out these activities effectively.

Looking more holistically at the question of where we’d give, we see three more advantages to GiveDirectly. One is our higher subjective confidence in it as an organization, which has implications for how much good we expect it to accomplish if unexpected situations arise, if there is something fundamentally off about our understanding of its activities, etc. The second issue is learning: we believe that with GiveDirectly, we have a clear sense of what we expect additional dollars to lead to and a strong expectation that we’ll be able to meaningfully compare our expectations with what actually happens in the future – that we’ll be able to assess the outcome of our recommendation via charity updates. We don’t believe the same to be true of SCI, based on the updates we’ve done over the past year (which have included some difficulties in communicating). Finally, we see more “upside” for GiveDirectly because we see it as experimenting with an intervention that is (wrongly, in our view) unusual in the aid world.

Staff members do not all agree on how important each of these individual factors are, but when considering all of them together, we are broadly in agreement that dollars given to GiveDirectly will accomplish more good.

Implications

Donors who are place high value on helping GiveWell by giving should either give to all three charities (with our recommended allocation as a reference point) or should make a gift to GiveWell for regranting at our discretion. Our recommended allocation is provided so that donors looking to support multiple charities, and looking to us for guidance on amounts, will give in the proportions that we feel will accomplish the most total good, including contributing to GiveWell’s ability to learn. (We’re planning to write more in the future about how we might better convey our recommended allocations. We suspect that, especially in the case of the #2 and #3 charities, our recommendations might be better expressed in terms of absolute dollars than in proportions of total money moved.)

Donors who are seeking simply to do the most direct good – excluding benefits to GiveWell – should focus their support on the Against Malaria Foundation, the charity that performs best by our criteria and the charity that we would support if we could only support one. We see no benefit to spreading donations out over multiple organizations, except for the potential benefits of “giving to learn” or in response to room for more funding issues (which we do not foresee for our recommended charities at expected funding levels). Donors who disagree with our recommendation of AMF may wish to consider both GiveDirectly and SCI as alternatives.

Comments

More on the ranking of our top charities — 6 Comments

  1. I was very pleased to see that givedirectly was included near the top this year.
    Even though I completely accept that the cost/benefit ratio of the other two may be smaller for the other two, I think that when choosing different courses, one should focus on the net benefits, not on the ratio. (It is not so much “I” that think that, it is on textbooks of cost-benefit analysis).
    Using that criteria, cash transfers could absorb all of the charity money in the world and still provide positive net benefits. That means they have world changing potential.
    I believe that as a relatively new charity, the more money givedirectly attracts, the easier it will be for it to grow. Which is why I would rank them even higher than AMF, right on the top. The more money we can attract for cash transfers, the faster a possible – and much needed – revolution in charity may happen. Who knows? it could it even affect governmental donations.

  2. I’ve been struggling to understand the recommendation of GiveDirectly, and this post partially clarifies your rationale. The criteria of giving to learn makes conceptual sense to me, but I don’t understand why GiveDirectly is the best opportunity for giving to learn.

    I would expect giving to learn to be most useful for causes and charities that have a reasonable potential for being at least as good as GiveWell’s other top charities. In that way, learning more about them would help GiveWell gain the knowledge to either promote or demote them. So it is not clear to me why GiveWell places such a high priority on learning about cash transfers, which GiveWell stated “are likely to be significantly less cost-effective (in terms of “good accomplished per dollar donated”) than deworming.” (Quote is from this blog post.)

    Can you provide more details about why cash transfers is the cause GiveWell places such a high priority on learning about? How will learning about cash transfers translate to greater impact down the road?

  3. Hi Eric, a few responses:

    • Note that we haven’t called GiveDirectly one of the top three learning opportunities we’ve seen. We consider it one of the top three charities we’ve seen by our standard criteria, and when ranking these three charities, we factored in its potential as a learning opportunity. That said, we do think GiveDirectly presents an outstanding learning opportunity.
    • We aren’t just seeking to learn about particular interventions for purposes of scaling up those interventions; we’re seeking to learn more broadly and generally about aid. We feel that following past recommended charities has resulted in learning generalizable things (not just learning about the interventions carried out by those charities).
    • In GiveDirectly’s case, we believe that the debate over cash transfers has implications for how aid projects ought to be designed in general. The question of how well low-income people in the developing world make decisions and investments for themselves seems fundamental and potentially far-reaching in implications. Experimenting with different approaches to cash transfers could shed light on this question.
    • Another point worth noting about GiveDirectly is that – conceptually, on a global basis – we’d guess that there is far more room for more funding for cash transfers than for bednets or deworming. Both of the latter are fairly far from fully funded now, but if at some point in the future the amount of aid going to evidence-backed interventions increases dramatically, we think it is plausible that they could become so; the prospect of reaching all of the world’s poor with a GiveDirectly-style program appears much further off. (We haven’t formalized/quantified this view and it isn’t core to the points made in the above post, but I thought it was worth noting.)
  4. I find your account of the effectiveness of a donation to AMF convincing. One question I am left with is – given that there is a strong evidence base for saving lives cost-effectively through bednet distributions, why is there still room for more funding? Why haven’t the big foundations and aid agencies that pay attention to evidence already filled this gap as far as is possible, at least until hitting strongly decreasing marginal returns due to unsupportive political environments, low levels of infection etc?

    Do you have an account of why this might be? And have any of the foundations (including foundations that give some support to bednet distribution) given their account of why they have left gaps in bednet coverage in favour of other projects?

    I understand it’s not your responsibility to try to explain the reasoning of others. However, it would be helpful to hear some suggestions for why the funding gap exists – eg you reason differently from the foundations, they spread donations inefficiently for institutional reasons, they have access to more effective giving opportunities that individual donors don’t have, etc.

  5. Ben – thanks for the question. I think the answer is both:

    • Aid agencies do spend substantially on bednets, but there remains quite a bit of need because the global need for bednets is exceptionally large (in the billions of dollars), and I suspect that decision-makers are typically reticent to make bets of that size on a single program.
    • Our impression is that major foundations tend to seek “leverage” rather than simply paying for service delivery themselves (though they of course have given significant funding to bednet distribution programs). We think this is a logical choice given their size and capacity, and our work around loosening our criteria to explore more complex charitable options (e.g. GiveWell Labs) is aimed at trying to further explore those opportunities ourselves.