The GiveWell Blog

Donor coordination and the “giver’s dilemma”

(updated on: ) | by Holden

This year we’ve dealt with some particularly intense manifestations of what one might call the “giver’s dilemma.”

Imagine that two donors, Alice and Bob, are both considering supporting a charity whose room for more funding is $X, and each is willing to give the full $X to close that gap. If Alice finds out about Bob’s plans, her incentive is to give nothing to the charity, since she knows Bob will fill its funding gap. Conversely, if Bob finds out about Alice’s funding plans, his incentive is to give nothing to the charity and perhaps support another instead. This creates a problematic situation in which neither Alice nor Bob has the incentive to be honest with the other about his/her giving plans and preferences – and each has the incentive to try to wait out the other’s decision.

In this stylized example, our picture of the ideal resolution is for Alice and Bob to be honest with each other and agree to “split” the funding gap: each should give $X*(1/2). In a world where this behavior is expected, people can be honest with each other about their plans and ensure relatively “fair” resolutions rather than withholding information and trying to anticipate each other’s behavior.

In real life, the situation can be more complicated. This year, in finalizing our funding targets for top charities, we faced the following three questions. In each case, we are actively trying to avoid behavior that amounts to “waiting out” or “canceling out” donors who are using our research, and trying to preserve the incentive of each donor to give as planned and share information about his/her giving.

1. Should Good Ventures wait until the end of giving season to decide on the size of its grants? (Note: we resolved this question last year and stuck to the same resolution this year, but it remains a puzzling question and a good example for this topic.)

Once we arrive at a “target amount” we would like to see each charity raise, we could wait until the end of giving season before recommending grants to Good Ventures. Since Good Ventures is giving a great deal (a bit more than we expect all individual donors combined to give over the course of the year), this would probably ensure that each charity hits the exact target we set for it, regardless of what individual donors decide.

We think this would be a very problematic approach. It would render individuals’ decisions about which charities to support effectively meaningless: each dollar given to one of our top charities would be fungible with dollars given to other top charities, and anyone who gave to one of our top charities would effectively/implicitly be supporting “GiveWell’s overall allocation” rather than the charity of their choice. This, in turn, would give individuals the ongoing incentive to try to wait out Good Ventures, and would potentially reduce both giving and reporting of giving.

Instead, we and Good Ventures agreed that the Good Ventures grants should be finalized and announced before the start of giving season. Our recommendations have taken into account what we expect individuals to do, but our projections are approximate and non-binding, and any given individual decision has real impact on the total raised by the recipient charity.

It’s true that in an indirect, approximate sense, supporting a given charity now may change our projections of what will happen next year and therefore cause Good Ventures to give less to the target charity next year. However, this is a very indirect and rough link, since situations change dramatically from year to year; our ability to predict donor behavior is quite limited; and we generally aim for round numbers with Good Ventures grants.

2. Should we post updates on how giving is going, so that donors can adjust their behavior accordingly?

Posting too many updates would risk a similar problem to the one described above. We don’t want a situation in which each donor’s gift to charity X causes another donor to give less to the charity; this would create an incentive for donors to try to wait each other out.

We do intend to post a notice when a given charity appears sure to hit its “maximum” – the most we think it could absorb before hitting a point of seriously diminishing returns. This will help donors avoid supporting a charity to go well over what we think is a reasonable amount of funding, without allowing donors to “cancel each other out” at a more granular level. In other words, if you give to a charity, you can expect that your donation will raise the total amount the charity takes in, by the amount of your donation – unless the charity ends up easily hitting its maximum, in which case your donation will be partly offset by the behavior of other donors who agree with us about where the maximum ought to be. We think this is a reasonable practical position.

Note that we will also be doing an update in early 2015 on each charity’s room for more funding situation. 2015 donors may take this information into account and may therefore do some degree of “canceling out” 2014 donors. Our basic position is that “canceling out” can be a good thing when it is a result of charities’ running out of room for more funding, and is acceptable/inevitable over relatively long periods of time, but that we should avoid situations in which some GiveWell-influenced donors are “canceling out” others at a more granular level and targeting an exact amount of total money moved.

3. How should we handle major donors who have privately told us about their giving plans and disagree with us on ideal funding targets?

A donor recently told us of their intention to give $1 million to SCI. We and the donor disagree about what the right “maximum” for SCI is: we put it at $6.3 million, while the donor – who is particularly excited about SCI relative to our other top charities – would rather see SCI as close as possible to the very upper end of its range, meaning they would put the maximum at $8.3 million. (This donor is relying on our analysis of room for more funding but has slightly different values.)

If we set SCI’s total target at $6.3 million, and took into account our knowledge of this donor’s plans, we would recommend a very small amount of giving – perhaps none at all – this giving season, since we believe SCI will hit $6.3 million between the $3 million from Good Ventures, $1 million from this donor, and other sources of funding that we detailed in our previous post. The end result would be that SCI raised about $6.3 million, while the donor gave $1 million. On the other hand, if the donor had not shared their plans with us, and we set the total target at $6.3 million, we would recommend $1 million more in support to SCI this giving season; the donor could wait for the end of giving season before making the gift. The end result would be that SCI raised about $7.3 million, while the donor gave $1 million.

We discussed what to do about this situation at length, and ended up pursuing a version of the “split the difference” heuristic. Since an informational advantage on our part would lead to SCI’s raising a total of ~$6.3 million, while an informational advantage on the donor’s part would lead to SCI’s raising a total of ~$7.3 million (in both cases including $1 million from the donor), we have set the total target at $6.8 million – the midpoint of those two figures. This is a compromise between “making no use of the information the donor has given us” (which we feel would be unfair to our audience) and “making full use of the information the donor has given us” (which we feel would be antagonistic to the donor and would give an incentive to not share plans with us in the future). We’ve discussed this resolution with the donor and agreed to it as representing good-faith behavior on both sides.

Your thoughts?

In all of the above cases, we’re largely improvising and looking for pragmatic solutions. We haven’t worked out a detailed framework for laying out principles and responding to different possible situations regarding the “giver’s dilemma.” We’d love thoughts from our audience on these matters.

Our updated top charities

(updated on: ) | by Elie

Our top charities are (in alphabetical order):

We have recommended all four of these charities in the past.

We have also included four additional organizations on our top charities page as standout charities. They are (in alphabetical order):

In the case of ICCIDD, GAIN-USI, and DMI, we expect to learn substantially more in the coming years (both through further investigation and through further progress by the organizations); we see a strong possibility that these will become top-tier recommended charities in the future, and we can see reasons that impact-minded donors could choose to support them today.

Ranking our top charities against each other is difficult and laden with judgment calls, particularly since:

  • Our cost-effectiveness analyses are non-robust, and reasonable people could reach a very wide variety of conclusions regarding which charity accomplishes the most good per dollar.
  • The charity we estimate as having the weakest cost-effectiveness (GiveDirectly) is also the one that we feel has the strongest organizational performance and the most direct, robust connection between donations and impact.
  • We do not currently feel highly confident in our cost-effectiveness estimates. We changed a number of inputs to our estimates recently. We did not have time to fully consider and vet them, and we plan to put more work into these estimates over the next few months. We do not expect our estimates to change significantly but given the fact that we have been updating them very recently, we would not be surprised if they do. We plan to publish a post soon detailing the major changes and most debatable assumptions in our current estimates. We consider the lateness of major revisions to this year’s estimates a shortcoming (and will be adding it to our mistakes page when we do our annual review).
  • This year we expect to influence a significant amount of donations. In some past years, we’ve been able to assume that each dollar of donations to an organization is about equally effective. This year, we could easily see one or more of our top charities reach the point of diminishing returns to additional donations and/or close its funding gap entirely.
  • We’ve been trying to predict and coordinate donations from Good Ventures, from individual donors, and from major donors who have given us private information about their plans. In so doing, we’ve run into game-theoretic challenges. If two donors are interested in funding the same organization, each has an incentive to downplay his/her interest in the hopes that the other will provide more of the funding. We’ve been trying to avoid reinforcing such incentives. We discuss how these considerations affected our targets below, and we plan to elaborate on this issue in a future post.
  • In past years, we’ve worked on an annual cycle, refreshing our recommendations each December. This year, because we anticipate closing (or nearly closing) the funding gaps of some of our top charities during giving season and moving a significant amount of money (~$5 million) after giving season before our next scheduled refresh, we plan to update our recommendations based solely on room for more funding in the middle of next year. We’re tentatively planning to do this on April 1st, the earliest we will realistically be able to post an update on charities’ ongoing funding needs that accounts for the funds they will receive over the next few months. This plan also raises questions about donor agency and coordination; we plan to discuss this in a future post.

We’ve tried to balance these considerations against each other and come up with an “ideal allocation” of the ~$7.5 million in estimated “money moved” we expect to influence (not counting grants from Good Ventures) over the next 4 months. Details are below. Based on this allocation, for any donors looking to give as we would, we recommend an allocation of $5 to AMF (67%), $1 to SCI (13%), $1 to GiveDirectly (13%) and $.50 to DtWI (7%) for every $7.50 given.

Good Ventures is planning to make grants of $5 million to each of AMF and GiveDirectly, $3 million to SCI, and $250,000 to DtWI. Good Ventures also plans to make grants of $250,000 to each of the standout organizations. We advised on these grants a few weeks ago, and did so while weighing our funding targets for each charity and forecasts of what other donors are likely to do; parts of our picture have since changed, and these grants do not represent the allocation we would advise donors to use nor do they reflect our views about the relative ranking of these organizations. We made sure to settle on and announce these grants before giving season so that no donor would have to grapple with questions about Good Ventures’s likely actions (more in our upcoming post on donor coordination), and Good Ventures will not be making additional grants to these charities in the near to medium future (6-12 months) unless there are substantive updates on things like evidence bases and capacity for absorbing money (i.e. Good Ventures will not be giving further simply in response to new information about donor behavior over the next 4 months).

Below we provide:

  • Additional detail on each of these eight organizations, including (for past recommendations) major changes over the past year, strengths and weaknesses for each, and our understanding of each organization’s room for more funding (which forms the basis for our funding targets and recommended allocation). More
  • The thinking behind our funding targets and recommended allocation. More
  • The process we followed that led to these top charities. More
  • Brief notes on giving now vs. giving later and giving to GiveWell vs. our top charities. More

Conference call to discuss our recommendations
We are planning to hold a conference call at 5:30pm EST on Wednesday, December 3rd to discuss our recommendations and answer questions. If you’d like to join the call, please register using this online form. If you can’t make this date but would be interested in joining another call at a later date, please indicate this on the registration form.

Top charities
We present information on our top charities in alphabetical order.

Against Malaria Foundation (AMF)
Our full review of AMF is here.

Important changes in the last 12 months

We named AMF our #1-ranked charity at the end of 2011. Over the next 2 years, AMF received more than $10 million on the basis of our recommendation but struggled to identify opportunities to use the funds it had received. At the end of 2013, we announced that we planned not to recommend additional donations to AMF until it committed the bulk of its current funds. This did not reflect a negative view of AMF; instead it reflected room for more funding related issues. More detail in this blog post.

In 2014, AMF finalized several distributions in Malawi and the Democratic Republic of the Congo (DRC) with three different implementing partners (two of which account for the bulk of the nets to be distributed). In 2014, it committed approximately $8.4 million to distributions which will take place before January 1, 2016 (some of which have already begun) and now has $6.8 million available for future distributions. $1.7 million of this is committed to a distribution scheduled for 2017 (and could potentially be allocated to distributions taking place sooner). Excluding the 2017 distribution, AMF has committed approximately $11.2 million to distributions in its history.

AMF continued to collect and share follow up information on its programs. We covered these reports in our August 2014 AMF update.

Funding gap

AMF requires access to funding in order to negotiate deals because it cannot initiate discussions with potential partners unless it is confident that it will have sufficient funding to support its future agreements. The funding it currently holds would enable it to fund approximately 3 distributions at a scale similar to what it has funded recently.

AMF has told us that it has a pipeline of possible future net distributions that add up to $36 million (details in our review).

We see some reason for caution in thinking about AMF’s room for more funding. It has made strong progress on being able to negotiate distributions and commit funds. However, as of today there have only been two large-scale distributions that have moved forward far enough for data to be available. Both of these are significantly smaller than distributions AMF has recently or will soon fund, and both are in the same area with the same partner as each other. Some of the recently negotiated distributions could prove more challenging (since they are in DRC).

If AMF received an additional $10 million in total over the next 4 months, it would have about twice as much funding available as the total it committed to large-scale distributions in 2014. (As stated above, it committed $8.4 million to distributions taking place before 2017 and has $6.8 million available for further commitments.) If it received $25 million, it would have about 4 times that total. 2-4 times past distributions seems like a range that would allow AMF to do significantly more than it has in the past, without going so far beyond its past capacity as to raise serious scaling concerns.

We believe that $10 million total (the low end of that range), which means $5 million after the Good Ventures grant, is an appropriate target after which further donations are likely better off going to other charities.

Key considerations:

  • Program impact and cost-effectiveness. Our best guess is that distributing bednets is in the same cost-effectiveness range as deworming programs and more cost-effective than cash transfers by a factor of 5-10. Our estimates are subject to substantial uncertainty. (Note: all our cost-effectiveness analyses are available here. Our file for bednets is here (.xls), and the comparison to deworming, cash transfers and iodine is here (.xls).)
  • Directness and robustness of the case for impact. We believe that the connection between AMF receiving funds and those funds helping very poor individuals is less direct than GiveDirectly’s and more direct than SCI’s or DtWI’s. The uncertainty of our estimates is driven by a combination of AMF’s challenges historically disbursing the funds it receives and a general recognition that aid programs, even those as straightforward as bednets, carry significant risks of failure via ineffective use of nets, insecticide resistance or other risks we don’t yet recognize relative to GiveDirectly’s program. AMF conducts extensive monitoring of its program; these results have generally indicated that people use the nets they receive.
  • Transparency and communication. AMF has been extremely communicative and open with us. We feel we have a better understanding of AMF than SCI and worse than GiveDirectly. In particular, were something to go wrong in one of AMF’s distributions, we believe we would eventually find out (something we are not sure of in the case of SCI), but we believe our understanding would be less quick and complete than it would be for problems associated with GiveDirectly’s program (which has more of a track record of consistent intensive followup).
  • Risks:
    • Two of AMF’s recent distributions (and much of its future pipeline) will take place in the DRC. Our impression is that the DRC is a particularly difficult place to work, and it is possible that AMF’s distributions there will struggle or fail. We view this as a moderate risk.
    • We are not highly confident that AMF will be able to finalize additional distributions and do so quickly. AMF could struggle again to agree to distribution deals, leading to long delays before it spends funds. We view this as a relatively minor risk because the likely worst case scenario is that AMF spends the funds slowly (or returns funds to donors).
    • We remain concerned about the possibility of resistance to the insecticides used in bednets. There don’t appear to be major updates on this front since our 2012 investigation into the matter; we take the lack of major news as a minor positive update.

A note on how quickly we expect AMF to spend the funds it receives. AMF works by sourcing, evaluating and negotiating deals for net distributions. This process takes time and requires AMF to have significant access to funding – it cannot approach a country to begin negotiations unless it is confident that it will have sufficient funding to pay for the nets it offers. We would not be surprised if AMF fails to reach additional deals in the next 12 months. We do expect it to commit the majority of its available funds (that it will have as of this coming January) within the next 24 months. If AMF does not make much progress in committing funds in the next 12 months, we will adjust our recommendation for 2015 accordingly, possibly recommending a lower target level of funds or suspending the recommendation entirely (depending on the specifics of the situation).

Our full review of AMF is here.

Deworm the World Initiative, (DtWI), led by Evidence Action
Our full review of DtWI is here.

Important changes in the last 12 months

Dr. Kevin Croke released a new study of a randomized controlled trial of a deworming program showing large, long-term impacts from deworming programs (for more, see this blog post). This study is a significant positive update on the impacts of deworming and increased our confidence that deworming programs have significant long-term impacts.

DtWI spent the funds it received due to GiveWell’s recommendation largely as we anticipated; it now has some (though limited) room for more funding.

In 2014, two events affected DtWI’s projection of the additional funding it would require to scale up in India:

  • The Children’s Investment Fund Foundation (CIFF), a major foundation that had supported DtWI’s programs in Kenya, agreed to a 6-year, $17.7 million grant to support DtWI’s expansion to additional states in India and technical assistance to the Government of India for a national deworming program. With these funds, DtWI does not require significant additional funding to support its India expansion.
  • The new Indian government expressed interest in conducting a single deworming day nationally with increased national attention and resources. Advocating for such a policy and assisting the national government in creating a plan became the major focus of DtWI’s India work in 2014, which both reduced the amount of time it was able to spend generating interest in heavy DtWI involvement in new states and also required little funding since there were few costs of that project aside from staff time. We see this as positive news regarding DtWI’s potential impact; it may simply reduce DtWI’s further need for funds from individual donors.

Together, these changes led DtWI to the conclusion that funding is no longer the bottleneck to reaching more people in India. (More detail in this blog post.)

Funding gap

DtWI told us that it seeks $1.3 million over the next two years. We expect it to allocate approximately 30% of the additional funds it receives for work related to expanding school-based, mass deworming programs (including related operating and impact evaluation expenses) and will allocate other funds to priorities that are less directly connected to expanding and evaluating deworming programs (investigating ways to combine other evidence-based programs with deworming rollouts, supplementing a project supported by another funder).

Good Ventures has announced a $250,000 grant to DtWI, leaving it with $1.05 million in remaining room for more funding over the next two years. We would ideally like DtWI to receive an additional $500,000 (for a total of $750,000) to provide it with more than half of its two-year gap.

Key considerations:

  • Program impact and cost-effectiveness. Our current calculations indicate that DtWI-associated deworming, when accounting for DtWI’s potential “leverage” in influencing government funds, has extremely strong cost-effectiveness, better than bednets and 10-20 times better than cash transfers. Our estimates are subject to substantial uncertainty. (Note: all our cost-effectiveness analyses are available here. Our file for deworming, cash transfers and iodine is here (.xls).)
  • Directness and robustness of the case for impact. DtWI doesn’t carry out deworming programs itself; it advocates for and provides technical assistance to governments implementing deworming programs, making direct assessments of its impact challenging. There are substantial potential advantages to supporting such an organization, as it may be able to have more impact per dollar by influencing government policy than by simply carrying out programs on its own, but this situation also complicates impact assessment. While we believe DtWI is impactful, our evidence is limited, and in addition, there is always a risk that future expansions will prove more difficult than past ones. In addition, DtWI is now largely raising funds to support research projects that are not directly connected to short-term implementation of deworming programs. We do not have a view about the value of these research projects.
  • Transparency and communication. DtWI has been communicative and open with us. We have only recommended DtWI for one year and therefore have less history with it than AMF, GiveDirectly, or SCI, but we believe that were something to go wrong with DtWI’s work, we would be able to learn about it and report on it.
  • Risks:
    • DtWI is part of a larger organization, Evidence Action, so changes that affect Evidence Action (and its other programs) could indirectly impact DtWI. For example, if a major event occurs (either positive or negative) for Evidence Action, it is likely that it would reduce the time some staff could devote to DtWI.
    • Most of DtWI’s funding is in the form of restricted funding from large, institutional funders. We are not sure how DtWI’s plans would change in response to a large funder offering it significant support to undertake a project not directly in line with its current plans.

Our full review of DtWI is here.

GiveDirectly
Our full review of GiveDirectly is here.

Important changes in the last 12 months

GiveDirectly continued to scale up significantly, utilizing most of the funding it received at the end of last year. It continued to share informative and detailed monitoring information with us. Overall, it grew its operations while maintaining high quality.

In June, three of its board members launched Segovia, a for-profit company aimed at improving the efficiency of cash transfer distributions in the developing world (see our blog post on Segovia for more information).

GiveDirectly is working with other researchers to begin a very large study on cash transfers and the impact they have on broader economic factors such as inflation and job growth. This study will include a long-term follow up component as well. GiveDirectly told us that the ideal sample size for this study, which is randomized at the village level, would require $15 million for cash transfers. Baseline data collection for the study began in August 2014. GiveDirectly has preregistered its plans for measurement and analysis (more information in our review).

Funding gap

GiveDirectly has scaled up significantly over the past year, spending (or committing to spend by enrolling recipients) approximately $13.6 million of the $17.4 million it received last year. (It also allocated an additional $1.8 million to other organizational costs.) It now believes that it could spend up to $40 million in a year.

We believe this is a reasonable cap for GiveDirectly and would not hesitate to see it receive this amount. However, due to other charities’ significantly superior estimated cost-effectiveness, we are seeking larger total amounts for them. We hope that GiveDirectly will receive at least $1 million from individual donors (excluding Good Ventures) this giving season as a result of our recommendation.

Key considerations:

  • Program impact and cost-effectiveness. Our best guess is that deworming or distributing bednets achieves 5-10 times more humanitarian benefit per dollar donated than cash transfers. Our estimates are subject to substantial uncertainty. (Note: all our cost-effectiveness analyses are available here. Our file for deworming, cash transfers and iodine is here (.xls).)
  • Directness and robustness of the case for impact. GiveDirectly collects and shares a significant amount of relevant information about its activities. The data it collects show that it successfully directs cash to very poor people, that recipients generally spend funds productively (sometimes on food, clothing, or school fees, other times on investments in a business or home infrastructure), and that it leads to very low levels of interpersonal conflict and tension. We are more confident in the impact of GiveDirectly’s work than in that of any of the other charities discussed in this post.
  • Transparency and communication. GiveDirectly has always communicated clearly and openly with us. It has tended to raise problems to us before we ask about them, and we generally believe that we have a very clear view of its operations. We feel more confident about our ability to keep track of future challenges than with any of the other charities discussed in this post.
  • Risks: GiveDirectly has scaled (and hopes to continue to scale) quickly. Thus far, it has significantly increased the amount of money it can move with limited issues as a result. The case of staff fraud that GiveDirectly detected is one example of an issue possibly caused by its pace of scaling, but its response demonstrated the transparency we expect.

Our full review of GiveDirectly is here.

Schistosomiasis Control Initiative (SCI)
Our full review of SCI is here.

Important changes in the last 12 months

As discussed above regarding DtWI, Dr. Kevin Croke released a new study of a randomized controlled trial of a deworming program showing large, long-term impacts from deworming programs (for more, see this blog post). This study is a significant positive update on the impacts of deworming and increased our confidence that deworming programs have significant long-term impacts.

We continued our work revisiting SCI’s case for impact (detailed here). There appear to have been major problems with some, though not all, of the studies we had relied on (pre-2013) to assess SCI’s impact. SCI shared some additional monitoring information with us which supported the conclusion that its programs have generally succeeded, though these reports have significant limitations.

We also reviewed the papers of several academics who had previously been critical of SCI’s activities. We found little in this literature to change our views on SCI’s programs.

We spent significantly more time with SCI in 2014 (including a 3-day visit to its headquarters in London) than we had in previous years, aiming to improve our understanding of its operations and spending. The picture that emerged was more detailed though largely consistent with what we believed before. Specifically:

  • We are less confident in our understanding of how SCI has spent unrestricted funds. At the end of 2013, we believed we had a relatively strong understanding of SCI’s unrestricted spending, but after spending additional time reviewing reports and discussing with SCI staff, we have more questions today than we did a year ago.
  • We have better information about how SCI plans to use additional funds it receives and the constraints, besides funding, that SCI faces in utilizing additional funding (more in our review).

Funding gap

SCI told us that it has approximately $3.8 million worth of opportunities that it would be highly likely to undertake if it had the funding available. (Some of this would be spent in 2015 and some held for the following year to ensure programs can continue once started). It believes it could possibly absorb an additional $4.5 million (up to $8.3 million total) for opportunities that are more speculative. Overall, our best guess is that SCI will use up to approximately $6.3 million and, beyond that, would build up reserves.

Partly for reasons of donor coordination, we have set its target at $6.8 million total (more below). We hope that SCI will receive $1 million from individual donors (excluding Good Ventures) this giving season as a result of our recommendation.

Key considerations:

  • Program impact and cost-effectiveness. Our best guess is that deworming is roughly as cost-effective as distributing bednets and more cost-effective than cash transfers by a factor of 5-10. Our estimates are subject to substantial uncertainty. (Note: all our cost-effectiveness analyses are available here. Our file for deworming, cash transfers and iodine is here (.xls).)
  • Directness and robustness of the case for impact. We have seen some evidence demonstrating that SCI successfully deworms children, though this evidence is relatively thin. Nevertheless, deworming is a relatively straightforward program, and we think it is likely (though far from certain) that SCI is successfully deworming people. We have had difficulties communicating with SCI (see below), which has reduced our ability to understand it; we have also spent significant time interviewing SCI staff and reviewing documents over the past 5 years and have found minor but not major concerns.
  • Transparency and communication. We have had consistent difficulties communicating with SCI. Specifically, (a) we had a major miscommunication with SCI about the meaning of its self-evaluations (more) and (b) although we have spent significant time with SCI, we remain unsure of how SCI has spent funds and how much funding it has available (and we believe SCI itself does not have a clear understanding of this). Importantly, if there is a future unanticipated problem with SCI’s programs, we don’t feel confident that we will become aware of it; this contrasts with AMF and GiveDirectly, both of which we feel we have a strong ability to follow up.
  • Risks: There are significantly more unknown risks with SCI than our other top charities due to our limited understanding of its activities. We hope for SCI to have $6.8 million available, which is significantly more unrestricted funding than it has had available in the past.

Our full review of SCI is here.


Summary
The table below summarizes the key considerations for our four top charities.

Consideration AMF DtWI GiveDirectly SCI
Program estimated cost-effectiveness (relative to cash transfers) 5-10x 10-20x 1x 5-10x (and possibly more)
Directness and robustness of the case for impact Strong Weakest Strongest Moderate
Transparency and communication Strong Strong Strongest Weakest
Ongoing monitoring and likelihood of detecting future problems Strong Strong Strongest Weakest
Organizational track record of rolling out program Moderate Moderate Strong Strong
Room for more funding (more below) High Limited Very high Limited when accounting for all donors

 

Note the absence of two criteria we have put weight on in years past:

  • Program evidence of effectiveness. With the new evidence about deworming, we think differences on this front are much reduced, though we still think net distribution and cash transfers have more robust cases than deworming.
  • Potential for innovation/upside. All of these organizations are fairly mature at this point, and we expect each to get significant revenue this giving season.

Standouts

Much of the work we did this year went into investigating potential new additions to our top charities list. The strongest contenders we found are discussed below.

Ultimately, none of these made it into our top tier of recommendations, but that could easily change in the future. We believe that more investigative effort could result in a much better understanding of GAIN-USI (discussed below) and potentially a top-tier recommendation. Meanwhile, ICCIDD and DMI (also discussed below) do not have the track record we’d want to see for our top tier of recommendations, but in both cases we expect major developments in the next year. Specifically, ICCIDD will have a substantially larger working budget (due to GiveWell money moved), and DMI may have new data from its randomized controlled trial that could cause a significant upgrade in its status.

These are all strong giving opportunities, and we’ve vetted them all relatively thoroughly. Two work on a program (universal salt iodization) that we believe has excellent cost-effectiveness and a strong evidence base, and the other two have recently released data from randomized evaluations of their own programs (something that is very rare among charities). We have thoroughly vetted each of these organizations, including site visits. And we can see arguments for supporting these organizations in lieu of our top charities this year, though we ultimately recommend our top charities above them.

Below are some brief comments on each standout organization. Donors interested in learning more should read our full reviews of each organization.

Development Media International (DMI) produces radio and television broadcasts in developing countries that encourage people to adopt improved health practices, such as exclusive breastfeeding of infants and seeking treatment for symptoms associated with fatal diseases. Its programs reach many people for relatively little money, so if its program successfully changes listeners’ behavior, it may be extremely cost-effective. It is in the midst of running a randomized controlled trial of its program; the midline results were released earlier this year, at which point we blogged about them.

At midline, the study found moderate increases (relative to the control group) in self-reported health behaviors. Our attempt to estimate the likely mortality impact of these behaviors – when accounting for other concerns about the generalizability of the study – implied cost-effectiveness worse than AMF’s. This isn’t sufficient for a recommendation this year, as DMI has much less of a track record than our top charities. However, if endline results hit DMI’s targeted mortality impact, we would expect to adjust our estimate significantly, and DMI could become a top charity.

DMI’s current budget is approximately $2.5 million; it has told us it expects to receive approximately $2.5-$4 million from existing funders in the next year and could absorb an additional $6-$7.5 million, which it would either use to supplement a program already broadcasting in a country or move into a new country, depending on how much it received.

Our cost-effectiveness analysis for DMI is here (.xls).

Our full review of DMI is here.

GAIN-USI. GAIN’s Universal Salt Iodization (USI) program supports salt iodization programs. There is strong evidence that salt iodization programs have a significant, positive effect on children’s cognitive development, and we consider the program to accomplish (very roughly speaking) comparable good per dollar to bednets and deworming (see our intervention report).

GAIN-USI does not work directly to iodize salt; rather, it supports governments and private companies to do so, which could lead to leveraged impact of donations or to diminished impact depending on its effectiveness. We tried but were unable to document a demonstrable track record of impact; we believe it may have had significant impacts, but we are unable to be confident in this with what we know now. More investigation next year could change this picture.

GAIN’s USI program was one of the recipients of a large, multi-year grant from the Bill and Melinda Gates Foundation. The grant ends in 2015 and has yet to be renewed; we are unsure of whether it will be.

Donors whose primary interest is supporting a strong intervention, and who are comfortable supporting a large and reputable organization whose role is to promote and support the intervention (but whose track record we cannot assess at this time), should strongly consider supporting GAIN’s USI program.

GAIN is a large organization running many programs, so donors should consider the possibility that funds restricted to GAIN’s USI program might effectively support its other efforts (more on this general concern here). GAIN told us that it has very little unrestricted funding, so it is unlikely to be able to reallocate funds from other programs to continue to support USI work. It is possible that resources that are shared across programs (such as some staff) could be shifted toward other programs if resources for USI increased, but we would guess that this effect would be small.

Our cost-effectiveness analysis for deworming, cash transfers and iodine is here (.xls).

Our full review of GAIN is here.

International Council for the Control of Iodine Deficiency Disorders Global Network (ICCIDD). Like GAIN-USI, ICCIDD supports (via advocacy and technical assistance rather than implementation) salt iodization, and as with GAIN-USI, we tried but were unable to establish a track record of successfully contributing to iodization programs. Unlike GAIN-USI, ICCIDD is small, operating on a budget of approximately half a million dollars per year, and relies heavily on volunteer time. We believe that additional funding in the range of a few hundred thousand dollars could have a significant positive impact on its operations.

Good Ventures has granted a total of $350,000 to ICCIDD this year ($100,000 as a participation grant and $250,000 with the grants announced today), and we would be happy to see ICCIDD receive a few hundred thousand dollars more, after which point we would be more hesitant as it would be more than doubling its budget. We hope that ICCIDD will use the additional funding to improve its capacity and potentially become a top charity in the future.

Our cost-effectiveness analysis for deworming, cash transfers and iodine is here (.xls).

Our full review of ICCIDD is here.

Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bed nets, clean cook stoves and solar lights.

It completed a randomized controlled trial of its program and measured a 27% reduction in child mortality. We estimate that Living Goods saves a life for roughly each $10,000 it spends, approximately 3 times as much as our estimate for the cost per life saved of AMF’s program. Living Goods has been operating on a budget of $3 million per year and aims to scale up to operate on a budget of $10 million per year, of which it expects to receive approximately two-thirds from existing funders.

Our cost-effectiveness analysis for Living Goods is here (.xls).

Our full review of Living Goods is here.

Funding targets by charity
In order to give guidance to donors seeking to give as we would, we’ve come up with funding targets for each charity. These targets are based on “dividing up” $7.5 million in money moved, which is our best guess for how much individual donors will give based on our recommendations over the next 4 months.

We are using the following principles in setting targets:

  • We’d like each top charity to receive a substantial amount of funding. When a charity receives substantial funding at our recommendation, it (a) gives that charity good reason to continue working with us, reporting to us, and helping us learn further about its activities; (b) gives that charity the opportunity to continue building its track record and demonstrating its capabilities, information we will use in future years; and (c) continues to reinforce the idea that GiveWell-recommended charities receive substantial funding – the main incentive charities have to participate in our process.
  • All else equal, we’d like stronger overall charities – defined as those that accomplish more good per dollar, taking all considerations into account – to receive more funding.
  • Each charity has a conceptual “maximum” past which we think donations would hit strongly diminishing returns. We aren’t allocating any “money moved” to a charity in excess of the max; beyond that point, we think the money is better spent supporting other top charities.

We are also taking the announced Good Ventures grants into account. These grants were recommended using similar considerations, though some of our information has changed.

Our targets are as follows. Note the distinction between “total max” (the most we’d be comfortable seeing a charity take in, at which point we would make an announcement), “total target” (the total amount we would like to see this charity take in, including Good Ventures grants and other donations), “target from individuals” (the amount we are seeking specifically from GiveWell-influenced individual over the next four months), and “max from individuals” (the most we’d be comfortable seeing a charity take in, taking into account what we know about other donors’ plans).

  • Against Malaria Foundation: $5 million target from individuals, $5 million max from individuals. As discussed in the section on AMF, our ideal amount for AMF to take in would be $10 million this giving season, and Good Ventures has already committed $5 million. We therefore target $5 million for AMF.
  • Deworm the World Initiative: $0.5 million target from individuals, $1 million max from individuals. We think Deworm the World Initiative is an outstanding giving opportunity with limited room for more funding, as discussed above.
  • Schistosomiasis Control Initiative: $1 million target from individuals, $1 million max from individuals. We believe SCI will end the giving season with $3 million from Good Ventures, $1 million from a major donor who discussed his plans with us, $1 million in donations that we expect to come from non-GiveWell-related sources (based on projections from past years rather than on knowledge of specific donors). We also believe it has $1 million in cash available for the $6.3-$8.3 million in opportunities we describe above. In total, then, SCI already can expect to have $6 million available, which would be around the maximum we’d recommend in isolation. However, our discussion with the possible $1 million donor has led us to set a higher overall “total target” than we would have otherwise, settling on a total target of $6.8 million. (We plan to elaborate on our thoughts about donor coordination and donor agency in a future post.) Since we are hoping for SCI to have a total of $6.8 million available for its activities, we are recommending $1 million in donations from GiveWell-influenced individuals this giving season. (We are rounding $0.8 million in estimated remaining gap to $1 million in recommended giving since these figures are not precise, and we see value in round numbers for our targets.)
  • GiveDirectly: $1 million target from individuals, $25 million max from individuals. We believe GiveDirectly could absorb up to $40 million total ($5 million from the Good Ventures grant, $10 million we expect it to receive from non-GiveWell-related sources already, and $25 million on top of that). However, our revised cost-effectiveness estimates (which we will discuss more in a future post) now classify cash transfers as significantly less cost-effective than bednet distribution and deworming, by a factor of around 5-10. In addition, the $5 million grant from Good Ventures and the funds we expect it to receive from elsewhere means that GiveDirectly will raise nearly as much in its next fiscal year as it did last year. Given that we anticipate moving roughly $7.5 million from individual donors in the next four months, we’d like to direct roughly $1 million of those donations to GiveDirectly. Note that GiveDirectly is, by a substantial amount, the organization we feel has performed best and most consistently in carrying out its intervention and providing quality data on the results, and people who are particularly skeptical of cost-effectiveness estimates are likely to find it the most appealing. We also are very excited about the future of GiveDirectly, in terms of its continuing ability to produce useful information via studies and its potential to grow and raise more from sources unconnected to GiveWell, though at this point we feel GiveDirectly is mature enough that further donations are not crucial in helping it toward this goal.

Summary table (all figures in USD millions):

Charity Total max (including all donations) Total target (including all donations) Donations committed or expected from Good Ventures and non-GiveWell sources Target from individuals Max from individuals
Against Malaria Foundation 10 10 5 5 5
Schistosomiasis Control Initiative 6.8 6.8 6 1 1
Deworm the World Initiative 1.3 0.75 0.25 0.5 1
GiveDirectly 40 16 15 1 25

For donations beyond the ~$7.5 million total we’re projecting over the next four months, we think the decision of which charity to support would be particularly difficult. Of our top charities, only GiveDirectly would have clear room for more funding after receiving an amount in line with the above, but the others – and to a lesser extent, some of our standout charities – have significantly superior estimated cost-effectiveness according to our latest analyses. We will be continuing to stress-test and reflect on these analyses as we reflect on the question of how to modify our recommendations once the above targets are hit.

Our research process in 2014
This section describes the new work we did in 2014 to supplement our previous work on defining and identifying top charities. See the process page on our website for our overall process.

This year, we completed an investigation of one new intervention (salt iodization). We made substantial progress on several others (maternal and neonatal tetanus immunization campaigns, mass drug administration for lymphatic filariasis, and vitamin A supplementation) but did not complete them.

We also stayed up to date on the research for bednets, cash transfers and deworming and made a substantial update to our view on deworming, based on a new study by Kevin Croke.

We did not conduct an extensive search for new charities this year. We feel that we have a relatively good understanding of the existing charities that could potentially meet our criteria, based on past searches (see the process page on our website for more information). Instead, we solicited applications from organizations that we viewed as contenders for recommendations. (Living Goods is an exception; it contacted us with the results from its randomized controlled trial.)

A February post laid out which organizations we were hoping to investigate and why.

In addition to the 4 standout charities, we also considered Nothing but Nets (a bednets organization that declined to participate in our process), Evidence Action’s Dispensers for Safe Water program (which is forthcoming), the Center for Neglected Tropical Disease and UNICEF’s maternal and neonatal tetanus program. In the case of the latter two, we ran out of time to complete the relevant intervention reports this year (due to prioritizing other work, which seemed more likely to lead to new recommendations) and plan to complete them in 2015.

Brief notes on giving now vs. later and supporting GiveWell vs. top charities
Giving now vs. giving later

Last year, some staff members chose to save some of their charitable giving budget for future giving opportunities, and we discussed the considerations about giving now vs. later in this post.

This year, we think the situation is a bit different, as AMF has returned to our top charities list, the case for both SCI and GiveDirectly has improved (due to new evidence on deworming and GiveDirectly’s strong performance in disbursing cash transfers), and we have extensively investigated possible other options. With these changes, we feel that (unlike last year) this year is an excellent year to give a substantial amount if you are interested primarily on our top charities work. We think our top charity recommendations are unlikely to improve a great deal (i.e. they’re unlikely to improve enough to make saving worthwhile) in the coming years. A couple considerations that might be relevant in weighing the decision to give now versus later:

  • Will the giving opportunities available in the future be better than the ones we have identified now? There are competing factors. On one hand, our research capacity has expanded significantly over the past 2 years, and this has given us the ability to research more opportunities both in our traditional, top charities work and the Open Philanthropy Project. On the other, the world is getting better and some of the best opportunities available today (e.g., deworming, bednets, salt iodization) may no longer be available 10 years from now. We now feel that we’ve investigated a large proportion of realistic short-to-medium-term contenders for top charity recommendations. If money moved ends up exceeding the ~$7.5 million we’re projecting over the next four months, a stronger case for waiting may emerge, as many of the strongest charities will be near what we think they can productively absorb in the short term (and our standout charities may become recommended next year, as discussed in the section on standouts).
  • How much funding will be available in the future to the opportunities we identify? Our impression is that funding available for the opportunities we identify has and will continue to grow significantly. Good Ventures is a part of this, but we hope that other future, major philanthropists will consider supporting our recommendations as Good Ventures has.

Donors interested in supporting opportunities that come from the Open Philanthropy Project have a stronger case for saving to give later. Note that it could be several years before the Open Philanthropy Project has recommendations suitable for individual donors, and these recommendations will likely reflect a very different process, very different criteria, and a much higher tolerance for high-risk opportunities that are difficult to fully explain and defend in writing (though we will work hard to lay out the basic case).

Giving to GiveWell vs. our top charities

We have grown significantly over the past 2 years and continue to raise funds to support our operations. The funds we have received have enabled us to expand our staff. Without this increased capacity, we would not have been able to consider as many organizations as we did this year.

We plan to post an update soon about our budget situation. The most up to date information available is linked from our August board meeting. The short story is that we are still seeking additional donations. For the first time this year, our checkout form will ask donors to consider allocating 10% of their donation to our operating expenses. This option is not yet live on our website; we hope to implement this change in the next few weeks.

Deworm the World Initiative (led by Evidence Action) update

(updated on: ) | by Elie

Summary

The Deworm the World Initiative (DtWI), led by Evidence Action, received approximately $2.3 million as a result of GiveWell’s recommendation last year. While there were some deviations, it largely allocated these funds as we expected.

DtWI now has limited room for funding; it is currently seeking to raise an additional $1.3 million to support its activities in 2015 and 2016. We expect it to allocate approximately 30% of the additional funds it receives for work related to expanding school-based, mass deworming programs and funding related operating expenses (including impact evaluation related expenses), and will allocate other funds to priorities that are less directly connected to expanding and evaluating deworming programs (investigating ways to combine other evidence-based programs with deworming rollouts, supplementing a project supported by another funder).

We currently expect to release updated recommendations by December 1st. We think it is likely that the Deworm the World Initiative will remain on our top charities list.

How did DtWI spend the money it received due to GiveWell, and how does this compare to our expectations?

GiveWell directed approximately $2.3 million to the Deworm the World Initiative since we added it to our top charities list in December 2013.

At the time of our recommendation, we expected DtWI to spend additional funds in the following ways; we did not have precise estimates for how much it would spend in each category:

  • Some portion to provide reserves for DtWI, both to make the organization more resilient and to allow it to respond to high impact opportunities
  • Some portion to allow DtWI to offer a lower-intensity level of assistance to regions that didn’t require its standard level of assistance
  • Some portion to support expansion to new states in India

It has allocated these funds as follows (years when we expect funds to be spent in parentheses; 2014 means funds have been spent):

  • $881,000 – ongoing reserves. Our understanding is that DtWI does not have plans to spend these funds in the near future. Instead, these funds make DtWI more robust as an organization: for example, it is less likely to need to significantly shift priorities in order to fundraise and it is more likely to be able to respond quickly to high-impact opportunities it identifies.
  • $509,000 – expansion into new countries (2015 and 2016). This includes preliminary work in Ethiopia, Indonesia, Philippines to support possible future work and $104,000 for prevalence surveys and technical assistance to the government and partner organization in Vietnam.
  • $430,000 – ongoing work in India (2014 and early 2015). This will fund a follow-up prevalence survey in Bihar to assess the impact of three rounds of deworming on worm prevalence and intensity, and enable expansion to preschool children there, as well as contribute to the third round of the Rajasthan and Delhi programs.
  • $207,000 – contribution to elimination research primarily funded by the Children’s Investment Fund Foundation (CIFF) and the Bill and Melinda Gates Foundation (BMGF) (2015-2017). CIFF and BMGF provided approximately $1.6 million in funding to the Deworm the World Initiative and the London School of Hygiene and Tropical Medicine to conduct research on the feasibility and cost effectiveness of breaking transmission of soil-transmitted helminths. Breaking transmission would potentially require a different approach (likely covering more than just school-aged children) than DtWI’s standard school-based deworming model.
  • $151,000 – DtWI overhead (2014). These funds support DtWI as an organization but are not directly programmed (e.g., a portion of Alix Zwane’s, the Executive Director of Evidence Action salary, Evidence Action financial staff, etc.). Note that DtWI estimated $151,000 based on allocating 15% of programmed GiveWell-sourced funding to DtWI overhead. DtWI said it could more explicitly track these funds but would be time consuming to do so. We agreed that more detailed accounting was not necessary.
  • $129,000 – additional staff (2014). In 2014, DtWI hired (a) a deputy director to support its programming worldwide and (b) someone to focus on its impact evaluation. The latter hire is likely to be doing work on the breaking transmission work discussed below. We allocate some of this line item to expansion and related operating expenses and some to research.

Overall, DtWI’s funding decisions seem reasonable to us and are broadly consistent with what we anticipated.

  • 46% ($1,067,000) supported expanding deworming programs and funding related operating expenses (including impact evaluation related expenses). This includes the deputy director who supports the organization as a whole but is necessary to expanded work in India and other new countries and half of the salary for the impact-evaluation-focused new staffer since he works on programmatic and technical support across DtWI.
  • 38% ($881,000) supported ongoing reserves.
  • 10% ($241,000) supported research that we had not anticipated (including the other half of the new staffer since he is spending a significant part of his time on this research).
  • 6% (151,000) supported DtWI as a whole.

How would DtWI spend additional funds?

The Deworm the World Initiative seeks an additional $1.3 million to support its activities in 2015 and 2016. DtWI expects to spend $377,000 of the $1.3 million (29%) it seeks on work related to expanding school-based mass deworming programs and funding related operating expenses (including impact evaluation related expenses). More specifically, these activities would be:

  • $230,000: staff to support expansion in India, new countries, and related operating and evaluation expenses. This line item is the salary for the deputy director and part of the salary for the impact evaluation focused staff member described above.
  • $144,000: DtWI overhead (described above).
  • $500,000: evaluation of new evidence-based programs that leverage deworming. We have limited detail about what this would entail. One idea that DtWI has investigated is the possibility of distributing bednets along with deworming pills in schools as an alternative distribution mechanism to national net distributions. Another is including hand-washing educational programming alongside deworming days. This line item includes $50,000 to support DtWI’s evaluation of its hygiene and deworming program funded by Dubai Cares and $50,000 to enable DtWI to hire a senior epidemiologist.
  • $230,000: staff to support evaluation of DtWI’s work in Kenya. This work is primarily funded by CIFF. DtWI believes that additional resources can improve significantly the quality of the analysis done regarding the cost effectiveness of breaking transmission. This line item includes $100,000 to support the impact evaluation focused staff member described above.
  • $170,000: implementation support for the integrated deworming, sanitation and hygiene education program in Vietnam, in partnership with Thrive Networks.

Why is DtWI seeking additional funds primarily to support research and evaluation rather than scale up? What changed in the past year?

In 2014, two events affected DtWI’s projection of the additional funding it would require to scale up in India:

  1. The Children’s Investment Fund Foundation (CIFF), a major foundation that had supported DtWI’s programs in Kenya, agreed to a 6-year, $17.7 million grant to support DtWI’s expansion to additional states in India and technical assistance to the Government of India for a national deworming program. At the end of 2013, DtWI believed it was reasonably likely that it would not receive this grant and had not anticipated how quickly it would come through. With these funds, DtWI does not require significant additional funding to support its India expansion.
  2. The new Indian government expressed interest in conducting a single deworming day nationally with increased national attention and resources. Advocating for such a policy and assisting the national government in creating a plan became the major focus of DtWI’s India work in 2014, which both reduced the amount of time it was able to spend generating interest in heavy DtWI involvement in new states, and also required little funding since there were few costs of that project aside from staff time. DtWI believes that the first national deworming day will likely happen in February 2015.

Together, these changes led DtWI to the conclusion that funding is no longer the bottleneck to reaching more people in India.

More broadly, we believe that if donors close both the $1.3-million 2-year funding gap of DtWI and the ~5-$8-million funding gap of the Schistosomiasis Control Initiative (SCI), another deworming organization we recommend, funding will not be the primary bottleneck to deworming programs’ scaling in general. Overall, our impression is that there is currently more funding available for scaling up deworming programs than capacity at organizations to utilize funds for scaleup.

Dr. Zwane believes that DtWI’s research agenda is important for two reasons:

  1. She believes it is possible that this research will demonstrate that other approaches to deworming are more cost-effective, such as eliminating worms from areas to avoid the need for mass treatments, or combining deworming with other interventions such as bednet distributions or hygiene education.
  2. She would like DtWI to consistently provide useful information to funders and policymakers and undertaking this research will enable it to continue doing so.

Notes on other deworming implementers and funders

It is not unlikely that GiveWell-directed donors will close the funding gaps of both DtWI and Schistosomiasis Control Initiative in the coming few months. Because of this, we also asked Alix Zwane (Executive Director of Evidence Action) about other implementers and funders working on deworming.

Implementers

Dr. Zwane told us DtWI and SCI are the two primary organizations that focus primarily on expanding countrywide deworming programs. Other organizations work on deworming but are not as directly focused on scaleup with government partners to her knowledge. There are other NGOs that work on other neglected tropical diseases (e.g., SightSavers) and school health (e.g., Partnership for Child Development), but Dr. Zwane is less familiar with the reach and scope of the service delivery they support.

Organizations that do a smaller amount of deworming implementation include UNICEF, Micronutrient Initiative and Vitamin Angels, which have begun adding deworming pills to their vitamin A supplementation programs, and WaterAid, which adds deworming to some of its water and sanitation programs.

IMA World Health, Helen Keller International, Sight Savers International, The MENTOR Initiative, and possibly others implement deworming programs supported by the funders discussed below. We have yet to speak with these organizations and have little information about their deworming programs or funding needs.

According to Dr. Zwane, the Global Network for Neglected Tropical Diseases works primarily on advocacy and does not focus on deworming, specifically, while Children without Worms coordinates partners globally and does not work on providing technical assistance for program delivery directly currently to her knowledge.

Funders

Major funders of deworming service delivery include the following: Dubai Cares, The END Fund, CIFF, the UK government’s Department for International Development (DFID), Michael and Susan Dell Foundation, and the US government’s USAID.

According to Dr. Zwane, these funders are interested in supporting scale-up, and she believes that DtWI will be in a strong position to raise funds for scale up from them if and when funding becomes a bottleneck. These funders are less likely to fund the types of activities to which DtWI has allocated GiveWell-directed funding.

A longer list of organizations working on deworming is available in this document, from a recent meeting of groups that are part of the STH Coalition.

Inaugural open thread

| by Alexander

A commenter recently suggested we try out a periodic open thread. Let’s give it a shot!

Our goal is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org if there’s feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

If this is helpful, we’ll continue to do them periodically in the future.

Quick update on Ebola (full update to come later)

| by Holden

At the moment, we don’t see highly time-sensitive giving opportunities that we are likely to recommend around responding to the Ebola outbreak in west Africa. We still plan to put out a detailed blog post describing what we’ve done to investigate the situation and what we think the best options for donors are, but we have now postponed this post in order to prioritize work on top charities. We hope to have the Ebola post out by early December.

Our ongoing review of GAIN

(updated on: ) | by Natalie Crispin

Note: GAIN has reviewed a draft of this post and added responses below.

The Global Alliance for Improved Nutrition (GAIN) focuses on reducing malnutrition by fortifying foods and condiments with essential nutrients and through other interventions. We have been considering GAIN for a 2014 recommendation for its work on Universal Salt Iodization (USI) because of our impression that iodization has strong evidence of effectiveness, cost-effectiveness, and room for more funding (our take on salt iodization is in our post on ICCIDD), and our understanding that GAIN is one of only a few organizations working on salt iodization on a large scale in many different countries. We decided to prioritize GAIN after speaking with Greg S. Garrett from GAIN as part of our investigation of ICCIDD.

We’ve now spent a considerable amount of time talking to GAIN and analyzing documents GAIN shared with us. This post shares what we’ve learned so far and what questions we’re planning to focus on throughout the rest of our investigation. (For more detail, see our detailed interim review.)

Broadly, GAIN’s USI work includes advocacy, technical assistance, supplying equipment and building supply systems for potassium iodate and iodized salt, training government officials and salt producers, and monitoring coverage and impact, among other activities.

GAIN has completed the first phase of our investigation process and we view it as a contender for a future recommendation. We now plan to make a $100,000 grant to GAIN (as part of our “top charity participation grants,” funded by Good Ventures). However, we are planning to prioritize work on other organizations first because we think that will yield a higher return on our time. GAIN is a challenging organization for us to evaluate, because its work (e.g., training/monitoring) is often several steps removed from increases in the impact we hope to see (e.g., improvements in iodization rates); we have evaluated and recommended organizations like this before, but with the information we currently have from GAIN, we feel we are still far from a full understanding of GAIN’s work and impact.

Questions we hope to answer in our ongoing analysis

We feel that we are still at a relatively early stage of our investigation of GAIN and have many important, open questions.

What’s GAIN’s track record at improving USI programs?

There are two components of this question, and we have only limited information on both: (1) in the countries that GAIN has worked, is a higher percentage of salt being iodized at the correct concentration than it was previously, and/or have measures of iodine levels in the population improved? and (2) what evidence is there that GAIN caused any improvements that are observed?

On (1), the trend seems quite mixed: in a set of nine countries, use of adequately iodized salt decreased in three countries, increased by a small amount in four countries, and increased substantially in two countries. Were we to rely on this data, we would need more information on the data sources. The data is not necessarily comparable within or across countries due to different survey design and salt iodine assessment methodology.

GAIN’s response: We have advocated globally and ensured that within our target countries a sub-sample of salt will be tested for iodine using a recognized quantitative method in any assessment taking place. This is reflected in the fact that eight of the nine update assessments measured salt iodine quantitatively, and all forthcoming (2015) end of project assessments will include quantitative analysis for the entire salt sample.

In the interim, the data we have access to (from surveys and other reliable sources of information since project start) for nine of our target countries show that baseline coverage of adequately iodized salt was 72.6%. This coverage has increased to 80.3% in these nine countries over the project period to date, representing an increase in the reach of adequately iodized salt to approximately 195 million people, mostly in China, Ethiopia, India and Pakistan.

We’ve looked more closely at iodization trends in four countries that GAIN highlighted to us in case studies of its work. In Bangladesh and India, rates of adequately iodized salt increased by a small amount (4 to 7 percentage points). In Ethiopia, GAIN reports that iodization rates have roughly doubled (from 20 to 40%) over a period that included GAIN’s involvement in the country (but follow up data is incomplete). In the fourth country, the Philippines, GAIN has only provided data for a group of salt producers it worked with, and found that samples meeting iodization standards increased from less than a third to over 80%.

We put limited weight on these claims – particularly in the last two examples – because we have not seen details of how the data was collected.

GAIN’s response: In India, which is self-sufficient in salt supply, the most recent data are based on supply QC reports from the national salt management information system, the development of which was a major achievement of GAIN support to the Salt Dept. This system has greatly facilitated obtaining timely and higher quality data, comparable over time, on iodized salt supply and quality. The system and rapid review time also increases accountability of salt producers as well as regulatory monitoring staff. How the supply QC data relate to household access to iodized salt will be determined after the national survey which will be conducted before the end of 2014.

2013 data for Ethiopia were presented at the Micronutrient Forum by the Ethiopian Public Health Institute. These salt iodine data are from a sample of over 5,000 households, selected from 354 enumeration areas to provide regionally and nationally representative data. The full national micronutrient survey, conducted by the end of 2014, will repeat the collection of salt samples and also include assessment of iodine status. It is expected that this will have improved considerably, in line with increased access to iodized salt, since the very low baseline. GAIN has been a key player working with the salt producers in challenging conditions to establish a sustainable KIO3 supply, develop iodization technology and is currently working on these as well as on improving quality control and regulatory monitoring of the salt supply with FMHACA.

For (2), linking GAIN to improvements in specific countries will be difficult because its activities (such as technical assistance, monitoring, training, etc.) tend to be a few steps removed from the outcomes of improved salt iodization and iodine consumption rates. Furthermore, GAIN generally works in collaboration with other groups (more below), making attribution to one party difficult or impossible. For more details on GAIN’s activities and reported accomplishments in these countries, see our interim review.

GAIN’s response: GAIN focuses its work primarily on proven drivers of supply which are mapped against the 2011 WHO/CDC logic model for micronutrient interventions in public health.  GAIN works closely with partners (government, private sector and other agencies in particular UNICEF) and therefore attributing progress to one agency alone and set of activities can be challenging. 

Do we have a comprehensive understanding of GAIN’s USI activities?

We have found it challenging to understand what, specifically, GAIN’s work on USI involves. This is because GAIN’s activities on USI vary considerably across countries and it works in 16 countries, and GAIN works closely with UNICEF, ICCIDD, and governments on this work, making it difficult to understand GAIN’s role. To date we have focused on understanding GAIN’s work in 4 countries because (a) GAIN has shared the most information for these countries, and (b) together they account for roughly 50% of its primary grant for USI.

Details in our interim review of GAIN.

GAIN’s response: As explained above, GAIN focuses its work primarily on proven drivers of supply which are mapped against the 2011 WHO/CDC logic model for micronutrient interventions in public health.  GAIN works closely with partners (government, private sector and other agencies in particular UNICEF) and therefore attributing progress to one agency alone and set of activities can be challenging.

Specifically, in addition to ongoing policy and advocacy and monitoring of national programs (e.g. designing and delivering coverage surveys) GAIN’s work can be summarized by the following four proven supply drivers: 1) improving quality assurance quality control/regulatory monitoring/improving testing capability; 2) integrating the use of adequately iodized salt in the food industry; 3) designing models for sustained KIO3 procurement; and 4) delivering innovations along the supply chain which apply positive pressure on iodized salt supply.

Room for more funding

GAIN’s USI work is mostly supported by a large, multi-year grant from the Gates Foundation that is ending in 2015. It is not yet clear if the Gates Foundation will renew this support. It seems likely that if it does not, GAIN will have a funding gap for its USI work. We have not yet discussed with GAIN how it would prioritize USI work at different levels of funding.

GAIN’s response: GAIN is concerned by the apparent lack of continued funding for iodine nutrition across the sector. GAIN is not alone in the sector and others are also struggling to fund what we believe to be critical activities towards improving global iodine intakes. This includes technical assistance, monitoring and evaluation and policy and advocacy towards continued improvement of national iodization programs and other proven iodine interventions. Global progress towards the virtual elimination of iodine deficiency disorders has been enormous to date but there is much work to be done. GAIN hopes to see commitments made by funders for this important work to ensure sustainability and to complete these efforts.