# Our updated top charities for giving season 2016

We have refreshed our top charity rankings and recommendations. We now have seven top charities: our four top charities from last year and three new additions. We have also added two new organizations to our list of charities that we think deserve special recognition (previously called “standout” charities).

## Room for more funding analysis

Types of funding gaps

We’ve previously outlined how we categorize charities’ funding gaps into incentives, capacity-relevant funding, and execution levels 1, 2, and 3. In short:

• Incentive funding: We seek to ensure that each top charity receives a significant amount of funding (and to a lesser extent, that charities worthy of special recognition receive funding as well). We think this is important for long-run incentives to encourage other organizations to seek to meet these criteria. This year, we are increasing the top charity incentive from $1 million to$2.5 million.
• Capacity-relevant funding: Funding that we believe has the potential to create a significantly better giving opportunity in the future. With one exception, we don’t believe that any of our top charities have capacity-relevant gaps this year. We have designated the first $2 million of Sightsavers’ room for more funding as capacity-relevant because seeing results from a small number of Sightsavers deworming programs would significantly expand the evidence base for its deworming work and has the potential to lead us to want to support Sightsavers at a much higher level in the future (more). • Execution funding: Funding that allows charities to implement more of their core programs. We separated this funding into three levels: level 1 is the amount at which we think there is a 50% chance that the charity will be bottlenecked by funding; level 2 is a 20% chance of being bottlenecked by funding, and level 3 is a 5% chance. Ranking funding gaps The first million dollars to a charity can have a very different impact from, e.g., the 20th millionth dollar. Accordingly, we have created a ranking of individual funding gaps that accounts for both (a) the quality of the charity and the good accomplished by its program per dollar, and (b) whether a given level of funding is capacity-relevant and whether it is highly or only marginally likely to be needed in the coming year. The below table lays out our ranking of funding gaps. When gaps have the same “Priority,” this indicates that they are tied. When gaps are tied, we recommend filling them by giving each equal dollar amounts until one is filled, and then following the same procedure with the remaining tied gaps. See footnote for more.* The table below includes the amount we expect Good Ventures to give to our top charities. For reasons the Open Philanthropy Project will lay out in another post, we expect that Good Ventures will cap its giving to GiveWell’s top charities this year at$50 million. We expect that Good Ventures will start with funding the highest-rated gaps and work its way down, in order to accomplish as much good as possible.

Note that we do not always place a charity’s full execution level at the same rank and in some cases rank the first portion of a given charity’s execution level ahead of the remainder. This is because many of our top charities are relatively close to each other in terms of their estimated cost-effectiveness (and thus, the value of their execution funding). For reasons we’ve written about in the past, we believe it is inappropriate to put too much weight on relatively small differences in explicit cost-effectiveness estimates. Because we expect that there are diminishing returns to funding, we would guess that the cost-effectiveness of a charity’s funding gap falls as it receives more funding.

Priority Charity Amount, in millions USD (of which, expected from Good Ventures*) Type Comment
1 Deworm the World $2.5 (all) Incentive 1 SCI$2.5 (all) Incentive
1 Sightsavers $2.5 (all) Incentive 1 AMF$2.5 (all) Incentive
1 GiveDirectly $2.5 (all) Incentive 1 END Fund$2.5 (all) Incentive
1 Malaria Consortium $2.5 (all) Incentive 1 Other charities worthy of special recognition$1.5 (all) Incentive $250,000 each for six charities 3 SCI$6.5 (all) Fills rest of execution level 1 Highest cost-effectiveness of remaining level 1 gaps
4 AMF $8.5 (all) First part of execution level 1 Similar cost-effectiveness to END Fund and Sightsavers and greater understanding of the organization. Expect declining cost-effectiveness within Level 1, and see other benefits (incentives) to switching to END Fund and Sightsavers after this point. 5 END Fund$2.5 (all) Middle part of execution level 1 Given relatively limited knowledge of charity, capping total recommendation at $5 million 6 Sightsavers$0.5 (all) Fills rest of execution level 1 Similar cost-effectiveness to AMF and the END Fund
7 Deworm the World $2.0 (all) Fills execution level 2 Highest-ranked level 2 gap. Highest cost-effectiveness and confidence in organization 8 SCI$4.5 (all) First part of execution level 2 Highest cost-effectiveness of remaining level 2 gaps
9 Malaria Consortium $2.5 (all) Part of execution level 1 Given relatively limited knowledge of charity, capping total recommendation at$5 million
10 AMF $18.6 ($5.1) Part of execution level 1 Expect declining cost-effectiveness within level 1; ranked other gaps higher due to this and incentive effects
11 SCI $4.5 ($0) Fills execution level 2 Roughly expected to be more cost-effective than the remaining $49 million of AMF level 1 * Also includes$1 million that GiveWell holds for grants to top charities. More below.

## Summary of key considerations for top charities

The table below summarizes the key considerations for our seven top charities. More detail is provided below as well as in the charity reviews.

Consideration AMF Malaria Consortium Deworm the World END Fund SCI Sightsavers GiveDirectly
Estimated cost-effectiveness (relative to cash transfers) ~4x ~4x ~10x ~4x ~8x ~5x Baseline
Our level of knowledge about the organization High Relatively low High Relatively low High Relatively low High
Primary benefits of the intervention Under-5 deaths averted and possible increased income in adulthood Possible increased income in adulthood Immediate increase in consumption and assets
Ease of communication Moderate Strong Strong Strong Moderate Moderate Strongest
Ongoing monitoring and likelihood of detecting future problems Moderate Moderate Strong Moderate Moderate Moderate Strongest
Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation High: less than half of Execution Level 1 filled High: not quantified, but could likely use significantly more funding Low: Execution Levels 1 and 2 filled High: half of Execution Level 1 filled Moderate: Execution Level 1 and some of Level 2 filled Moderate: Execution Level 1 filled Very high: less than 15% of Execution Level 1 filled

## Our recommendation to donors

If Good Ventures uses a budget of $50 million to top charities and follows our prioritization of funding gaps, it will make the following grants (in millions of dollars, rounded to one decimal place): • AMF:$15.1
• Deworm the World: $4.5 • END Fund:$5.0
• GiveDirectly: $2.5 • Malaria Consortium:$5.0
• SCI: $13.5 • Sightsavers:$3.0
• Grants to other charities worthy of special recognition: $1.5 We also hold about$1 million that is restricted to granting out to top charities. We plan to use this to make a grant to AMF, which is the next funding gap on the list after the expected grants from Good Ventures.

We estimate that non-Good Ventures donors will give approximately $27 million between now and the start of June 2017; we expect to refresh our recommendations to donors in mid-June. Of this, we expect$18 million will be allocated according to our recommendation for marginal donations, while $9 million will be given based on our top charity list—this$9 million is considered ‘expected funding’ for each charity and therefore subtracted from their room for more funding.

$18 million spans two gaps in our prioritized list, so we are recommending that donors split their gift, with 75% going to AMF and 25% going to SCI, or give to GiveWell for making grants at our discretion and we will use the funds to fill in the next highest priority gaps. ## Details on new top charities Before this year, our top charity list had remained nearly the same for several years. This means that we have spent hundreds of hours talking to these groups, reading their documents, visiting their work in the field, and modeling their cost-effectiveness. We have spent considerably less time on our new top charities, particularly Malaria Consortium, and have not visited their work in the field (though we met with Sightsavers’ team in Ghana). We believe our new top charities are outstanding giving opportunities, though we think there is a higher risk that further investigation will lead to changes in our views about these groups. A note about deworming Four of our top charities, including two new top charities, support programs that treat schistosomiasis and soil-transmitted helminthiasis (STH) (“deworming”). We estimate that SCI and Deworm the World’s deworming programs are more cost effective than mass bednet campaigns, but our estimates are subject to substantial uncertainty. For Sightsavers and END Fund, our greater uncertainty about cost per treatment and prevalence of infection in the areas where they work leads us to the conclusion that the cost-effectiveness of their work is on par with that of bednets. It’s important to note that we view deworming as high expected value, but this is due to a relatively low probability of very high impact. Our cost-effectiveness model implies that most staff members believe you should use a multiplier of less than 1% compared to the impact (increased income in adulthood) found in the original trials—this could be thought of as assigning some chance that deworming programs have no impact, and some chance that the impact exists but will be smaller than was measured in those trials. Full discussion in this blog post. Our 2016 cost-effectiveness analysis is here. This year, David Roodman conducted an investigation into the evidence for deworming’s impact on long-term life outcomes. David will write more about this in a future post, but in short, we think the strength of the case for deworming is similar to last year’s, with some evidence looking weaker, new evidence that was shared with us in an early form this year being too preliminary to incorporate, and a key piece of evidence standing up to additional scrutiny. END Fund (for work on deworming) Our full review of END Fund is here. Overview The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming. About 60% of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list. We have not yet seen monitoring results on the number of children reached in END Fund-supported programs. The END Fund has instituted a requirement that grantees conduct coverage surveys and the first results will be available in early 2017. While we generally put little weight on plans for future monitoring, we feel that the END Fund’s commitment is unusually credible because surveys are already underway or upcoming in the next few months, we are familiar enough with the type of survey being used (from research on other deworming groups) that we were able to ask critical questions, and the END Fund provided specific answers to our questions. We have more limited information on some questions for the END Fund than we do for the top charities we have recommended for several years. We do not have a robust cost per treatment figure, and also have limited information on infection prevalence and intensity. Funding gap We estimate that the END Fund could productively use between$10 million (50% confidence) and $22 million (5% confidence) in the next year to expand its work on deworming. By our estimation, about a third of this would be used to fund other NTD programs. This estimate is based on (a) a list of deworming funding opportunities that the END Fund had identified as of October and its expectation of identifying additional opportunities over the course of the year (excluding opportunities to grant funding to Deworm the World, SCI, or Sightsavers, which we count in those organizations’ room for more funding); and (b) our rough estimate of how much funding the END Fund will raise. The END Fund is a fairly new organization whose revenue comes primarily from a small number of major donors so it is hard to predict how much funding it will raise. The END Fund’s list of identified opportunities includes both programs that END Fund has supported in past years and opportunities to get new programs off the ground. Sightsavers (for work on deworming) Our full review of Sightsavers is here. Overview Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and, more specifically, advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections. We believe that deworming is a highly cost-effective program and that there is moderately strong evidence that Sightsavers has succeeded in achieving fairly high coverage rates for some of its past NTD programs. We feel that the monitoring data we have from SCI and Deworm the World is somewhat stronger than what we have from Sightsavers—in particular, the coverage surveys that Sightsavers has done to date were on NTD programs that largely did not include deworming. Sightsavers plans to do annual coverage surveys on programs that are supported by GiveWell-influenced funding. We have more limited information on some questions for Sightsavers than we do for the top charities we have recommended for several years. We do not have a robust cost-per-treatment figure, though the information we have suggests that it is in the same range as the cost-per-treatment figures for SCI and Deworm the World. We also have limited information on infection prevalence and intensity in the places Sightsavers works. This limits our ability to robustly compare Sightsavers’ cost effectiveness to other top charities, but our best guess is that the cost-effectiveness of the deworming charities we recommend is similar. Funding gap We believe Sightsavers could productively use or commit between$3.0 million (50% confidence) and $10.1 million (5% confidence) in funding restricted to programs with a deworming component in 2017. This estimate is based on (a) a list of deworming funding opportunities that Sightsavers created for us; and (b) our understanding that Sightsavers would not allocate much unrestricted funding to these opportunities in the absence of GiveWell funding. It’s difficult to know whether other funders might step in to fund this work, but Sightsavers believes that is unlikely and deworming has not been a major priority for Sightsavers to date. Sightsavers’ list of opportunities includes both adding deworming to existing NTD mass distribution programs and establishing new integrated NTD programs that would include deworming and spans work in Nigeria, Guinea-Bissau, Democratic Republic of Congo, Guinea, Cameroon, Cote d’Ivoire, and possibly South Sudan. Malaria Consortium (for work on seasonal malaria chemoprevention) Our full review of Malaria Consortium is here. Overview Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria. The evidence for SMC appears strong (stronger than deworming and not quite as strong as bednets), but we have not yet examined the intervention at nearly the same level that we have for bednets, deworming, unconditional cash transfers, or other priority programs. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately powered to find an impact on mortality. Malaria Consortium and its partners have conducted studies in most of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, but leave us with some remaining questions about the program’s impact. Overall, we have more limited information on some questions for Malaria Consortium than we do for the top charities we have recommended for several years. We have remaining questions on cost per child per year and on offsetting effects from possible drug resistance and disease rebound. Funding gap We have not yet attempted to estimate Malaria Consortium’s maximum room for more funding. We would guess that Malaria Consortium could productively use at least an additional$30 million to scale up its SMC activities over the next three to four years. We have a general understanding of where additional funds would be used but have not yet asked for a high level of detail on potential bottlenecks to scaling up.

We do not believe Malaria Consortium has substantial unrestricted funding available for scaling up its support of SMC programs and expect its restricted funding for SMC to remain steady or decrease in the next few years.

## Details on top charities we are continuing to recommend

Against Malaria Foundation (AMF)

Our full review of AMF is here.

Background

AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net distributions (for protection against malaria) in developing countries. There is strong evidence that distributing nets reduces child mortality and malaria cases.

AMF provides a level of public disclosure and tracking of distributions that we have not seen from any other net distribution charity.

We estimate that AMF’s program is roughly 4 times as cost effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others.

Important changes in the last 12 months

In 2016, AMF significantly increased the number and size of distributions it committed funding to. Prior to 2015, it had completed (large-scale) distributions in two countries, Malawi and Democratic Republic of Congo (DRC). In 2016, it completed a distribution in Ghana and committed to supporting distributions in an additional three countries, including an agreement to contribute $28 million to a campaign in Uganda, its largest agreement to date by far. AMF has continued to collect and share information on its past large-scale distributions. This includes both data from registering households to receive nets (and, in some cases, data on the number of nets each household received) and follow-up surveys to determine whether nets are in place and in use. Our research in 2016 has led us to moderately weaken our assessment of the quality of AMF’s follow up surveys. In short, we learned that the surveys in Malawi have not used fully randomized selection of households and that the first two surveys in DRC were not reliable (full discussion in this blog post). We expect to see follow-up surveys from Ghana and DRC in the next few months that could expand AMF’s track record of collecting this type of data. We also learned that AMF has not been carrying out data audits in the way we believed it was (though this was not a major surprise as we had not asked AMF for details of the auditing process previously). AMF has generally been communicative and open with us. We noted in our mid-year update that AMF had been slower to share documentation for some distributions; however, we haven’t had concerns about this in the second half of the year. In August 2016, four GiveWell staff visited Ghana where an AMF-funded distribution had recently been completed. We met with AMF’s program manager, partner organizations, and government representatives and visited households in semi-urban and rural areas (notes and photos from our trip). Our estimate of the cost-effectiveness of nets has fallen relative to cash transfers since our mid-year update. At that point, we estimated that nets were ~10x as cost-effective as cash transfers, and now we estimate that they are ~4x as cost-effective as cash transfers. This change was partially driven by changes in GiveWell staff’s judgments on the tradeoff between saving lives of children under five and improving lives (through increased income and consumption) in our model, and partially driven by AMF beginning to fund bed net distributions in countries with lower malaria burdens than Malawi or DRC. Funding gap AMF currently holds$17.8 million, and expects to commit $12.9 million of this soon. We estimate it will receive an additional$4 million by June 2017 ($2 million from donors not influenced by GiveWell and$2 million from donors who give based on our top charity list) that it could use for future distributions. Together, we expect that AMF will have about $9 million for new spending and commitments in 2017. We estimate that AMF could productively use or commit between$87 million (50% confidence) and $200 million (5% confidence) in the next year. We arrived at this estimate from a rough estimate of the total Africa-wide funding gap for nets in the next three years (from the African Leaders Malaria Alliance)—estimated at$125 million per year. The estimate is rough in large part because the Global Fund to Fight AIDS, Tuberculosis and Malaria, the largest funder of LLINs, works on three-year cycles and has not yet determined how much funding it will allocate for LLINs for 2018-2020. We talked to people involved in country-level planning of mass net distributions and the Global Fund, who agreed with the general conclusion that there were likely to be large funding gaps in the next few years. In mid-2016, AMF had to put some plans on hold due to lack of funding.

We now believe that AMF has a strong track record of finding distribution partners to work with and coming to agreements with governments, and we do not expect that to be a limiting factor for AMF. The main risks we see to AMF’s ability to scale are the possibility that funding from other funders is sufficient (since our estimate of the gap is quite rough), the likelihood that government actors have limited capacity for discussions with AMF during a year in which they are applying for Global Fund funding, AMF’s staff capacity to manage discussions with additional countries (it has only a few staff members), and whether gaps will be spread across many countries or located in difficult operating environments. We believe the probability of any specific one of these things impeding AMF’s progress is low.

We believe there are differences in cost-effectiveness within execution level 1 and believe the value of filling the first part of AMF’s gap may be higher than additional funding at higher levels. This is because AMF’s priorities include committing to large distributions in the second half of 2019 and 2020, which increases the uncertainty about whether funding would have been available from another source.

We and AMF have discussed a few possibilities for how AMF might fill funding gaps. AMF favors an approach where it purchases a large number of nets for a small number of countries. This approach has some advantages including efficiency for AMF and leverage in influencing how distributions are carried out. Our view is that the risk of displacing a large amount of funding from other funders using this approach outweighs the benefits. If AMF did displace a large amount of funding which would otherwise have gone to nets, that could make donations applied to these distributions considerably less cost-effective. More details on our assessment of AMF’s funding gap are in our full review.

Deworm the World Initiative, led by Evidence Action

Our full review of Deworm the World is here.

Background

Deworm the World (evidenceaction.org/#deworm-the-world), led by Evidence Action, advocates for, supports, and evaluates deworming programs. It has worked in India and Kenya for several years and has recently expanded to Nigeria, Vietnam, and Ethiopia.

Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality.

As noted above, we believe that Deworm the World is slightly more cost-effective than SCI, more cost-effective than AMF and the other deworming charities, and about 10 times as cost-effective as cash transfers.

Important changes in the last 12 months

Deworm the World has made somewhat slower progress than expected in expanding to new countries. In late 2015, Good Ventures, on GiveWell’s recommendation, made a grant of $10.8 million to Deworm the World to fund its execution level 1 and 2 gaps. Execution level 1 funding was to give Deworm the World sufficient resources to expand into Pakistan and another country. Deworm the World has funded a prevalence survey in Pakistan, which is a precursor to funding treatments in the country. It has not expanded into a further country that it was not already expecting to work in. As a result, we believe that Deworm the World has somewhat limited room for more funding this year. Overall, we have more confidence in our understanding of Deworm the World and its parent organization Evidence Action’s spending, revenues, and financial position than we did in previous years. While trying to better understand this information this year, we found several errors. We are not fully confident that all errors have been corrected, though we are encouraged by the fact that we are now getting enough information to be able to spot inconsistencies. Evidence Action has been working to overhaul its financial system this year. Our review of Deworm the World has focused on two countries, Kenya and India, where it has worked the longest. In 2016, we saw the first results of a program in another country (Vietnam), as well as continued high-quality monitoring from Kenya and India. The Vietnam results indicate that Deworm the World is using similar monitoring processes in new countries as it has in Kenya and India and that results in Vietnam have been reasonably strong. Evidence Action hired Jeff Brown (formerly Interim CEO of the Global Innovation Fund) as CEO in 2015. Recently Evidence Action announced that he has resigned and has not yet been replaced. Our guess is this is unlikely to be disruptive to Deworm the World’s work; Grace Hollister remains Director of the Deworm the World Initiative. Funding gap We believe that there is a 50% chance that Deworm the World will be slightly constrained by funding in the next year and that additional funds would increase the chances that it is able to take advantage of any high-value opportunities it encounters. We estimate that if it received an additional$4.5 million its chances of being constrained by funding would be reduced to 20% and at $13.4 million in additional funding, this would be reduced to 5%. In the next year, Deworm the World expects to expand its work in India and Nigeria and may have opportunities to begin treatments in Pakistan and Indonesia. It is also interested in using unrestricted funding to continue its work in Kenya, and puts a high priority on this program. Its work in Kenya has to date been funded primarily by the Children’s Investment Fund Foundation (CIFF) and this support is set to expire in mid 2017. It is unclear to us whether CIFF will continue providing funding for the program and, if so, for how long. Due to the possibility that Deworm the World unrestricted funding may displace funding from CIFF, and, to a lesser extent, the END Fund and other donors, we consider the opportunity to fund the Kenya program to be less cost-effective in expectation than it would be if we were confident in the size of the gap. More details in our full review. Schistosomiasis Control Initiative (SCI) Our full review of SCI is here. Background SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify recipient countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs. SCI has conducted studies in about two-thirds of the countries it works in to determine whether its programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact. As noted above, we believe that SCI is slightly less cost-effective than Deworm the World, more cost-effective than AMF and the other deworming charities, and about 8 times as cost-effective as cash transfers. Important changes in the last 12 months In past years, we’ve written that we had significant concerns about SCI’s financial reporting and financial management, and the clarity of our communication with SCI. In June, we wrote that we had learned of two substantial errors in SCI’s financial managment and reporting that began in 2015. We also noted that we thought that SCI’s financial management and financial reporting, as well as the clarity of its communication with us overall, had improved significantly. In the second half of the year, SCI communicated clearly with us about its plans for deworming programs next year and its room for more funding. SCI reports that it has continued to scale up its deworming programs over the past year and that it plans to start up new deworming programs in two states in Nigeria before the end of its current budget year. This year, SCI has shared a few more coverage surveys from deworming programs in Ethiopia, Madagascar, and Mozambique that found reasonably high coverage. Professor Alan Fenwick, Founder and Director of SCI for over a decade, retired from his position this year, though will continue his involvement in fundraising and advocacy. The former Deputy Director, Wendy Harrison, is the new Director. Funding gap We estimate that SCI could productively use or commit a maximum of between$9.0 million (50% confidence) and $21.4 million (5% confidence) in additional unrestricted funding in its next budget year. Its funding sources have been fairly steady in recent years with about half of its revenue in the form of restricted grants, particularly from the UK government’s Department for International Development (this grant runs through 2018), and half from unrestricted donations, a majority of which were driven by GiveWell’s recommendation. We estimate that SCI will have around$5.4 million in unrestricted funding available to allocate to its 2017-18 budget year (in addition to $6.5 million in restricted funding). SCI has a strong track record of starting and scaling up programs in a large number of countries. SCI believes it could expand significantly with additional funding, reaching more people in the countries it works in and expanding to Nigeria and possibly Chad. More details in our full review. GiveDirectly Our full review of GiveDirectly is here. Background GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82% overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people. We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. It appears that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected (more). It continues to experiment heavily, with the aim of improving how its own and government cash transfer programs are run. It has recently started work on evaluations that benchmark programs against cash with the aim of influencing the broader international aid sector to use its funding more cost-effectively. We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost effective than cash. Important changes in the last 12 months GiveDirectly has continued to scale up significantly, reaching a pace of delivering$21 million on an annual basis in the first part of 2016 and expecting to reach a pace of $50 million on an annual basis at the end of 2016. It has continued to share informative and detailed monitoring information with us. Given its strong and consistent monitoring in the past, we have taken a lighter-touch approach to evaluating its processes and results this year. The big news for GiveDirectly this year was around partnerships and experimentation. It expanded into Rwanda (its third country) and launched a program to compare, with a randomized controlled trial, another aid program to cash transfers (details expected to be public next year). The program is being funded by a large institutional funder and Google.org. It expects to do additional “benchmarking” studies with the institutional funder, using funds from Good Ventures’ 2015$25 million grant, over the next few years.

It also began fundraising for and started a pilot of a universal basic income (UBI) guarantee—a program providing long-term, ongoing cash transfers sufficient for basic needs, which will be evaluated with a randomized controlled trial comparing the program to GiveDirectly’s standard lump sum transfers. The initial UBI program and study is expected to cost $30 million. We estimate that it is less cost-effective than GiveDirectly’s standard model, but it could have impact on policy makers that isn’t captured in our analysis. We noted previously that Segovia, a for-profit technology company that develops software for cash transfer program implementers and which was started and is partially owned by GiveDirectly’s co-founders, would provide its software for free to GiveDirectly to avoid conflicts of interest. However, in 2016, after realizing that providing free services to GiveDirectly was too costly for Segovia (customizing the product for GiveDirectly required much more Segovia staff time than initially expected), the two organizations negotiated a new contract under which GiveDirectly will compensate Segovia for its services. GiveDirectly wrote about this decision here. GiveDirectly told us that it recused all people with ties to both organizations from this decision and evaluated alternatives to Segovia. Although we believe that there are possibilities for bias in this decision and in future decisions concerning Segovia, and we have not deeply vetted GiveDirectly’s connection with Segovia, overall we think GiveDirectly’s choices were reasonable. However, we believe that reasonable people might disagree with this opinion, which is in part based on our personal experience working closely with GiveDirectly’s staff for several years. Funding gap We believe that GiveDirectly is very likely to be constrained by funding next year. GiveDirectly has been rapidly building its capacity to enroll recipients and deliver funds, while some of its revenue has been redirected to its universal basic income guarantee program (either because of greater donor interest in that program or by GiveDirectly focusing its fundraising efforts on it). We expect GiveDirectly to have about$20 million for standard cash transfers in its 2017 budget year. This includes raising about $15.8 million from non-GiveWell-influenced sources between now and halfway through its 2017 budget year (August 2017) and$4 million from donors who give because GiveDirectly is on GiveWell’s top charity list. $4 million is much less than GiveWell-influenced donors gave in the last year. This is because several large donors are supporting GiveDirectly’s universal basic income guarantee program this year and because one large donor gave a multi-year grant that we don’t expect to repeat this year. GiveDirectly is currently on pace (with no additional hiring) to have four full teams operating its standard cash transfer model in 2017. To fully utilize four teams, it would need$28 million more than we expect it to raise. We accordingly expect that GiveDirectly will downsize somewhat in 2017, because we do not project it raising sufficient funds to fully utilize the increased capacity it has built to transfer money. Given recent growth, we believe that GiveDirectly could easily scale beyond four teams and we estimate that at $46 million more than we expect it to raise ($66 million total for standard transfers), it would have a 50% chance of being constrained by funding.

## Other charities worthy of special recognition

Last year, we recommended four organizations as “standouts.” This year we are calling this list “other charities worthy of special recognition.” We’ve added two organizations to the list: Food Fortification Initiative and Project Healthy Children. Although our recommendation to donors is to give to our top charities over these charities, they stand out from the vast majority of organizations we have considered in terms of the evidence base for their work and their transparency, and they offer additional giving options for donors who feel highly aligned with their work.

We don’t follow these organizations as closely as we do our top charities. We generally have one or two calls per year with each group, publish notes on our conversations, and follow up on any major developments.

We provide brief updates on these charities below:

• Organizations that have conducted randomized controlled trials of their programs:
• Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its program and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15% or more) and data from health facilities finding an increase in facility visits. (The results, because the trial was only completed in the last year, are not yet published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. We noted last year that DMI was planning to conduct another survey for the RCT in late 2016; it has decided not to move forward with this, but is interested in conducting new research studies in other countries, if it is able to raise the money to do so. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI, with conversation notes and documents from 2016, is here.
• Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a randomized controlled trial of its program and measured a 27% reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of cash. Living Goods is scaling up its program and may need additional funding in the future, but has not yet been limited by funding. We published an update on Living Goods in mid-2016. Our 2014 review of Living Goods is here.
• Organizations working on micronutrient fortification: We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence.
• Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. Our full review is here.
• Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN, published in 2016 based on research done in 2015, is here.
• IGN. Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-influenced funding has made up a large part of its funding in the past year. This year, we published an update on our investigation into IGN’s work in select countries in 2015 and notes from our conversation with IGN to learn about its progress in 2016 and plans for 2017. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN, from 2014, is here.
• Project Healthy Children (PHC). PHC aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs. Our review is preliminary and in particular we do not have a recent update on how PHC would use additional funding. Our review of PHC, published in 2016 but based on information collected in 2015, is here.

## Our research process in 2016

We plan to detail the work we completed this year in a future post as part of our annual review process. Much of this work, particularly our experimental work and work on prioritizing interventions for further investigation, is aimed at improving our recommendations in future years. Here we highlight the key research that led to our current recommendations. See our process page for our overall process.

• As in previous years, we did intensive follow up with each of our top charities, including publishing updated reviews mid-year. We had several conversations by phone with each organization, met in person with Deworm the World, SCI, and AMF (over the course of a 4-day site visit to Ghana), and reviewed documents they shared with us.
• In 2015 and 2016, we sought to expand top charity room for more funding and consider alternatives to our top charities by inviting other groups that work on deworming, bednet distributions, and micronutrient fortification to apply. This led to adding Sightsavers, the END Fund, Project Healthy Children, and Food Fortification Initiative to our lists this year. Episcopal Relief & Development’s NetsforLife® Program, Micronutrient Initiative, and Nothing but Nets declined to fully participate in our review process.
• We completed intervention reports on voluntary medical male circumcision (VMMC) and cataract surgery. We asked VMMC groups PSI (declined to fully participate) and the Centre for HIV and AIDS Prevention Studies (pending) to apply. We had conversations with several charities working on cataract surgery and have not yet asked any to apply.
• We did very preliminary investigations into a large number of interventions and prioritized a few for further work. This led to interim intervention reports on seasonal malaria chemoprevention (SMC), integrated community case management (iCCM) and ready-to-use therapeutic foods for treating severe acute malnutrition and recommending Malaria Consortium for its work on SMC.
• We stayed up to date on the research for bednets, cash transfers, and deworming. We published a report on insecticide resistance and its implications for bednet programs. A blog post on our work on deworming is forthcoming. We did not find major new research on cash transfers that affected our recommendation of GiveDirectly.

## Giving to GiveWell vs. top charities

GiveWell and the Open Philanthropy Project are planning to split into two organizations in the first half of 2017. The split means that it is likely that GiveWell will retain much of the assets of the previously larger organization while reducing its expenses. We think it’s fairly likely that our excess assets policy will be triggered and that we will grant out some unrestricted funds. Given that expectation, our recommendation to donors is:

• If you have supported GiveWell’s operations in the past, we ask that you consider maintaining your support. It is fairly likely that these funds will be used this year for grants to top charities, but giving unrestricted signals your support for our operations and allows us to better project future revenue and make plans based on that. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and minimize staff time spent on fundraising.
• If you have not supported GiveWell’s operations in the past, we ask that you consider checking the box on our donate form to add 10% to help fund GiveWell’s operations. In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations.

Footnotes:

* For example, if $30 million were available to fund gaps of$10 million, $5 million, and$100 million, we would recommend allocating the funds so that the $10 million and$5 million gaps were fully filled and the $100 million gap received$15 million.

• Colin Rust on November 29, 2016 at 10:58 am said:

Just in time for Giving Tuesday, GiveWell sponsored a video about why malaria is so deadly, with a shout-out to AMF at the end.

• Michael Dickens on November 29, 2016 at 9:54 pm said:

> Our estimate of the cost-effectiveness of nets has fallen relative to cash transfers since our mid-year update. […] This change was partially driven by changes in GiveWell staff’s judgments on the tradeoff between saving lives of children under five and improving lives (through increased income and consumption) in our model…

In the 2016 cost-effectiveness spreadsheet, I see this tradeoff quantified in the “DALYs averted per death of an under-5 averted — AMF” row. I don’t see anything similar on the 2015 spreadsheet. In 2015 were you implicitly assuming that this value is 36.53 (the value in the “DALYs per life” cell)?

• Thanks for the great work guys. Couple questions.

1. How should we interpret the $50m build of Current Investment Assets in AML? 2. Does AML provide any transparency on their investory of nets? This is a pretty standard balance sheet line item, so a bit confusing that it is absent. • Andrew Martin on November 30, 2016 at 1:11 pm said: Hi Michael, You are correct that in our 2015 cost-effectiveness analysis model, all GiveWell staff used 36.53 as the “DALYs per life” input (sourced from Lopez et al. 2006, pg 402, Table 5.1, average of average male and female life expectancy at age 5, using 3% discounting and standard age weights). In our 2016 mid-year update model, some staff chose different values for this input to reflect their personal views about the moral value of saving infants’ lives. The average input among GiveWell staff for this value in the mid-2016 model was 26. In our most recent model, the average of GiveWell staff inputs for “DALYs averted per life of an under 5 saved – AMF” is lower, at 9. This change was driven by a few different factors. The set of staff members who contributed inputs for our most recent model is different than the set of staff members who contributed to the mid-2016 model. Three staff members with the highest “DALYs per death of a young child averted” inputs in mid-2016 did not contribute to the most recent model. Additionally, we added an estimate of reduction in adult mortality due to bed nets in our most recent update. This caused staff to engage in more depth than we had in the past with forming views on the relative value of saving lives at different ages. Many GiveWell staff concluded that they valued saving adult lives more highly (relative to saving the lives of children under 5) than the standard age-weights in Lopez et al 2006 imply, and adjusted values for DALYs averted per life of an under 5 saved to represent these values. [Edit: following sentence added after original posting] If you think your values may differ from GiveWell staff for this input, I’d encourage you to visit our cost-effectiveness analyses page, make an editable copy of our cost-effectiveness spreadsheet, and input your own values on the “Parameters” sheet in the “Anyone” column. • Natalie Crispin on November 30, 2016 at 1:20 pm said: Alex – I don’t follow your first question. Could you clarify? On your second question, when AMF signs an agreement to fund nets, it orders them from the manufacturer and has them shipped directly to the country where they will be distributed. It does not hold inventory. • Natalie Crispin on December 1, 2016 at 4:40 pm said: Alex – In response to your first question, Rob Mather at AMF pointed out to me that the Current Investment Assets you are referring to are in its most recent financial statements. AMF held$50.1 million in low-risk investment accounts as of June 30, 2016. This includes both funds that were committed to signed agreements to fund nets and other costs ($38.8 million total as of November 2016) and funds that are expected to be committed to distributions shortly ($12.9 million as of November 2016). More discussion of AMF’s committed and uncommitted funds in our review of AMF and in this document from AMF.

• Toby Ord on December 2, 2016 at 11:40 am said:

Great to see your updated recommendations. I was particularly happy to see the more nuanced breakdown of funding gaps into your three execution levels, representing a form of diminishing marginal value of donations, rather than sharply delineated gaps. I think one can go even further in this direction, for even if an organisation would stop being financially constrained in the next financial year if it received some large amount of donations, these donations would constitute a strong signal (or even financial guarantee) that may allow it to expand its capacity to spend donations faster in future years.

I also found it very helpful to be able to see your own estimates of comparative cost-effectiveness of all Top Charities to that of GiveDirectly. However, I’m a bit surprised that GiveDirectly itself made it onto the top charities list if it is only 10% as effective (in expectation) as Deworm the World, and at most 25% as effective as any of the others. Losing 90% or 75% of the value a donor could have created is a big deal, so I would have thought it more suited for Special Recognition rather than Top Charity. I also note that the other 6 are in a cluster with the most effective being only 2.5x the least, while GiveDirectly is estimated as a full 4x below that.

I understand that there was previously a lot of additional value in recommending it to help get it stably launched (something I didn’t sufficiently appreciate at the time). But given its success, I would have expected a return to simply measuring it by marginal value of donations, which brings us back to it being estimated at only 10% to 25% the value of the others in the list.

• Ben Millwood on December 4, 2016 at 5:42 am said:

It surprises me to see that AMF’s estimated cost-per-life-saved-equivalent number hasn’t changed a lot ($3221) despite the bit about “Our estimate of the cost-effectiveness of nets has fallen relative to cash transfers”. Do cash transfers look a lot better than they used to, or am I missing something else? (I know I’m nto supposed to take these numbers too literally, but I like to be up to date on which number I’m not taking too literally 🙂 ) • Avi Norowitz on December 4, 2016 at 8:30 am said: Ben, My understanding is that it’s the estimate of the value of an under 5 life saved that GiveWell has substantially changed, not the cost. See Natalie’s comment above in reply to Michael. • Colin Rust on December 4, 2016 at 3:42 pm said: Toby Ord, on recommending GiveDirectly, despite it being significantly less effective according to GiveWell’s best estimate, I’d note: 1. It’s only getting 5% of the recommended dollars ($2.5m of \$50m) from Good Ventures, less than any other recommended charity.
2. Furthermore, it has no weight in GiveWell’s recommendation for other donors.
3. On the other hand, philosophically it is very different from the other charities. Some donors might plausibly judge it to be a better use of charitable dollars.
4. One can imagine (and hope!) that deworming and anti-malaria interventions will be fully funded within a few years. But essentially no matter what, there will be capacity to deploy charitable dollars in cash transfers for the foreseeable future.

GiveWell A little thing, but I find “charity worthy of special recognition” a mouthful. Was there some problematic connotation with “standout charity”?

• Christian Smith on December 5, 2016 at 11:51 pm said:

Hi Ben—

As Avi mentioned above, the average value staff members placed on the life of a child under 5 was substantially lower in this latest update than in our mid-year update.

The “cost per life saved equivalent” is a measure of how expensive it is to accomplish something that we consider as good as AMF saving the life of a young child. Since the value of saving a young child was a lot lower in this update, GiveDirectly’s cost per life saved equivalent fell substantially.

• Catherine (GiveWell) on December 7, 2016 at 12:00 pm said:

Hi Toby—

GiveDirectly is on our list because we think our cost-effectiveness analyses have sufficient uncertainty that donors with particular (plausible) values and beliefs may find GiveDirectly to be competitive with the other top charities we recommend.

See, for example, GiveWell staff’s estimates of the cost-effectiveness of our top charities: https://docs.google.com/spreadsheets/d/1KiWfiAGX_QZhRbC9xkzf3I8IqsXC5kkr-nwY_feVlcM/edit#gid=1034883018; some GiveWell staff estimate the cost-effectiveness of the Against Malaria Foundation to be closer to 1-2x GiveDirectly (others are closer to 10x+). Additionally, these estimates may overstate the differences. Depending on one’s priors and one’s picture of the usefulness/robustness of the formal estimates, one might consider a 2x (or even 10x) difference in the estimates to be essentially uninformative about the differences in cost-effectiveness. And these estimates explicitly do not include factors such as organizational strength and propensity to continue innovating, both of which GiveDirectly stands out on (see https://www.givewell.org/charities/give-directly#GiveDirectly_as_an_organization and the relevant section of this blog post).

• You wrote the following about execution funding gaps:

“Execution funding: Funding that allows charities to implement more of their core programs. We separated this funding into three levels: level 1 is the amount at which we think there is a 50% chance that the charity will be bottlenecked by funding; level 2 is a 20% chance of being bottlenecked by funding, and level 3 is a 5% chance.”

It seems likely that every charity will be bottlenecked by funding eventually at some point in the future, so one has to specify a time frame for those estimates. Natural assumptions would be either “at some point in 2017” or “at some point in the first half of 2017” – which one is it?

Also, am I correct to assume that the incentive funding amounts are annual amounts (and not paid out twice per year , i.e. when recommendations are updated, or something like that)?

• Natalie Crispin on February 1, 2017 at 1:04 pm said:

Jonas –

The execution funding levels are set based on chances of being bottlenecked by funding in the next year (generally the calendar year, but charity budgeting cycles can vary, so it can differ case by case). That includes being bottlenecked by not having funds to spend in that year and also includes being bottlenecked on the ability to move forward with plans for future years due to not having or not expecting to have funds on hand.

The incentive grants are made annually.

• Thanks, that’s helpful! I think this page is now the most elaborate operationalization of RFMF on the internet. 🙂

• Andreas Zollmann on February 13, 2017 at 12:13 am said:

Concerning the anti-malaria charities: have you considered the potential impact of CRISPR gene drive technology?
Bill Gates thinks this will be deployable in two years: https://www.technologyreview.com/s/602304/bill-gates-doubles-his-bet-on-wiping-out-mosquitoes-with-gene-editing/

As this would completely eradicate malaria if it works, making these charities obsolete, you should discount their valuations based on the probability and expected time of this technology to work.

• Avi Norowitz on February 13, 2017 at 11:06 am said:

Andreas,

I don’t think gene drives would be appropriate as a GiveWell recommendation at this time, but the Open Philanthropy Project has issued a grant to create a working group on gene drives:

https://www.openphilanthropy.org/focus/scientific-research/miscellaneous/foundation-national-institutes-health-working-group

There is also some discussion about the use of gene drives and other transgenic mosquitoes for malaria control in this GiveWell research discussion transcript starting at 28:00 (page 7) and again at 41:56 (page 10):

http://files.givewell.org/files/Research%20meetings/GiveWell_NYC_June_2016_Research_Event_Top_Charities.pdf

The audio is available here:

https://www.givewell.org/files/Research%20meetings/GiveWell_NYC_June_2016_Research_Event_Top_Charities_Audio.mp3

Full disclosure: I am speaker 9, asking the question about Oxitec transgenic mosquitoes.

Avi

• Avi Norowitz on February 13, 2017 at 11:34 am said:

Andreas,

Upon rereading your question, I realize I didn’t really answer it. In regard to your question specifically: The 2 year estimate for the widespread use of gene drives for malaria control seems wildly overly optimistic to me given the potential risks of gene drives. GiveWell estimates that long lasting insecticide-treated nets last only 2.22 years anyway, although there is sometimes a multi-year delay from donations to distributions of nets.

https://www.givewell.org/international/technical/programs/insecticide-treated-nets

Avi

• Natalie Crispin on February 13, 2017 at 12:14 pm said:

Andreas – It’s an interesting point and something we will definitely keep an eye on as this technology develops and consider how it affects the value of distributing insecticide-treated nets and seasonal malaria chemoprevention (SMC). I agree with Avi about the 2-year estimate for the large-scale deployment of anti-malaria gene drive technology.

As Avi also noted, the effects of nets and SMC are primarily (and, in our model, entirely) on malaria cases in the next few years (for nets, which last ~2 years, though there is generally a delay of 1-3 years between donations and nets deployed) and over the next annual period of elevated malaria rates (for SMC). I would guess that we are still 5 or more years away from large-scale deployment of anti-malaria gene drive technology, and so this does not affect my valuation of donations to AMF significantly, but people with more knowledge of the research and regulatory hurtles may disagree.

• Andreas Zollmann on February 13, 2017 at 2:28 pm said:

Thanks for clearing that up guys.

• Catherine on February 13, 2017 at 6:08 pm said:

Hi Colin,

We decided to change the name of “other charities worthy of special recognition” back to “standout charities.” More here: https://blog.givewell.org/2016/12/14/december-2016-open-thread/#comment-941297.