This post is more than 7 years old

This is the second of four posts that form our annual review and plan for the following year. The first post reviewed our progress in 2016. The following two posts will cover GiveWell’s progress and plans as an organization and metrics on our influence on donations in 2016.

Our primary research goals for 2017 are to:

• Speed up our output of new intervention assessments, by hiring a Senior Fellow and by improving our process for reviewing interventions at a shallow level.
• Increase the number of promising charities that apply for our recommendation. Alternatively, we may learn why we have relatively few strong applicants and decide whether to change our process as a result. Research Analyst Chelsea Tabart will spend most of her time on this project.
• Through GiveWell Incubation Grants, fund projects that may lead to more top charity contenders in the future and consider grantees No Lean Season and Zusha! as potential 2017 top charities.
• Further improve the robustness and usability of our cost-effectiveness model.
• Improve our process for following the progress of current top charities to reduce staff time, while maintaining quality. We also have some specific goals (discussed below) with respect to answering open questions about current top charities.

We discuss each of these goals in more depth below.

Intervention assessments

Intervention assessments are key to our research process. We generally only consider recommending funding for programs that are implementing one of our priority programs (an exception is if an organization has done rigorous evaluation of its own program, though in practice we have found this to be very rare). In recent years, we have completed few intervention reports, which has limited our ability to consider new potential top charities. We plan to increase the rate at which we form views on interventions this year by:

• Hiring a Senior Fellow (or possibly more than one). We expect a Senior Fellow to have a Ph.D. in economics, public health, or statistics or equivalent experience and to focus on in-depth evidence reviews and cost-effectiveness assessments of interventions that appear promising after a shallower investigation. In addition, Open Philanthropy Project Senior Advisor David Roodman may spend some more time on intervention related work.
• Doing low-intensity research on a large number of promising interventions. We generally start with a two to four hour “quick intervention assessment,” and then prioritize interventions for a 20-30 hour “interim intervention report” (example). We don’t yet have a good sense of how many of these of these we will complete this year, because we’re unsure both about how much capacity we will have for this work and about how many promising interventions there will be at each step in the process.
• Continuing to improve our systems for ensuring that we become aware of promising interventions and new relevant research as it becomes available. We expect to learn about additional interventions by tracking new research, particularly randomized controlled trials, in global health and development and by talking to select organizations about programs they run that they think we should look into.

Charity applications

In the past few years, we have been surprised by how little interest there has been from charities in applying for a GiveWell recommendation. Our impression is that for global health and development charities there are relatively few funders of our size: in 2015, we tracked $110 million given due to our research; we are in the process of compiling the data for 2016, but expect it to be in the range of $80-90 million to recommended charities. We would like to better understand whether we have failed to get the word out about the potential value we offer or communicate well about our process and charities’ likelihood of success, or, alternatively, whether charities are making well-informed decisions about their fit with our criteria. (More on why we think more charities should consider applying for a GiveWell recommendation in this post.)

This year, we have designated GiveWell Research Analyst Chelsea Tabart as charity liaison. Her role is to increase and improve our pipeline of top charity contenders by answering charities’ questions about our process and which program(s) they should apply with, encouraging promising organizations to apply, and, through these conversations, understanding what the barriers are to more charities applying.

We aim by the end of the year to have a stronger pipeline of charities applying, have confidence that we are not missing strong contenders, or understand how we should adjust our process in the future.

Incubation Grants

We made significant progress on Incubation Grants in 2016 and plan in 2017 to largely continue with ongoing engagements, while being open to new grantmaking opportunities that are brought to our attention.

Among early-to-mid stage grants, we plan to spend the most time on working with IDinsight and New Incentives (where our feedback is needed to move the projects forward), and a smaller amount of time on Results for Development and Charity Science: Health (where we are only following along with ongoing projects).

Another major priority will be following up on two later-stage grantees, No Lean Season and Zusha!, groups that are contenders for a top charity recommendation in 2017. For No Lean Season, a program run by Evidence Action, our main outstanding questions are whether the program will have room for more funding in 2018 and whether monitoring will be high quality as the program scales. We have similar questions about Zusha! and in addition are awaiting randomized controlled trial results that are expected later this year.

Cost-effectiveness model

We plan to continue making improvements to our cost-effectiveness model and the data it draws on (separate from adding new interventions to the model, which is part of the intervention report work discussed above). Projects we are currently prioritizing include:

• Making it more straightforward to see how personal values are incorporated into the model and what the implications of those values are.
• Revisiting the prevalence and intensity adjustment that we use to compare the average per-person impact of deworming in places that our top charities work to the locations where the studies that found long-term impact of deworming were conducted. More in this post.
• Improving the insecticide-treated nets model by revisiting how it incorporates effects on adult mortality and adjustments for regions with different malaria burdens and changes in malaria burden over time.

Current top charities

Our goal this year is to maintain the quality of top charity updates while decreasing the amount of staff time we spend and we ask top charities to spend on this work. Below, we detail our plans for following up with each charity.

Deworm the World Initiative, GiveDirectly, and Schistosomiasis Control Initiative (SCI)

We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.

Against Malaria Foundation (AMF)

We have two major outstanding questions about AMF that we hope to make progress on this year:

1. Will AMF’s monitoring processes be high quality? We wrote about our concerns about AMF’s past monitoring last year and expect new information to be available this year.
2. Going forward, AMF aims to fund larger distributions and commit funding further ahead of when a distribution is scheduled to occur than it has, for the most part, done in the past. Will this increase the extent to which AMF funds displace funds from other sources, or will there continue to be evidence that AMF’s funds are largely adding to the total number of nets distributed? More on this question in our review of AMF.

To help us make progress on these questions, we and AMF have agreed to have monthly calls to discuss questions we have about the monitoring AMF is producing and what AMF is learning about distributions it is considering funding. We will likely also seek out calls with AMF’s partner organizations to discuss these questions.

In order to estimate AMF’s room for more funding, we will seek out information on the location and size of funding gaps for mass net distribution campaigns from AMF, the African Leaders Malaria Alliance, and possibly other funders of nets. As we have in the past, we will use this information in conjunction with conversations with AMF about non-funding bottlenecks to its ability to fill various gaps.

The END Fund – deworming program

Compared with the charities we have recommended for several years, we have more open questions about the END Fund. The main questions we plan to seek more information on this year are:

1. We have not yet seen monitoring on par with that from our other top charities from the END Fund. We expect results from coverage surveys from END Fund programs this year. Will these surveys be high quality and demonstrate that the END Fund is funding successful programs?
2. We have not yet tried to compare the cost-effectiveness of the END Fund to our other top charities in our cost-effectiveness model. We will be seeking additional information from the END Fund about cost per treatment and baseline infection rates.
3. Questions around room for more funding: the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, actual and projected revenue from other sources, and what deworming grantmaking opportunities the END Fund expects to have.

We visited the END Fund’s programs in Rwanda and Idjwi island, DRC in January 2017 and will publish notes and photos from our visit shortly.

Malaria Consortium – seasonal malaria chemoprevention program

As with the END Fund, we have more open questions about Malaria Consortium than we do for the charities we have recommended for several years. Our main priorities are:

1. Further research on the evidence of effectiveness, cost-effectiveness, and potential downsides of seasonal malaria chemoprevention (SMC) (due to time constraints we have not yet completed a full intervention report, though we felt sufficiently confident in the intervention to recommend Malaria Consortium).
2. Getting a better understanding of the methodology Malaria Consortium uses for estimating coverage rates.
3. Completing a more in-depth room for more funding analysis for the program for 2018 than we did for 2017.

Malaria Consortium expects to have several new studies of its SMC programs to share in April 2017 (details).

We may visit a Malaria Consortium seasonal malaria chemoprevention program in summer 2017.

Sightsavers – deworming program

As with the END Fund and Malaria Consortium, we have more open questions about Sightsavers than we do for the charities we have recommended for several years. We expect to make limited progress this year because the first deworming mass drug administration funded with GiveWell-influenced funds is not expected to take place until September at the earliest and monitoring results aren’t expected until early 2018. Because Sightsavers has done fairly little deworming in the past year, we don’t expect to be able to learn much from its ongoing programs. Our main priorities for the year are:

1. Getting more information from Sightsavers about baseline prevalence and intensity of worm infections in the areas it is working, to inform our cost-effectiveness analysis.
2. Using Sightsavers’ budget for the projects and planned treatment numbers to improve our estimate of the cost per treatment – another input into our cost-effectiveness analysis. Our current cost per treatment estimate is very rough.
3. Completing a room for more funding analysis for 2018.

Standout charities

Standout charities are groups that we have a large amount of information about and that meet some but not all of our criteria. Because we have not followed them closely over time, it is possible that they may now be a stronger fit (or that they no longer focus on the program we reviewed). We plan to have at least one phone call with each of these groups to discuss whether anything has changed that might lead us to reopen consideration of the organization as a potential top charity. Due to our focus on organizations that are most likely to become top charities, we don’t expect to make this work a priority beyond that.