As discussed in our previous post, in 2013, we hoped to maintain the quality of our traditional research and recommendations while spending limited senior staff time (note 1) on our traditional work. This year, we plan to put significantly more total staff time (though roughly the same amount of senior staff time) into this work.

Beyond the additional research we do, an overarching goal remains building capacity for our traditional research and recommendations. Last year, we reduced the amount of senior staff time necessary for GiveWell’s traditional work, and we hope to reduce it further this year by continuing to train staff to take over the roles filled by senior staff in the past.

In addition to the new charities and interventions we plan to consider (discussed below), we will also continue to publish updates on our past top charities: GiveDirectly, the Schistosomiasis Control Initiative, the Deworm the World Initiative (led by Evidence Action), and the Against Malaria Foundation. We will also maintain our “open-door policy” so that any organization that believes it is a strong contender for our recommendations can apply via our application page.

Given our current capacity, we anticipate being able to consider 5 or more potential top charities and 5-10 additional interventions. We are also undertaking more experimental work that might lead to additional GiveWell top charities in the future.

Our plan for our traditional work is substantially more ambitious than what we planned last year (and than what we did – evaluate 1 new top charity and a few additional interventions, most of which we hope to publish material on shortly). This reflects our substantially increased staff, discussed in the previous post.

Priority charities

While maintaining our “open door” policy, we also plan to actively pursue investigations of several particularly promising charities. We believe that, due to our growing influence and improved processes, we will be able to get a greater level of engagement from charities we actively prioritize than we have in the past.

We chose charities primarily based on our best guess at the likelihood of eventually becoming a top-rated organization (often informed by our past interactions with the charity in question). We also gave additional weight to organizations that we guessed could plausibly be significantly better than our current top charities.

To arrive at this list, we (a) reviewed our full list of eligible organizations (i.e., charities running priority programs; (b) searched the Gates Foundation (the primary major funder focused on global health) grants database; (c) conducted Google searches for organizations working on our priority interventions (listed below).

• The International Council for the Control of Iodine Deficiency Disorders (ICCIDD) Global Network. ICCIDD focuses on technical assistance and advocacy for salt iodization.
• Center for Neglected Tropical Disease (CNTD). CNTD is based at the Liverpool School of Tropical Medicine and focuses on lymphatic filariasis (LF), a neglected tropical disease. Our impression is that it is plays a similar role in LF to the one the Schistosomiasis Control Initiative (a recommended charity of ours) plays in deworming.
• Nothing but Nets. Mass distribution of long-lasting insecticide treated nets to prevent malaria is a cheap and effective program.
• UNICEF’s maternal and neonatal tetanus supplemental immunization campaigns, the Measles and Rubella Initiative and the MenAfriVac vaccine rollout. We plan to revisit room for more funding and cost-effectiveness for these programs.

We also plan to conduct the following charity-related research:

• Complete a cost-effectiveness estimate for Dispensers for Safe Water, the second program run by Evidence Action, the organization which runs the Deworm the World Initiative.
• Review the midline results of Development Media International’s randomized controlled trial due in March 2014.
• If we have sufficient capacity, consider additional behavior change or nutrition organizations pending the completion of the necessary intervention reports.

All of our current top charities are relatively small (annual budgets of up to approximately $10 million) and – with the exception of Evidence Action – focus on only one program. With the exception of the Against Malaria Foundation, all of the charities we have recommended since 2011 have been run by people with academic backgrounds.

These characteristics are not strict criteria of ours, but recognition of the fact that they are shared by our current top charities leads us to believe that (a) these characteristics are a reasonable way to prioritize organizations and (b) working with organizations that don’t match these characteristics will likely be more time consuming. Our 2014 plans reflect this judgment. Most simply, we have prioritized organizations that pattern match our current top charities (i.e., ICCIDD and CNTD).

Priority intervention reports

We are currently near publication of a number of reports:

• Supplemental maternal and neonatal tetanus immunization campaigns
• Salt iodization
• Vitamin A supplementation
• Polio eradication

In 2014, we aim to complete 5-10 additional reports, roughly prioritized in the order below. We prioritized programs based on a combination of (a) our best guess at their cost-effectiveness and (b) the likelihood that, if we were to determine that an intervention is particularly promising, we could then find a charity implementing it to review and recommend.

To create this list, we aimed to cast a wide net, reviewing the interventions listed in (a) the Disease Control Priorities Report, (b) WHO-Choice, (c) the Copenhagen Consensus, and (d) the Lancet series on nutrition. An excel file with all interventions listed is available here.

• Nutrition programs
  • Iron fortification/supplementation
  • Folate fortification/supplementation
  • Zinc fortification/supplementation
  • Multi-vitamin fortification/supplementation (e.g., Sprinkles)
  • Others including maternal multiple micronutrient for pregnant/fertile women and calcium for pregnant/fertile women
• Behavior change:
  • Community-led total sanitation
  • Handwashing promotion
  • Condom promotion
  • Breastfeeding promotion
• Prevention of mother-to-child transmission of HIV (PMTCT)
• Oral rehydration therapy (with or without zinc)
• Treatments for HIV/AIDS, malaria, or pneumonia
• Intermittent preventive treatment of malaria for pregnant women (IPTp)

Experimental work to “seed” potential additional top charities

A major obstacle we face in our traditional work is the lack of charities that will likely be able to meet our criteria.

In light of this, we are working on several projects this year that are relatively experimental and may eventually lead to new top charities.

In January 2014, Good Ventures made a $100,000 grant to New Incentives, an organization focused on conditional cash transfers. New Incentives is a startup organization and does not currently qualify as a potential GiveWell top charity, but because it (a) runs a program that has significant evidence supporting it and (b) we believe that its founder, Svetha Janumpalli, is committed to allowing us to thoroughly vet her activities, we believe it is plausible that New Incentives will eventually become a GiveWell top charity. We are open to recommending other grants to other groups meeting similar criteria, if we come across them.

We would also guess that there are interventions that (a) could be considered priority programs if they had additional evidence supporting them and (b) could be scaled with additional funding. We intend to speak with organizations that conduct research (including Innovations for Poverty Action, among others) as well as Evidence Action (whose mission is to scale up evidence-based programs) to determine whether there are any interventions for which this is the case. If we identify strong contenders, we (in partnership with Good Ventures) will consider funding this research.

Finally, aside from the charities listed above, we know of no organizations focused on implementing priority programs. Engaging with large “mega-charities” has always posed a challenge for us for two reasons: (a) they tend to have large public relations and fundraising departments from which we have struggled to get helpful programmatic information and (b) because they run so many programs, we have struggled to feel confident that a donation restricted to “Program X” would in fact lead to more of Program X being implemented. On the other hand, we recognize that many major funders work directly with these large entities to implement programs, and while we know that such an investigation would be time consuming, we may now be at the point in our development where it is worthwhile.

We have not yet decided to move forward with engaging with a mega-charity; our plan is to first complete the necessary intervention reports and then assess whether any interventions/mega-charities appear to be sufficiently promising to be worth addressing the inherent challenges of this investigation.

Note 1: In this post, senior staff refers to Elie, Holden, and Alexander.

This is the second post (of six) we’re planning to make focused on our self-evaluation and future plans.

This post reviews last year’s progress on our traditional work of finding and recommending evidence-based, thoroughly vetted charities that serve the global poor. It has two parts. First, we look back at the plans we laid out in early 2013 and compare our progress against them. Then, we reflect on the state of our traditional work and what we learned about it in 2013.

Note: in past years, we answered a series of questions about GiveWell’s progress as a donor resource and as a project. Considering that template this year, we feel that those questions (which we first used in our 2009 annual review) have become stale and no longer represent the best approach to evaluating our progress. Instead, we quote from last year’s writeup on plans and goals for the year, and compare to our actual progress during the year.

Summary

At the beginning of 2013, we felt that our traditional research and recommendations were relatively strong and would remain a valuable resource with limited effort put into improving them. We therefore aimed to limit senior staff time (note 1) on it to enable progress on GiveWell Labs.

We ultimately spent significantly more senior staff time on our traditional work than we had anticipated, which we felt was necessary to maintain the quality of our research, though we did achieve our big-picture goal of allocating more senior staff time to GiveWell Labs than previously and making more progress on that front.

We kept our research up to date, including major updates on our top charities, and added a new top charity. We also hired 6 additional research analysts, which should enable us to increase the capacity devoted to our traditional work in the future. Growth in web traffic and donations remained strong (details in our forthcoming metrics post).

Media attention to our traditional work has increased, and we now see it as both more important and more challenging to sustain this work in the future. We expect substantially more production next year, due to our increased capacity and better success getting engagement from charities (more in the next post).

The items that we consider essential for our “traditional” work are:

• Continuing to do charity updates … on our existing top charities.
• Reviewing any charity we come across that looks like it has a substantial chance of meeting our traditional criteria as well as, or better than, our current #1 charity (which would require not only that the charity itself has outstanding transparency, but also that the intervention it works on has an outstanding academic evidence base). We have created an application page for charities that believe they can meet these criteria.
• Hiring. As mentioned previously, we believe our process has reached a point where we ought to be able to hire, train and manage people to carry it out with substantially reduced involvement from senior staff. We are currently hiring for the Research Associate role, and if we could find strong Research Associates we would be able to be more thorough in our traditional work at little cost to GiveWell Labs.

Below, we review each, in turn.

• Charity updates. We had relatively long histories with each of our 3 top charities, and expected that periodic conversations and review of documents by non-senior staff would allow us to stay up to date. These updates became significantly more involved than we anticipated because there were major developments with each of our top 3 charities (we had not expected any developments that would meaningfully affect our rankings), and these took significant senior staff time to investigate and write up.
  • The Against Malaria Foundation (AMF), our #1-ranked charity in 2011 and 2012, did not finalize a distribution to which it could allocate the bulk of funds it had on hand. This led us to remove AMF from our recommended charities list due to lack of room for more funding (details in this post).
  • We learned new information about the monitoring and evaluation that had provided a substantial part of the case for the Schistosomiasis Control Initiative (SCI), an organization we have recommended since 2011 (details in this post). SCI also significantly improved its communication with us, addressing many of the problems we noted earlier in the year.
  • Researchers released results from a randomized controlled trial of GiveDirectly’s program in late November, providing a key piece of evidence about GiveDirectly’s progarm’s impact (details in this post). This update, while substantial, did not require significant senior staff time.
• New charity reviews. At the beginning of the year, we did not expect to find new top charities in 2013. In April, we began evaluating the Deworm the World Initiative (led by Evidence Action). Our familiarity with deworming and our existing intervention report on deworming made this investigation easier than it otherwise might have been; the fact that DtWI was the first technical assistance/advocacy organization we had reviewed made this analysis more complicated. In all, this investigation took about as long as we would have expected given that it was a new top charity. This investigation required significant senior staff time, but a substantial portion of that time involved training Timothy Telleen-Lawton, a new hire whom we anticipate requiring reduced oversight in the future. Timothy led the investigation of DtWI and did the bulk of the work on it.
• Hiring. We made major progress on this goal. In addition to Tim, we hired five new Research Analysts, increasing our capacity for (a) conversation notes processing; (b) “vetting”, i.e., carefully reviewing pages pre-publication to identify any errors; (c) completing intervention reports, something that some of the new hires proved able to do with little involvement from senior staff. We completed new intervention reports on water quality and vitamin A supplementation (forthcoming).
• Donation processing. While we did not articulate this as a goal for 2013, we significantly improved our donation processing system over the course of the year. Processing donations and metrics from 2012’s giving season had taken approximately 3 full-time months. The improvements we made last year allowed us to process more than twice as many donations using less than half the time we had previously. (We also believe that these changes will scale well as we continue to grow.) These improvements were led by Natalie Crispin and required almost no senior staff time.

Three major observations from 2013 will inform our plans going forward.

• Significant growth in attention to/interest in our traditional work. Last year, we saw significant growth in interest in our traditional work from media and donors. At the beginning of last year, we would have guessed that growth would slow, yet we saw significantly more media attention than we had in past years, which contributed to our more than doubling the number of donors giving to our top charities and significant web traffic growth. Both the number of donors (as well as the total amount given by <$5,000 donors) and web traffic grew at a faster rate last year than they had in previous years; growth in donations from larger donors slowed relative to last year’s growth, though this trend is less robust because of the smaller numbers of people involved. (We will share full details on these figures in our forthcoming metrics post.)
• The challenge of maintaining the quality of our research and recommendations. In early 2013, we expected to spend relatively little time maintaining the quality of our research and recommendations. As discussed above, this did not work out as we had planned, and we no longer believe that we can easily maintain the quality of our research and recommendations without the engagement of highly skilled/trained staff. More broadly, we continued to learn about (and write about) the fundamental difficulty of the “giving as consumption” model underlying our traditional work, and believe that continuing to provide a strong resource using this model could take quite a bit of work and ingenuity going forward.
• Increased interest from charities in receiving our recommendations. As our money moved has grown, we have seen increased interest from charities in engaging with us and our process. This willingness to engage has grown consistently over the course of GiveWell’s history, but we now believe we have reached a level of money moved where most organizations will seriously consider engaging with us, as opposed to just ignoring us. This should enable us to choose the organizations that we think are most promising; encourage them to apply; and if they choose not to, understand what led them to decline (e.g., insufficient staff time, limited monitoring data, etc.). In the past, charities chose not to participate for reasons we often didn’t fully understand and we hope that this has now changed. This should eventually improve the robustness of our top charities list.

Note 1: In this post, senior staff refers to Elie, Holden, and Alexander.

This is the first post (of six) we’re planning to make focused on our self-evaluation and future plans.

As in past years, we’re going to be posting our annual self-evaluation and plan as a series of blog posts. This post summarizes what changed for GiveWell in 2013 and what it means for the future. Future posts will elaborate.

This year, we are separately reviewing our “traditional work” (recommending top charities that focus on evidence-backed programs serving the global poor) and GiveWell Labs (our newer project, on which we work closely with Good Ventures).

For us, the major developments of 2013 were:

• We continued to see strong growth in money moved and improved media coverage, though we are still working on assembling the details. There was some extent to which larger donors gave less in 2013 than previously, due to saving for future opportunities (potentially including AMF), so we aren’t yet sure what the final figures for larger donors will be, but the overall picture is one of strong growth.
• We made major progress on building capacity. We made a total of 6 new hires, and these new hires led work on our new top charity review (more below) and on new reports on priority interventions. We also improved our donation processing system, further improving our forward-looking capacity. As a result, our 2014 plan for our traditional work is substantially more ambitious than our 2013 plan was.
• We added a new top charity (Deworm the World Initiative) and produced substantial updates on our existing top charities, including suspending one recommendation for room-for-more-funding-related reasons.
• We spent more senior staff time than anticipated on our traditional work (since there were major new developments concerning each of our top charities) as opposed to GiveWell Labs.
• We made substantial progress on GiveWell Labs: we refined our basic framework and goals, gained some basic familiarity with two major areas of philanthropy (policy-oriented philanthropy and scientific research funding) that we had had little exposure to previously, and examined multiple potential causes at different levels of depth. We are setting a “stretch goal” for 2014 of making substantial commitments (multiple years, substantial funding) to causes within the broad areas of policy-oriented philanthropy and global catastrophic risks.
• Our traditional work (“top charities”) seems both more challenging and more important than it was a year ago – the former because of how much senior time it continues to take to respond to new developments, the latter because of the increased media coverage and continuing growth in money moved that we saw. We now see a strong possibility that GiveWell Labs eventually will become a separate organization, though it will remain part of GiveWell in the short term. We plan minor steps to begin moving toward a separation, including potentially renaming GiveWell Labs.
• Fundraising remains a priority. We are currently fundraising for unrestricted support, supporting a team that is allocated flexibly between GiveWell Labs and our more traditional work.

Overall, we consider 2013 to have been a year of substantial progress on research (both GiveWell Labs and our traditional work), staff capacity, and influence. Our 2014 goals will be more concrete and ambitious than our 2013 goals were.

Of course, we also made some mistakes in 2013, and we’ve recently updated our shortcomings log to reflect them. In particular, we produced public updates on AMF too infrequently.

A series of future posts will go into more detail on the above points.