This is the second post (of five) that we’re planning to make focused on our self-evaluation and future plans.

This post answers a set of critical questions about the state of GiveWell as a donor resource. The questions are the same as last year’s.

Does GiveWell provide quality research that highlights truly outstanding charities in the areas it has covered?

Where we stood as of Feb 2012

We felt that current research was high-quality and up-to-date. However:

• We felt that there were multiple areas that could offer outstanding opportunities that we had not yet researched as thoroughly as we could have (particularly in the areas of nutrition, vaccinations, neglected tropical disease control, tuberculosis control, and research and development).
• We were not satisfied with the degree to which our research was “vetted.” It still seemed to us that we could make a substantial mistake or error in judgment, with too high a probability that it would remain unnoticed.
• We worried about our total “room for money moved,” which we estimated at $15-20 million in our top charities; it seemed possible to us that continued rapid growth could potentially lead us to “run out” of great giving opportunities.

Progress since Feb 2012

In 2012, we wrote that we wanted to:

1. Revisit the goal of having our work subjected to formal, consistent, credible external review.
2. Continue to look for more outstanding giving opportunities for individual donors, particularly in the areas we have identified as most promising (i.e. global health and nutrition).
3. Begin to look for more outstanding giving opportunities for individual donors through GiveWell Labs.

In 2012, we made limited progress on #1, strong progress on #2, and less than anticipated progress on #3:

1. We did not solicit any new external reviews of our work in 2012, and we did not formally revisit the goal of doing so. Rather than focusing on increasing formal expert review over the past year, we subjected our key pages to a higher level of pre-publication internal review, ensuring that pages and spreadsheets that play an important role in our final recommendations are thoroughly checked by at least one person who did not play a role in their production. We do not view this change as eliminating the eventual need for formal outside review, but we see it as adequate for our current needs. We also feel that the increased level of informal critical attention our research has received from the outside has lowered the need for formal external review (more on this in a future post).
2. We added GiveDirectly to our list of top-rated charities in November 2012, after a thorough review that included a site visit and review of the evidence for unconditional cash transfers. We also conducted further investigations in the area of global health and nutrition:
   • The most promising new interventions that we looked into, immunizations, Vitamin A supplementation (forthcoming), and salt iodization (forthcoming), do not appear to have the sort of room for more funding that our top charities do (i.e., we do not see opportunities to translate additional dollars directly into additional people reached).
   • We subjected our existing work on bednets and deworming to a higher level of scrutiny (see Revisiting the case for insecticide-treated nets, Insecticide resistance and malaria control, Revisiting the case for developmental effects of deworming, New Cochrane review of the Effectiveness of Deworming).
   • We continued to analyze organizations that seemed promising at the end of our 2011 round of research. We completed or updated reviews for many of these, including GiveDirectly (now a top-rated charity) and VillageReach (our #1-ranked charity from 2009-2011) in addition to Doctors without Borders, Deworm the World, PATH, and many others. (Our “newly published materials” email list provides links to all the materials we’ve recently published.)
3. In the realm of GiveWell Labs,
   • We spent a substantial amount of time this year working on meta-research, including recommending a grant to the U.S. Cochrane Center.
   • We also worked with Good Ventures on co-funding, including advising a $1 million co-funding project with the Gates Foundation.
   • We have also done some preliminary work on a few other causes, which we will be writing about in the future.

   However, we have not been able to devote as much time to GiveWell Labs as we would have liked, and progress has accordingly been slower than anticipated. We have not yet identified any giving opportunities that we are ready to recommend (aside from the two grants mentioned above, both funded by Good Ventures).

Where we stand

We continue to feel our research has identified outstanding giving opportunities for individual donors, with adequate capacity (room for more funding in top charities) to absorb the level of funding that we expect in 2013, but we believe that room for improvement remains across the three broad areas we identified in 2012: continuing to find ways to subject our research to scrutiny and quality control, finding more outstanding giving opportunities according to our traditional criteria, and broadening our criteria via GiveWell Labs.

Of these three, we think the most urgent need is to make more progress on GiveWell Labs. Progress on that front in 2012 was much slower than hoped, due to a smaller allocation of staff time than intended. In order to make more progress on GiveWell Labs in the future, we may need to put less time (in the short term) into the other two goals, while hoping eventually to expand our staff capacity so that we can pursue all three effectively.

What we can do to improve

We plan to prioritize work on GiveWell Labs more highly in 2013, devoting more staff time to research on new causes than we did in 2012. We aren’t yet sure how we will be addressing the other areas of improvement discussed above; it depends heavily on how much capacity we are able to devote to GiveWell’s traditional work while making sure that we are moving forward significantly faster on GiveWell Labs. How to allocate capacity between these two arms of GiveWell is a major question for the coming year, to be discussed further in a future post.

Is it practical for donors to evaluate and use GiveWell’s research in the areas it has covered?

Where we stood as of Feb 2012

Last year, we wrote:

We feel that evidence of our credibility has substantially improved and is now fairly strong, and that this is part of the reason for our increased money moved.

Aside from revisiting our approach to external evaluation, we believe the main path to improvement is to continue improving our knowledge of giving opportunities and the depth of our research. We also plan to increase the frequency of research meetings and conference calls to help donors engage with our research.

Progress since Feb 2012

• We have significantly deepened our research on the charities and interventions we recommend most highly, leading us to be more confident in our recommendations this year than last year. For instance, we considered the evidence regarding the rate of decay of insecticide-treated bednets in far more depth than we had previously; also see Revisiting the case for insecticide-treated nets, Insecticide resistance and malaria control, Revisiting the case for developmental effects of deworming, New Cochrane review of the Effectiveness of Deworming.
• We have held more in-person research meetings and public conference calls (five, total) than in previous years.

Where we stand

We believe that our current level of conference calls and research meetings is adequate to allow highly engaged people who prefer not to read the full text of our reviews to access and respond to our research, but we feel that we generally lack mid-level content that allows people who are moderately engaged to understand the basics of our research and conclusions.

What we can do to improve

We plan to maintain or slightly expand our set of conference calls and in-person research meetings in 2013. We hope to eventually provide more mid-level content, but do not expect to prioritize doing so in 2013.

How much funding can GiveWell’s top-rated charities effectively absorb?

Where we stood as of Feb 2012

We believed that AMF could absorb up to $15 million, and SCI could absorb a few million dollars as well. We had not yet begun work on GiveWell Labs in earnest.

Progress since Feb 2012

We have identified one additional top charity–GiveDirectly–which has significant room for more funding.

We recommended grants totaling $1.1 million under the rubric of GiveWell Labs, but we do not have a standing set of opportunities available to individual donors in this category.

Where we stand

We continue to believe that AMF could absorb substantial funding this year, in the $15-20 million range, and also believe that GiveDirectly and SCI could absorb several million dollars.

Partially because we have had another a year of growth and now have a better sense of what growth to expect in our “money moved” figures, we believe that “room for more funding” in our current top charities is more than adequate for 2013.

Our progress on GiveWell Labs has been slower than expected to date, and we believe it could be a substantial amount of time before we will have substantial “room for more money moved” in this area.

What we can do to improve

We intend to continue to look for more outstanding giving opportunities. This need is driven by the potential to find opportunities that may be better than our current offerings, rather than a fear that our current offerings will be exhausted in the short term. Currently, we see the promising avenues to finding these opportunities as:

• Deeply investigating the areas we have identified as most promising (largely within global health and nutrition).
• Moving forward on GiveWell Labs.

Broadly, this is a continuation of our strategy from 2012, though we expect to place more emphasis on the GiveWell Labs prong in 2013 than we did in 2012.