This is the second post (of six) we’re planning to make focused on our self-evaluation and future plans.
This post reviews last year’s progress on our traditional work of finding and recommending evidence-based, thoroughly vetted charities that serve the global poor. It has two parts. First, we look back at the plans we laid out in early 2013 and compare our progress against them. Then, we reflect on the state of our traditional work and what we learned about it in 2013.
Note: in past years, we answered a series of questions about GiveWell’s progress as a donor resource and as a project. Considering that template this year, we feel that those questions (which we first used in our 2009 annual review) have become stale and no longer represent the best approach to evaluating our progress. Instead, we quote from last year’s writeup on plans and goals for the year, and compare to our actual progress during the year.
Summary
At the beginning of 2013, we felt that our traditional research and recommendations were relatively strong and would remain a valuable resource with limited effort put into improving them. We therefore aimed to limit senior staff time (note 1) on it to enable progress on GiveWell Labs.
We ultimately spent significantly more senior staff time on our traditional work than we had anticipated, which we felt was necessary to maintain the quality of our research, though we did achieve our big-picture goal of allocating more senior staff time to GiveWell Labs than previously and making more progress on that front.
We kept our research up to date, including major updates on our top charities, and added a new top charity. We also hired 6 additional research analysts, which should enable us to increase the capacity devoted to our traditional work in the future. Growth in web traffic and donations remained strong (details in our forthcoming metrics post).
Media attention to our traditional work has increased, and we now see it as both more important and more challenging to sustain this work in the future. We expect substantially more production next year, due to our increased capacity and better success getting engagement from charities (more in the next post).
The items that we consider essential for our “traditional” work are:
- Continuing to do charity updates … on our existing top charities.
- Reviewing any charity we come across that looks like it has a substantial chance of meeting our traditional criteria as well as, or better than, our current #1 charity (which would require not only that the charity itself has outstanding transparency, but also that the intervention it works on has an outstanding academic evidence base). We have created an application page for charities that believe they can meet these criteria.
- Hiring. As mentioned previously, we believe our process has reached a point where we ought to be able to hire, train and manage people to carry it out with substantially reduced involvement from senior staff. We are currently hiring for the Research Associate role, and if we could find strong Research Associates we would be able to be more thorough in our traditional work at little cost to GiveWell Labs.
Below, we review each, in turn.
- Charity updates. We had relatively long histories with each of our 3 top charities, and expected that periodic conversations and review of documents by non-senior staff would allow us to stay up to date. These updates became significantly more involved than we anticipated because there were major developments with each of our top 3 charities (we had not expected any developments that would meaningfully affect our rankings), and these took significant senior staff time to investigate and write up.
- The Against Malaria Foundation (AMF), our #1-ranked charity in 2011 and 2012, did not finalize a distribution to which it could allocate the bulk of funds it had on hand. This led us to remove AMF from our recommended charities list due to lack of room for more funding (details in this post).
- We learned new information about the monitoring and evaluation that had provided a substantial part of the case for the Schistosomiasis Control Initiative (SCI), an organization we have recommended since 2011 (details in this post). SCI also significantly improved its communication with us, addressing many of the problems we noted earlier in the year.
- Researchers released results from a randomized controlled trial of GiveDirectly’s program in late November, providing a key piece of evidence about GiveDirectly’s progarm’s impact (details in this post). This update, while substantial, did not require significant senior staff time.
- New charity reviews. At the beginning of the year, we did not expect to find new top charities in 2013. In April, we began evaluating the Deworm the World Initiative (led by Evidence Action). Our familiarity with deworming and our existing intervention report on deworming made this investigation easier than it otherwise might have been; the fact that DtWI was the first technical assistance/advocacy organization we had reviewed made this analysis more complicated. In all, this investigation took about as long as we would have expected given that it was a new top charity. This investigation required significant senior staff time, but a substantial portion of that time involved training Timothy Telleen-Lawton, a new hire whom we anticipate requiring reduced oversight in the future. Timothy led the investigation of DtWI and did the bulk of the work on it.
- Hiring. We made major progress on this goal. In addition to Tim, we hired five new Research Analysts, increasing our capacity for (a) conversation notes processing; (b) “vetting”, i.e., carefully reviewing pages pre-publication to identify any errors; (c) completing intervention reports, something that some of the new hires proved able to do with little involvement from senior staff. We completed new intervention reports on water quality and vitamin A supplementation (forthcoming).
- Donation processing. While we did not articulate this as a goal for 2013, we significantly improved our donation processing system over the course of the year. Processing donations and metrics from 2012’s giving season had taken approximately 3 full-time months. The improvements we made last year allowed us to process more than twice as many donations using less than half the time we had previously. (We also believe that these changes will scale well as we continue to grow.) These improvements were led by Natalie Crispin and required almost no senior staff time.
Three major observations from 2013 will inform our plans going forward.
- Significant growth in attention to/interest in our traditional work. Last year, we saw significant growth in interest in our traditional work from media and donors. At the beginning of last year, we would have guessed that growth would slow, yet we saw significantly more media attention than we had in past years, which contributed to our more than doubling the number of donors giving to our top charities and significant web traffic growth. Both the number of donors (as well as the total amount given by <$5,000 donors) and web traffic grew at a faster rate last year than they had in previous years; growth in donations from larger donors slowed relative to last year’s growth, though this trend is less robust because of the smaller numbers of people involved. (We will share full details on these figures in our forthcoming metrics post.)
- The challenge of maintaining the quality of our research and recommendations. In early 2013, we expected to spend relatively little time maintaining the quality of our research and recommendations. As discussed above, this did not work out as we had planned, and we no longer believe that we can easily maintain the quality of our research and recommendations without the engagement of highly skilled/trained staff. More broadly, we continued to learn about (and write about) the fundamental difficulty of the “giving as consumption” model underlying our traditional work, and believe that continuing to provide a strong resource using this model could take quite a bit of work and ingenuity going forward.
- Increased interest from charities in receiving our recommendations. As our money moved has grown, we have seen increased interest from charities in engaging with us and our process. This willingness to engage has grown consistently over the course of GiveWell’s history, but we now believe we have reached a level of money moved where most organizations will seriously consider engaging with us, as opposed to just ignoring us. This should enable us to choose the organizations that we think are most promising; encourage them to apply; and if they choose not to, understand what led them to decline (e.g., insufficient staff time, limited monitoring data, etc.). In the past, charities chose not to participate for reasons we often didn’t fully understand and we hope that this has now changed. This should eventually improve the robustness of our top charities list.
Note 1: In this post, senior staff refers to Elie, Holden, and Alexander.