This is the fourth post (of six) we’re planning to make focused on our self-evaluation and future plans.

This post answers a set of critical questions for GiveWell stakeholders. The questions are the same as last year’s.

Is GiveWell’s research process “robust,” i.e., can it be continued and maintained without relying on the co-Founders?

Where we stood as of Feb 2011

We had two full-time, non-co-founder employees (analysts), a part-time employee, and plans to hire another in the summer. We wrote that, in total, we expected to have 3.5 analysts by the Fall of 2011.

Progress since Feb 2011

By the summer of 2011, we had 3 full-time analysts and two temporary summer analysts. One of our full-time analysts left in December 2011; we recently hired one of our former summer analysts, bringing us back to 3 full-time analysts.

The analysts have been with us different levels of time, but as they have been at GiveWell longer, each has continued to take on more responsibility.

Where we stand as of February 2012

We currently have 3 full-time analysts, and have made an offer to an analyst who will start in July, which would bring GiveWell to 4 full-time analysts. We continue to focus on recruiting and hope to reach 6 full-time analysts (8 total employees) summer 2012.

Analysts take the lead on most charity investigations; co-founders may provide basic guidance and sign off on work before it is published. GiveWell Labs, because of its experimental nature, will be led for the time being by co-founders.

What we can do to improve

• We continue to experience turnover. Two analysts who worked with us in 2011 have since left GiveWell. We don’t believe that turnover is indicative of a problem – GiveWell is a unique environment that fits some people and not others and we would not be surprised if turnover remains relatively high in the near future. We have tried to address turnover by improving our hiring process to select for the characteristics most likely to lead to success at GiveWell.
• In the past, we have exclusively hired “generalists” i.e., young, largely college-graduates who work on a little bit of everything at GiveWell – e.g., charity reviews, speaking with donors, answering emails, in-depth literature reviews. In the coming year, we plan on putting more thought into hiring people for more specialized roles, such as administration (we now have enough administrative work for a full-time hire) and literature reviews.

Does GiveWell present its research in a way that is likely to be persuasive and impactful (i.e., is GiveWell succeeding at “packaging” its research)?

Where we stood as of Feb 2011

We wrote, “We’re currently satisfied with the presentation of our content and don’t plan on emphasizing this goal in the near future.”

Progress since Feb 2011

None. This was not a priority over the past 12 months.

Where we stand as of February 2012

As traffic to our website has increased over the past 12 months, we would guess that the importance of better packaging our research has risen. In particular, we feel our site is poorly suited to donors who want to spend more than a few minutes but less than an hour on our site. (We have designed the site to make quick action easy and to provide significant depth, but we have no “middle level” of depth for gaining some information relatively quickly.)

What we can do to improve

We have a list of ideas for how to better package our research, and we may prioritize this in 2012. In particular, we are considering putting more work into a “middle level of detail” as described above.

This is the third post (of six) we’re planning to make focused on our self-evaluation and future plans.

This post answers a set of critical questions about the state of GiveWell as a donor resource. The questions are the same as last year’s.

Does GiveWell provide quality research that highlights truly outstanding charities in the areas it has covered?

Where we stood as of Feb 2011

• Internally, we were satisfied with the quality of our research as compared to other options for donors.
• We planned to complete a new round of research focused on international aid to find additional top charities.
• We planned to complete regular updates for VillageReach, our top-rated charity in 2009 and 2010 and the first charity to which we moved significant (now more than $2 million) funds.
• We felt a need for more substantial external checks on our research. In the previous year, we had several reviews completed, but we believed additional reviews were necessary.
• Our research process was constrained because of our inability to offer project-based funding.

Progress since Feb 2011

In Feb 2011, we wrote that we hoped for

More intensive examination of our top-rated charities (the ones that attract the lion’s share of our “money moved”), including in-person site visits, continual updates on room for more funding, and conversations with major funders who have agreed – or declined – to fund them.

In 2011, we did all of these things.

• The level of vetting to which we subject our top charities has increased significantly.
  • We visited all three leading contenders for our highest ratings in October before giving them our top rating. We intend to continue this and visit charities before we give them our top ratings.
  • We conducted extensive reviews of the cost-effectiveness and evidence of effectiveness for the programs run by our top charities, which went significantly beyond our independent assessments of research in previous years. (See our writeups on ITNs and deworming.)
• We have completed regular updates on VillageReach’s progress.
• We have received several additional external reviews of our research, but our attempts to significantly expand this process were not successful. We intend to revisit this in 2012.
• In addition, we launched GiveWell Labs, a new arm of our research process that will be open to any giving opportunity, no matter what form and what sector, in the hopes of improving our ability to find great giving opportunities.

As our influence has increased, our ability to get access to relevant people such as charity representatives, scholars, and major funders has improved. (For example, see our investigation of targeted vs. universal coverage for insecticide-treated nets.) This, along with growth in our team, have improved our ability to do in-depth research efficiently.

Where we stand

We feel that our current research is high-quality and up-to-date. We are not satisfied with our total “room for money moved.” We estimate that our top charities have roughly $15-20 million in available room for more funding, which is substantially more than we have directed to them, but not necessarily “enough” if our influence were to continue growing rapidly.

We also feel there are multiple areas that could offer outstanding opportunities that we have not yet researched as thoroughly as we could (particularly in the areas of nutrition, vaccinations, neglected tropical disease control, tuberculosis control, and research and development).

We also continue to see room for improvement in our coverage of top charities.

• We’d like to continue to increase our qualitative checks on top charities, particularly conversations with those who have funded them or chosen not to.
• We remain unsatisfied with the degree to which our research is “vetted.” It still seems to us that we could make a substantial mistake or error in judgment, with too high a probability that it would remain unnoticed. We feel that this is much less true today than ever before, because (a) our staff is larger and we subject important pages to multiple checks from different people; (b) our research draws more attention, including from donors who are giving large amounts and vetting our work fairly closely. We feel that the degree to which our work is “vetted” will grow as our overall influence and prominence grows, though putting more effort into formal external reviews help as well.

What we can do to improve

• Revisit the goal of having our work subjected to formal, consistent, credible external review.
• Continue to look for more outstanding giving opportunities for individual donors, particularly (a) in the areas we have identified as most promising (b) through GiveWell Labs.

This is the first post (of six) we’re planning to make focused on our self-evaluation and future plans.

As in past years, we’re going to be posting our annual self-evaluation and plan as a series of blog posts. This post summarizes what changed for GiveWell in 2011 and what it means for the future. Future posts will elaborate.

For us, the major developments of 2011 were:

• New contacts with major donors and the launch of GiveWell Labs. Prior to 2011, we had only one contact whom we would have classified as a “megadonor,” i.e., someone who seemed to have a reasonable chance of making extremely large gifts (in the range of $1 million or more) in the reasonably near future if we could generate good enough giving opportunities. In 2011, we made contact with several more, including Good Ventures.

  We realized that these contacts made it both more possible and more important to produce research of use to major philanthropists. We therefore launched GiveWell Labs, a new arm of our research process that will be open to any giving opportunity, no matter what form and what sector. GiveWell Labs made little progress in 2011 aside from being launched, but we expect it to be a major priority for 2012.

• Our money moved grew significantly, hitting $5 million for 2011. This was partly a function of the major donors mentioned above, but partly a function of continuing to see strong growth in money moved from smaller donors. Excluding GiveWell Labs and Good Ventures, our money moved figure roughly doubled over last year, which was consistent with the growth we’ve seen in past years.
• Our general level of “access” improved significantly. By “access,” we mean the ability to get relevant people (including charity staff, foundation staff, and academics) to engage with us and discuss relevant issues. This makes it possible to learn about relevant issues more quickly, improving both the quality and speed of our research, and reduces the problem of missing potential top charities because of their reluctance to engage with us.

  We believe that our access improved partly because of our relatively strong “money moved” figure for 2010, which we highlighted on our new For Charities page, and partly because of our improved reputation and network. Good Ventures has been very helpful on this front.

• Our needs (and opportunities) for more staff are growing. Over the summer of 2011, we had 7 total staff (2 of them temporary) and still felt that we could benefit from more capacity. We currently have 5 total staff. Prior to 2011, we had never had more than 4 total staff at one time.

  As we become more systematic and thorough, we see more opportunities to improve our research by hiring; GiveWell Labs may introduce the need for more capacity as well. And the increased level of attention we get has increased administrative work.
  Recruiting will be a major priority for 2012.

• We were successful in raising more operating funding. Fundraising was our #2 priority for 2011. We met our goal, with some help from both institutions and individuals. We balanced our budget for 2011 and project a balanced budget for 2012; if we succeed in making more good hires, we will need to raise more to cover our costs for 2013.
• We identified two strong top charities and intensified our focus on global health and nutrition. Identifying top charities was what took the most effort in 2011. We believe it’s no accident that all of the strongest charities we’ve found so far are within the broad area of global health and nutrition. We will be focusing on this area intensively in 2012; we believe that deeply investigating a set of priority interventions (and the organizations that carry them out) is the most promising route to finding more outstanding giving opportunities.
• We drew substantial attention for our work on disaster relief (particularly relating to the Japan crisis in March) and errors in World Health Organization cost-effectiveness figures. We believe that these contributed to our improvements in money moved, website traffic, and general reputation. We will continue to maintain our disaster relief research and will continue to deeply investigate research questions that are important to us.

Overall, it was a very encouraging year. Our work, our reputation and our influence all improved significantly, and we see substantial room for more improvement. We believe GiveWell now has enough impact to justify its operating expenses, and hope to have much more impact in the future.

Of course, we also made plenty of mistakes in 2011, and we’ve recently updated our shortcomings log to reflect them.