This is the second post (of five) we’re planning to make focused on our self-evaluation and future plans.

This post answers a set of critical questions about the state of GiveWell as a donor resource. The questions are taken from our 2010 list of questions, with one change: “Has GiveWell covered enough areas to be useful?” is replaced by “How much funding can GiveWell’s top-rated charities effectively absorb?” for the reasons outlined at our previous post.

For each question, we discuss

• Where we stood as of our previous self-evaluation and plan a year ago.
• Progress since then.
• Where we stand (compared to where we eventually hope to be).
• What we can do to improve from here.

Does GiveWell provide quality research that highlights truly outstanding charities in the areas it has covered?

Where we stood as of Feb 2010

• Internally, we were satisfied with the quality of our research as compared to other options for donors.

• We planned to conduct one or more site visits to the developing world, in order to inform our work with some direct observation.
• We felt a need for more substantial external checks on our research, and planned to subject it to strong, critical scrutiny from people with substantial relevant experience and credentials.

Progress since Feb 2010

• We have made substantial progress on seeing charities’ work up close.
  • In February of 2010, I visited VillageReach in Mozambique and Small Enterprise Foundation in South Africa and posted notes and multimedia to our website.
  • In August of 2010, GiveWell’s full-time staff (all three of us, plus a contractor who has sinced joined GiveWell as an employee) moved to Mumbai for 3 months. While there, we did site visits to 20 organizations (and visited some multiple times). We will be posting notes and multimedia to our website once we have resolved with the charities what we may and may not publish.
• We have made substantial progress on subjecting our research to strong, critical scrutiny from people with substantial relevant experience and credentials.
  • We have formalized the process for keeping our research up to date, crucial for soliciting formal feedback. (Last year, we had to update our year-old report before any work on external feedback was feasible.)
  • We have formalized the process for providing feedback on the different aspects of our research, and have gotten at least one submission for each category of assignment - see our page on external reviews.
• We have also improved the quality of our research in other ways.
  • We have systemized the process of keeping our research up to date, and updated our international aid report in mid-2010. This process can now be done partly by junior staff and partly by volunteers, with very little participation needed from Co-Founders.
  • We have intensified our focus on room for more funding and gotten particularly concrete answers on this question from two of our top charities: VillageReach and Nurse-Family Partnership.

Where we stand

We feel that our current research is high-quality and up-to-date and that the first set of external reviews reflects this. We are not fully satisfied with the number and credibility of these external reviews and hope to secure more of them.

What we can do to improve

• Pursue more external reviews of our research.
• More intensive examination of our top-rated charities (the ones that attract the lion’s share of our “money moved”), including in-person site visits, continual updates on room for more funding, and conversations with major funders who have agreed - or declined - to fund them.

Is it practical for donors to evaluate and use GiveWell’s research in the areas it has covered?

Where we stood as of Feb 2010

We were satisfied with the organization of our website, but felt that donors’ options for assessing our credibility were insufficient. We wrote, “We feel that we should have a single, easy-to-find roundup of available information on the credibility of our research.”

Progress since Feb 2010

• We have created a process for, and secured early examples of, formal external reviews of our research.
• We have created a page with testimonials from scholars, nonprofit professionals, and donors.

Where we stand

While we have created the basic process and template for both reviews and testimonials, and have some basic evidence of our credibility now easily available, we feel that the content of both pages could be much stronger.

What we can do to improve

Over the next year we intend to secure the most impressive testimonials we can and to significantly increase the number of external reviews by people with clearly relevant credentials. We predict a substantial improvement in both of these areas compared to the content that is there now.

How much funding can GiveWell’s top-rated charities effectively absorb?

Where we stood as of Feb 2010

• We did not consider “top-rated charities’ room for more funding” to be a key metric, and did consider “causes covered” to be a key metric. (More on our change of direction.) For future causes, we were considering sub-causes in international aid (disaster relief, charities focused on orphans and vulnerable children, and microfinance); U.S. equality of opportunity; disease research; and environmental issues.
• As of December 2009, we estimated $2.5 million in room for more funding for VillageReach.
• We estimated $20 million in room for more funding for Stop Tuberculosis Partnership, which then held our top “three star” rating (we changed both our ratings system and Stop TB’s classification in November 2010).
• We did not have strong or quantified information on other recommended organizations’ room for more funding.

Progress since Feb 2010

• We published research on several new causes: microfinance, U.S. equality of opportunity, and disaster relief. We also got new information that has led us to de-emphasize research on new causes.

• We got better information on Nurse-Family Partnership’s room for more funding.
• We have directed just over $1.5 million to top-rated charities, including just over $1.1 million to VillageReach. (Details)

Where we stand

Our gold medal charities currently have relatively little room for more funding. VillageReach had a $4 million projected need over six years as of December; its need now is probably under $3 million (we will soon be discussing with them and posting another update). Nurse-Family Partnership has a long-term but not short-term need for more funding.

We do not have very good information on our silver medal charities’ room for more funding, though this group involves some large charities (Stop Tuberculosis Partnership, KIPP) that can likely absorb tens of millions of dollars of funding.

(Read about our definitions of “gold medal” and “silver medal” charities.)

What we can do to improve

• Push for clearer information on room for more funding from our silver medal charities.
• Find more charities that we are comfortable giving the gold medal distinction. This is a top priority for the coming year.