This is the third post (of five) we’re planning to make focused on our self-evaluation and future plans.

This post answers a set of critical questions for GiveWell stakeholders. The questions are taken from our 2010 list of questions. For each question, we discuss

• Where we stood as of our previous self-evaluation and plan a year ago.
• Progress since then.
• Where we stand (compared to where we eventually hope to be).
• What we can do to improve from here.

Is GiveWell’s research process “robust,” i.e., can it be continued and maintained without relying on the co-Founders?

Where we stood as of Feb 2010

We had one full-time employee other than the co-Founders (Natalie Stone). We felt that she was capable of maintaining our international aid report with little oversight, while covering other causes would require heavy involvement from co-Founders.

Progress since Feb 2010

• Natalie Stone has now been with GiveWell for about 18 months. She has not only worked on maintaining and expanding the international aid report, but also did the bulk of the work for a new cause – microfinance – and has also taken on miscellaneous duties such as responding to emails sent to info@givewell.org
• We have also made another full-time hire, Simon Knutsson, who started as a volunteer, became a 30-40-hour-per-week contractor in March of 2010 and joined as a full-time employee in January 2011. He has done the bulk of the work for the U.S. Equality of Opportunity report.
• We have systemized the process of keeping our research up to date. This process can now be done partly by junior staff and partly by volunteers, with very little participation needed from Co-Founders.
• Because we now have useful work for volunteers to do, we’re able to use volunteer work as a way to evaluate potential hires, leading to two more new hires:
  • We hired one volunteer (Stephanie Wykstra) part-time; she has been working about 20 hours per week for us and plans on working full-time for us over the summer. She did the bulk of the work for our disaster relief report.
  • Another former volunteer (Alexander Berger) plans on coming to work full-time for us after he graduates college.

All in all, by this fall we expect to have 3.5 employees apart from the co-Founders. All will focus primarily on maintaining and expanding our existing research (and in particular, finding and evaluating particularly promising charities), a top priority for us, but all will experiment with and be available for other work as well, including work on researching new causes.

Where we stand

We have a promising set of junior staff and lots of valuable work for them to do, keeping our research up to date and thorough. These staff have promise as people who might eventually be able to run the organization entirely, but we don’t feel they’re at that point yet.

What we can do to improve

We plan on continuing to expand the scope of the work we assign to junior staff, as well as continuing to work with volunteers who may eventually become staff. This will take a good deal of time in terms of training/management overhead.

Does GiveWell present its research in a way that is likely to be persuasive and impactful (i.e., is GiveWell succeeding at “packaging” its research)?

Where we stood as of Feb 2010

We felt that our research was presented with sufficient clarity and good organization, though without emotional persuasiveness. We sought to add more external evidence of credibility and to better integrate the blog and website.

Progress since Feb 2010

As discussed in the previous post, we have added some external evidence of credibility to our website, and established a basic process for adding more. We have not made other progress on this metric; we do not consider improving the emotional persuasiveness of our work to be a priority.

Where we stand

We’re currently satisfied with the presentation of our content and don’t plan on emphasizing this goal in the near future.

What we can do to improve

At some point, more attention to the presentation of our research may be useful for broadening our audience. Right now, though, our impact appears to be growing rapidly just from our niche audience, and continuing to serve this audience well presents major challenges that are higher priorities.

Does GiveWell reach a lot of potential customers (i.e., is GiveWell succeeding at “marketing” its research)?

Where we stood as of Feb 2010; progress since Feb 2010; where we stand

As detailed previously,

• We tracked over $1.5 million in donations to top charities in 2010, compared to just over $1 million in 2009.
• Our website traffic nearly doubled from 2009 to 2010, and donations through the website nearly tripled. Our overall increase in money moved appears to be driven mostly by (a) a gain of $200,000 in six-figure donations; (b) new donors, largely acquired via search engine traffic and the outreach of Peter Singer.
• Our growth in online donations to recommended charities was significantly faster than that of the more established online donor resources (Charity Navigator and GuideStar); our total online donations remain lower than these resources’, but are now in the same ballpark.

We have also formed a content partnership with GuideStar.

The amount of money we’re influencing is now quite significant to our top charities. As discussed previously, we now have significant concerns about “room for more money moved.”

Ultimately, we have a loose goal of reaching the point where our “money moved” is equal to, at a minimum, 9x our operating expenses. We see our operating expenses as being between $400,000-$500,000 in a steady state (details forthcoming) so this would require seeing another 150-200% growth in our money moved.

What we can do to improve

We have many ideas to continue to improve our reach. At this point we feel the most promising are:

• Have more discussions with, and surveys of, those who use our research to decide where to give, in order to understand our audience, deepen our relationships, and ultimately find more such donors. A professional fundraiser could help significantly with this.
• Hold fundraising events in order to make it easier for our existing supporters to bring in new supporters. We held a small event in December of 2010 focusing on VillageReach; the cost of the event was about $3,000 and it resulted in a total of $17,800 in donations to VillageReach and $7,500 in donations to GiveWell.
• Public relations / press targeting projects aiming for earned media.
• More active pursuit of people we consider highly influential among our target audience (the better we understand our audience, the better our ability to seek out such people).
• Website improvements, including
  • Improving our search engine optimization. (Most of our gains in traffic in 2010 came via search.)
  • Minimizing the processing fees on donations.
  • Better encouraging donors to share their actions and recommendations with friends (using email and social media).
  • Offering donors an easy way to get reminded to donate in December. (In 2010 over 25% of the money donated through our website was on the very last day of the year.)
  • Improve our ability to track website users and understand how they found us.

Is GiveWell a healthy organization with an active Board, staff in appropriate roles, appropriate policies and procedures, etc.?

Where we stood as of Feb 2010

We were happy with our Board and other aspects of our organization.

Progress since Feb 2010

We remain happy with our Board and policies and procedures. However, we now have a significant need to raise more operating funds. In a nutshell,

• Our improved ability to recruit and employ junior staff has resulted in increased expenses, both present and future.
• We have lost two major sources of revenue (a major donor changed careers and the Hewlett Foundation lowered the size of its support).

Details will be forthcoming in a financial analysis document.

Where we stand

We have a significant need for more operating support and intend to make this a major priority for 2011.

What we can do to improve

We aim to raise operating support by

• Applying for support from major institutional funders.
• Having discussions with people who have a substantial history of being close to our project, including Board members and major “customer donors” (i.e., donors who have used our research to decide where to give), about the possibility of their supporting GiveWell directly.

We plan to continue to avoid soliciting funds from the public at large. We wish to avoid “competing with” our recommended charities for funding, and feel our credibility would be hindered if we were asking for money ourselves.

What is GiveWell’s overall impact, particularly in terms of donations influenced? Does it justify the expense of running GiveWell?

Where we stood as of Feb 2010

In 2009, we tracked just over $1 million in donations to our top charities, though over 30% of this $1 million came from pledges and restricted donations that had been made in 2008. We had a reasonably strong presence in our sector as members of the Alliance for Effective Social Investing and GuideStar Exchange Advisory Board.

Progress since Feb 2010

As discussed above, our website traffic and money moved have grown significantly and we now provide significant funding to our top-rated organizations.

We remain members of the Alliance for Effective Social Investing and GuideStar Exchange Advisory Board; we have also joined two other collaborative groups in our sector (Markets for Good and Market Development Working Group of the Social Impact Exchange) and begun a content partnership with GuideStar.

Where we stand

At this point we can point to significant impact for our top charities; the impact they, in turn, have with our additional funds remains to be seen, but we are well-positioned to evaluate it over time.

That said, our “money moved” is still not at the point we would eventually like to see, and we still do not know enough about our impact because we have little information on where donors would have given without our research. We are currently investigating the latter.

What we can do to improve

• Survey donors on where they would have given if not for our research, to get a better sense of our overall impact. (Currently in progress.)
• Pursue the previously outlined strategies for increasing both our “money moved” and “room for more money moved” (both are necessary).
• Pursue long-term sustainability of the organization – in particular, continue developing junior staff and raise the operating funding needed to maintain these staff.

This is the second post (of five) we’re planning to make focused on our self-evaluation and future plans.

This post answers a set of critical questions about the state of GiveWell as a donor resource. The questions are taken from our 2010 list of questions, with one change: “Has GiveWell covered enough areas to be useful?” is replaced by “How much funding can GiveWell’s top-rated charities effectively absorb?” for the reasons outlined at our previous post.

For each question, we discuss

• Where we stood as of our previous self-evaluation and plan a year ago.
• Progress since then.
• Where we stand (compared to where we eventually hope to be).
• What we can do to improve from here.

Does GiveWell provide quality research that highlights truly outstanding charities in the areas it has covered?

Where we stood as of Feb 2010

• Internally, we were satisfied with the quality of our research as compared to other options for donors.
• We planned to conduct one or more site visits to the developing world, in order to inform our work with some direct observation.
• We felt a need for more substantial external checks on our research, and planned to subject it to strong, critical scrutiny from people with substantial relevant experience and credentials.

Progress since Feb 2010

• We have made substantial progress on seeing charities’ work up close.
  • In February of 2010, I visited VillageReach in Mozambique and Small Enterprise Foundation in South Africa and posted notes and multimedia to our website.
  • In August of 2010, GiveWell’s full-time staff (all three of us, plus a contractor who has sinced joined GiveWell as an employee) moved to Mumbai for 3 months. While there, we did site visits to 20 organizations (and visited some multiple times). We will be posting notes and multimedia to our website once we have resolved with the charities what we may and may not publish.
• We have made substantial progress on subjecting our research to strong, critical scrutiny from people with substantial relevant experience and credentials.
  • We have formalized the process for keeping our research up to date, crucial for soliciting formal feedback. (Last year, we had to update our year-old report before any work on external feedback was feasible.)
  • We have formalized the process for providing feedback on the different aspects of our research, and have gotten at least one submission for each category of assignment – see our page on external reviews.
• We have also improved the quality of our research in other ways.
  • We have systemized the process of keeping our research up to date, and updated our international aid report in mid-2010. This process can now be done partly by junior staff and partly by volunteers, with very little participation needed from Co-Founders.
  • We have intensified our focus on room for more funding and gotten particularly concrete answers on this question from two of our top charities: VillageReach and Nurse-Family Partnership.

Where we stand

We feel that our current research is high-quality and up-to-date and that the first set of external reviews reflects this. We are not fully satisfied with the number and credibility of these external reviews and hope to secure more of them.

What we can do to improve

• Pursue more external reviews of our research.
• More intensive examination of our top-rated charities (the ones that attract the lion’s share of our “money moved”), including in-person site visits, continual updates on room for more funding, and conversations with major funders who have agreed – or declined – to fund them.

Is it practical for donors to evaluate and use GiveWell’s research in the areas it has covered?

Where we stood as of Feb 2010

We were satisfied with the organization of our website, but felt that donors’ options for assessing our credibility were insufficient. We wrote, “We feel that we should have a single, easy-to-find roundup of available information on the credibility of our research.”

Progress since Feb 2010

• We have created a process for, and secured early examples of, formal external reviews of our research.
• We have created a page with testimonials from scholars, nonprofit professionals, and donors.

Where we stand

While we have created the basic process and template for both reviews and testimonials, and have some basic evidence of our credibility now easily available, we feel that the content of both pages could be much stronger.

What we can do to improve

Over the next year we intend to secure the most impressive testimonials we can and to significantly increase the number of external reviews by people with clearly relevant credentials. We predict a substantial improvement in both of these areas compared to the content that is there now.

How much funding can GiveWell’s top-rated charities effectively absorb?

Where we stood as of Feb 2010

• We did not consider “top-rated charities’ room for more funding” to be a key metric, and did consider “causes covered” to be a key metric. (More on our change of direction.) For future causes, we were considering sub-causes in international aid (disaster relief, charities focused on orphans and vulnerable children, and microfinance); U.S. equality of opportunity; disease research; and environmental issues.
• As of December 2009, we estimated $2.5 million in room for more funding for VillageReach.
• We estimated $20 million in room for more funding for Stop Tuberculosis Partnership, which then held our top “three star” rating (we changed both our ratings system and Stop TB’s classification in November 2010).
• We did not have strong or quantified information on other recommended organizations’ room for more funding.

Progress since Feb 2010

• We published research on several new causes: microfinance, U.S. equality of opportunity, and disaster relief. We also got new information that has led us to de-emphasize research on new causes.
• We got better information on Nurse-Family Partnership’s room for more funding.
• We have directed just over $1.5 million to top-rated charities, including just over $1.1 million to VillageReach. (Details)

Where we stand

Our gold medal charities currently have relatively little room for more funding. VillageReach had a $4 million projected need over six years as of December; its need now is probably under $3 million (we will soon be discussing with them and posting another update). Nurse-Family Partnership has a long-term but not short-term need for more funding.

We do not have very good information on our silver medal charities’ room for more funding, though this group involves some large charities (Stop Tuberculosis Partnership, KIPP) that can likely absorb tens of millions of dollars of funding.

(Read about our definitions of “gold medal” and “silver medal” charities.)

What we can do to improve

• Push for clearer information on room for more funding from our silver medal charities.
• Find more charities that we are comfortable giving the gold medal distinction. This is a top priority for the coming year.

As we did last year, we’re going to be posting our annual self-evaluation and plan as a series of blog posts.

Our self-evaluation starts with a set of critical questions about GiveWell, intended to examine how well we’re accomplishing our high-level goals. For the most part, our high-level goals are the same as they were last year, and thus our questions for GiveWell are the same as the questions we laid out last year. However, there is one major change in how we see the goals of GiveWell: we no longer assign high importance to the number of causes covered / number of options we provide to donors. Instead, we just want to focus on finding outstanding charities that donors can be confident in, and closing those charities’ funding gaps to the extent that we can.

---

We see charity evaluation groups as falling into the following possible categories:

1. Groups that rate as many charities as possible. Donors come to them already having a particular charity in mind to give to, and search for that charity.
2. Groups that suggest charities for as many causes as possible. Donors come to them knowing what sort of cause they want to support (U.S. education, global health, etc.) but not which charity, and get a recommendation.
3. Groups that simply focus on finding outstanding charities. Donors come to them looking for outstanding giving opportunities (they are often issue-agnostic).

We started GiveWell as issue-agnostic donors looking for the best giving opportunities we could find, and we have always primarily been interested in #3. We’ve never had a serious interest in #1 above – distinguishing between “worst,” “bad,” “good,” and “better” is too much of a distraction when what we care about is “best.” But for most of our history, we’ve seen ourselves as possibly being on a trajectory toward becoming #2 in addition to #3. We’ve talked about covering a broad array of causes to interest as many donors as possible, thus increasing our influence and visibility. Many of those who have criticized us have focused on the small number of causes we’ve covered, and expressed the hope that we will eventually cover many more.

A couple of things have made us rethink this goal:

1. We see it as urgent, and difficult, to find more “gold medal” charities than we have so far.

When the 2010 holiday season was approaching, and we started thinking about our strategy for outreach, we realized that we only had one charity we could personally feel really good about aggressively raising money for. While we think all our recommended charities stand well above other donors’ options, it was only VillageReach for which I felt I could sit down with someone face-to-face about and say, “Give as much as you can to this one.”

Why? Because not only does VillageReach have outstanding evidence of effectiveness; it has outstanding bang-for-the-buck (in absolute terms, not just relative terms) and most importantly, it has a concrete plan for additional funding.

We personally don’t like raising money for groups that we can’t say all these things about. We’re happy to provide our recommendations for someone looking for the best microfinance charity, but we can’t honestly say that we think dollars given to it are accomplishing as much good as possible.

VillageReach had a “stretch” fundraising target for the year of $1.5 million. That gap has now been more than closed. VillageReach needs a total of $4.4 million for its multi-year project, so we are happy to leave it as our top-rated charity for now, but if our growth continues, the entire expansion could be funded by the end of the year. Thus, we are starting to get dangerously close to the point where the number of dollars we influence exceeds the number of dollars we know how to allocate very effectively.

We see this as an urgent situation. It would be a major problem for GiveWell if we essentially had more demand for our research (i.e., donors interested in following our recommendations) than supply (i.e., charities able to absorb this funding effectively). Thus, one of our top priorities for 2011 is finding more “gold medal” charities that we can give a wholehearted recommendation. We feel it would be a major distraction – and mistake – to try to find the “best of the bunch” within causes where there are no groups that really shine by our criteria.

2. The benefits of covering extra causes don’t seem very large.

The argument we’d always used for covering more causes was that covering more causes means appealing to more donors. In theory, this is true, but for the specific audience we seem to be attracting, the value of breadth seems surprisingly low.

The following charts show the percentage of our 2010 “money moved” (donations given to our recommended charities as a result of our research, which we track in a variety of ways) to different categories of charity. A future post will go into more detail on how we track “money moved” and what the figures were for 2010.

Note that VillageReach is our top-rated charity; Against Malaria Foundation is our only recommended charity that allows donors to get tax deductions in a variety of countries outside the U.S. (Canada, Australia and more); Nurse-Family Partnership (NFP) is our top-rated U.S. charity and was featured in a Nicholas Kristof column (we count donors coming from that column as “money moved” because Mr. Kristof has informed us that he included NFP in the column on our recommendation).

Bottom line – it looks like the overwhelming majority of our donors follow our recommendations regarding the promising cause and the most promising organization.

Even our top-rated U.S. charity didn’t attract nearly as much funding from our donors as our global health charities did. Meanwhile, the work we’ve put into accommodating donors interested in narrower causes – education and microfinance – has barely registered in terms of increased “money moved.”

And education and microfinance are still pretty broad, popular causes. I’d expect a “cancer research” or “global warming” recommendation to be pretty comparable to education and microfinance in terms of new money moved; I’d expect a narrower cause like “homelessness in New York City” to bring much less.

This doesn’t mean that all donors are issue-agnostic or global health fans. It means that our audience is. And we need to focus on serving our audience as well as possible.

Implications

We aren’t dropping the idea of “offering recommendations within lots of causes” entirely. We aren’t completely sure why our audience is the way it is, and recognize that over time the situation may change. Thus,

• We plan to maintain/update our existing research on all causes, including the causes above plus disaster relief (the newest addition).
• We plan to investigate more causes for the purpose of finding more outstanding charities. We will be writing up as much as we can of what we learn in these investigations, which may (along with our do-it-yourself evaluation guide) be helpful to donors interested in particular causes.
• We may provide donors a way to “lobby us” to cover more causes. I’ve thought about the idea of offering donors the chance to commit to give $X to our top-rated charity in a given cause, conditional on our covering that cause; if we got enough commitments for a given cause we would cover it.

However, we no longer consider “number of causes covered” a relevant metric for GiveWell and will be replacing it with “room for money moved,” i.e., the total amount of room for more funding of our very top-rated charities.

Some may be disappointed by this decision, seeing it as a retreat from the opportunity to be a resource for as many people as possible. But we’re not sure how much sense it really makes for an operation like ours to maximize its breadth. Giving decisions are deeply personal, and the kind of work we do is as well. We’ve mostly followed our personal values in making recommendations, and we’re attracting an audience that seems comfortable with these values.

This doesn’t mean that there’s no place for a GiveWell-type operation that focuses on (for example) U.S. causes; but perhaps we aren’t the people to run that operation. Perhaps it takes someone who truly believes that U.S. charities represent the best chance to help people (we do not) to do compelling research and attract the right audience for that cause. We’d be more than happy to see such a group spring up. For our part, we’re happy to remain a niche operation for a niche audience, as long as the niche is big enough. And it appears to be growing.