The GiveWell Blog

Self-evaluation: GiveWell as a donor resource

(updated on: ) | by Elie

This is the third post (of six) we’re planning to make focused on our self-evaluation and future plans.

This post answers a set of critical questions about the state of GiveWell as a donor resource. The questions are the same as last year’s.

Does GiveWell provide quality research that highlights truly outstanding charities in the areas it has covered?

Where we stood as of Feb 2011

  • Internally, we were satisfied with the quality of our research as compared to other options for donors.
  • We planned to complete a new round of research focused on international aid to find additional top charities.
  • We planned to complete regular updates for VillageReach, our top-rated charity in 2009 and 2010 and the first charity to which we moved significant (now more than $2 million) funds.
  • We felt a need for more substantial external checks on our research. In the previous year, we had several reviews completed, but we believed additional reviews were necessary.
  • Our research process was constrained because of our inability to offer project-based funding.

Progress since Feb 2011

In Feb 2011, we wrote that we hoped for

More intensive examination of our top-rated charities (the ones that attract the lion’s share of our “money moved”), including in-person site visits, continual updates on room for more funding, and conversations with major funders who have agreed – or declined – to fund them.

In 2011, we did all of these things.

  • The level of vetting to which we subject our top charities has increased significantly.
    • We visited all three leading contenders for our highest ratings in October before giving them our top rating. We intend to continue this and visit charities before we give them our top ratings.
    • We conducted extensive reviews of the cost-effectiveness and evidence of effectiveness for the programs run by our top charities, which went significantly beyond our independent assessments of research in previous years. (See our writeups on ITNs and deworming.)
  • We have completed regular updates on VillageReach’s progress.
  • We have received several additional external reviews of our research, but our attempts to significantly expand this process were not successful. We intend to revisit this in 2012.
  • In addition, we launched GiveWell Labs, a new arm of our research process that will be open to any giving opportunity, no matter what form and what sector, in the hopes of improving our ability to find great giving opportunities.

As our influence has increased, our ability to get access to relevant people such as charity representatives, scholars, and major funders has improved. (For example, see our investigation of targeted vs. universal coverage for insecticide-treated nets.) This, along with growth in our team, have improved our ability to do in-depth research efficiently.

Where we stand

We feel that our current research is high-quality and up-to-date. We are not satisfied with our total “room for money moved.” We estimate that our top charities have roughly $15-20 million in available room for more funding, which is substantially more than we have directed to them, but not necessarily “enough” if our influence were to continue growing rapidly.

We also feel there are multiple areas that could offer outstanding opportunities that we have not yet researched as thoroughly as we could (particularly in the areas of nutrition, vaccinations, neglected tropical disease control, tuberculosis control, and research and development).

We also continue to see room for improvement in our coverage of top charities.

  • We’d like to continue to increase our qualitative checks on top charities, particularly conversations with those who have funded them or chosen not to.
  • We remain unsatisfied with the degree to which our research is “vetted.” It still seems to us that we could make a substantial mistake or error in judgment, with too high a probability that it would remain unnoticed. We feel that this is much less true today than ever before, because (a) our staff is larger and we subject important pages to multiple checks from different people; (b) our research draws more attention, including from donors who are giving large amounts and vetting our work fairly closely. We feel that the degree to which our work is “vetted” will grow as our overall influence and prominence grows, though putting more effort into formal external reviews help as well.

What we can do to improve

  • Revisit the goal of having our work subjected to formal, consistent, credible external review.
  • Continue to look for more outstanding giving opportunities for individual donors, particularly (a) in the areas we have identified as most promising (b) through GiveWell Labs.

GiveWell annual review: Details on GiveWell’s money moved and web traffic

(updated on: ) | by Elie

This is the second post (of six) focused on our self-evaluation and future plans.

This post lays out highlights from our metrics report for 2011. For more detail, see our full metrics report (PDF).

1. In 2011, GiveWell moved $5,285,992 to our recommended charities, a significant increase over past years.

2. While our #1-ranked charity received the most funding, many organizations received significant funding due to our recommendations. Most of our money moved went to AMF (our #1-ranked charity as of December 2011), which received over $2 million. SCI (our #2-ranked charity as of December 2011) and VillageReach (our top-rated charity through November 2011) both received more than $600,000. Three additional organizations received more than $75,000 as a result of our recommendation and one other organization received more than $50,000.

3. Web traffic continued to grow.

4. The main sources of increased traffic were Google search (AdWords and organic).

5. The vast majority of our money moved came from a relatively small set of donors giving very large gifts.

Good Ventures grants to our top charities as well as funding to committed GiveWell Labs accounted for approximately 1/3 of our total money moved ($1.75m / $5.3m). Excluding those funds entirely, 55 donors giving $10,000 or more accounted for 70% of our money moved. In the table below, and in the following analyses of our 55 largest donors, we exclude funding from Good Ventures to our top charities and funding committed to GiveWell Labs.

GiveWell’s progress in 2011

(updated on: ) | by Holden

This is the first post (of six) we’re planning to make focused on our self-evaluation and future plans.

As in past years, we’re going to be posting our annual self-evaluation and plan as a series of blog posts. This post summarizes what changed for GiveWell in 2011 and what it means for the future. Future posts will elaborate.

For us, the major developments of 2011 were:

  • New contacts with major donors and the launch of GiveWell Labs. Prior to 2011, we had only one contact whom we would have classified as a “megadonor,” i.e., someone who seemed to have a reasonable chance of making extremely large gifts (in the range of $1 million or more) in the reasonably near future if we could generate good enough giving opportunities. In 2011, we made contact with several more, including Good Ventures.

    We realized that these contacts made it both more possible and more important to produce research of use to major philanthropists. We therefore launched GiveWell Labs, a new arm of our research process that will be open to any giving opportunity, no matter what form and what sector. GiveWell Labs made little progress in 2011 aside from being launched, but we expect it to be a major priority for 2012.

  • Our money moved grew significantly, hitting $5 million for 2011. This was partly a function of the major donors mentioned above, but partly a function of continuing to see strong growth in money moved from smaller donors. Excluding GiveWell Labs and Good Ventures, our money moved figure roughly doubled over last year, which was consistent with the growth we’ve seen in past years.
  • Our general level of “access” improved significantly. By “access,” we mean the ability to get relevant people (including charity staff, foundation staff, and academics) to engage with us and discuss relevant issues. This makes it possible to learn about relevant issues more quickly, improving both the quality and speed of our research, and reduces the problem of missing potential top charities because of their reluctance to engage with us.

    We believe that our access improved partly because of our relatively strong “money moved” figure for 2010, which we highlighted on our new For Charities page, and partly because of our improved reputation and network. Good Ventures has been very helpful on this front.

  • Our needs (and opportunities) for more staff are growing. Over the summer of 2011, we had 7 total staff (2 of them temporary) and still felt that we could benefit from more capacity. We currently have 5 total staff. Prior to 2011, we had never had more than 4 total staff at one time.

    As we become more systematic and thorough, we see more opportunities to improve our research by hiring; GiveWell Labs may introduce the need for more capacity as well. And the increased level of attention we get has increased administrative work.
    Recruiting will be a major priority for 2012.

  • We were successful in raising more operating funding. Fundraising was our #2 priority for 2011. We met our goal, with some help from both institutions and individuals. We balanced our budget for 2011 and project a balanced budget for 2012; if we succeed in making more good hires, we will need to raise more to cover our costs for 2013.
  • We identified two strong top charities and intensified our focus on global health and nutrition. Identifying top charities was what took the most effort in 2011. We believe it’s no accident that all of the strongest charities we’ve found so far are within the broad area of global health and nutrition. We will be focusing on this area intensively in 2012; we believe that deeply investigating a set of priority interventions (and the organizations that carry them out) is the most promising route to finding more outstanding giving opportunities.
  • We drew substantial attention for our work on disaster relief (particularly relating to the Japan crisis in March) and errors in World Health Organization cost-effectiveness figures. We believe that these contributed to our improvements in money moved, website traffic, and general reputation. We will continue to maintain our disaster relief research and will continue to deeply investigate research questions that are important to us.

Overall, it was a very encouraging year. Our work, our reputation and our influence all improved significantly, and we see substantial room for more improvement. We believe GiveWell now has enough impact to justify its operating expenses, and hope to have much more impact in the future.

Of course, we also made plenty of mistakes in 2011, and we’ve recently updated our shortcomings log to reflect them.

Accountability in philanthropy

(updated on: ) | by Holden

[Added August 27, 2014: GiveWell Labs is now known as the Open Philanthropy Project.]

We previously listed our five chief criteria for GiveWell Labs, an arm of our research process that will be open to any giving opportunity, no matter what form and what sector. This post further discusses the third of these criteria: “accountability.”

We’re OK with funding a project that might fail, but it’s very important to us that we be able to recognize, document, publicly discuss, and learn from such a failure if it happens.

This is the area in which we feel most strongly that current philanthropists are coming up short: they’re failing to learn (or at least, to help others learn) from their track records. For a simple example, take the issue of sustainability in developing-world aid.

  • A common goal of a philanthropic program is to see the government – or another funder – take over at some point, leading to lasting impact that doesn’t depend on continued funding.
  • We’ve seen many different approaches to accomplishing this. For example, VillageReach initially paid entirely for its own project, with the hope that the government would switch over to its model once the proof of concept had been established; now that that idea has failed to pan out, VillageReach is asking for more cost-sharing from the government up front as it re-implements its model.
  • Yet there appears to be so little evidence on what sorts of projects have and haven’t achieved sustainability in the past that one paper by prominent scholars argues that the whole idea of sustainability is an “illusion” (an argument endorsed by William Easterly).

Most of the funders we’ve talked to don’t seem to have very clear senses even of their own organizations’ track records (both good and bad). Even if funders are learning internally and informally from their own failures, they aren’t learning from each others’.

We believe that we have an unusual commitment to public and documented discussion of whether our giving ends up accomplishing what we hope. We’ve been releasing regular updates on VillageReach, the organization we directed the most money to in 2010, and we intend to do the same with our current top charities. These updates are specific and honest about both good and bad news (and there has been a fair amount of the latter).

What I learned in my first 6 months at GiveWell

(updated on: ) | by Alexander

I started work at GiveWell six months ago, just a few weeks after graduating from college. I had been following GiveWell pretty intensely for more than a year, since I had gotten back from my own trip to India. During that time, I had become a little obsessed: I had read the entire history of the blog and got really excited each time GiveWell finally posted the audio from the most recent board meeting.

Even as a serious GiveWell fan, though, there were a number of things that I didn’t know about the organization that I should have. These aren’t secrets or titillating stories about office politics, just some things that I’ve learned that I didn’t know before.

The biggest challenge remains “find outstanding giving opportunities” – not “get more eyeballs.” I wasn’t totally ignorant about the difficulty of finding outstanding giving opportunities, but I thought that GiveWell was clearly doing so better than anyone else working publicly, and that accordingly it should focus more on outreach, rather than improving research. As an outsider, I didn’t have a good sense of how much went into the recommendations or all the work that goes into charities that don’t end up receiving recommendations. I didn’t think it was easy, but it seemed like Holden and Elie pretty much had it under control, and that there was lots of low-hanging fruit on the outreach side.

As far as I can tell now, neither of those things are really true.

On the outreach front, GiveWell had already tried or looked into many different strategies, even if they hadn’t blogged about it. And because our users generally aren’t typical donors, a lot of the things that charities normally do to cultivate donors might be actively harmful for us. (But we definitely haven’t thought of everything, so please do let us know or comment if you have ideas for how we could “sell” our research better.)

On the research front, although it isn’t very hard to come up with better recommendations than other charity evaluators, we face two problems I hadn’t fully considered:

  • Room for more funding. Because a number of large funders are scooping up excellent funding opportunities in global health, many good chances to help people are already taken. We need to find charities that are good bets, but not so obviously good that they have all the funding they can productively use.
  • Our competition isn’t other charity rating organizations. This is about the baseline that GiveWell’s recommendations are compared to, rather than the competition for funding opportunities. For a long time, it has seemed natural for people to compare GiveWell to Charity Navigator or Philanthropedia, but as we continue to grow, I think that comparison becomes less and less salient.

    As we raise our ambitions with projects like GiveWell Labs, we will be “competing” not with other charity evaluators but with foundations. Because some foundations are extremely strategic, well-resourced, and focused on the same goal of doing as much good as possible, finding better giving opportunities than they do is a much higher burden.

Both of these problems become harder as GiveWell grows, because we’ll need to create more “room for money moved” and will more naturally be compared to foundations rather than other charity evaluators.

Evaluation of American Red Cross Haiti response

(updated on: ) | by Holden

We’ve been working on an update of our disaster relief report, and came across an American Red Cross evaluation from December 2010 stating:

If you would like to access this report, please get in contact with the ALNAP secretariat.

We emailed the ALNAP secretariat, saying:

I am writing from GiveWell, an independent, non-profit charity evaluator to request access to the ALNAP report “American Red Cross – Haiti” that is listed on the ALNAP website at http://www.alnap.org/node/7131.aspx. Would it be possible to send us a copy of the report?

The secretariat responded that the evaluation cannot be shared externally due to an in-house policy.

Why should this report be confidential over a year after its publication?

(Thanks to Eliza Scheffler for finding this.)

Update: the page linked to in this post regarding the evaluation appears to have been removed (very recently – I am writing this at 11:02am and it was up as of 10:30am). Here is Google’s cache of the site and here is a copy of the Google cache stored on our server (for when the Google cache expires).