The GiveWell Blog

GiveWell’s plan for 2011: Specifics of research

(updated on: ) | by Holden
This post is more than 13 years old

This is the fifth post (of five) focused on our self-evaluation and future plans.

A previous post outlined our top-level priorities for 2011. The most important priority is finding more top charities. This post lists our potential tactics for finding top charities; we are particularly interested in feedback on this topic.

These tactics are listed in order from “closest to our existing methodology; most likely to succeed” to “furthest departure from our existing methodology; most likely to take a lot of time before we can identify outstanding organizations.”

Tactic 1: deep investigations of charities with distinction.

We have a list of charities that have some form of distinction. This includes

We haven’t found sufficient information on these charities’ websites to recommend them, and based on past experience, this makes us pessimistic. However, we have begun the process of prioritizing how promising the different charities are; we will be contacting the most promising ones, interviewing staff, and thinking about the minimum information we would need to confidently recommend them. More so than in the past, we can now point to significant impact of our recommendations on donations, so we expect better access to these charities than in the past.

Tactic 2: investigating “low burden of proof” sub-causes in international aid

In the past, we’ve looked for direct evidence of charities’ impact on improving lives. As we’ve gotten more context and experience with international aid, however, a couple of causes have stood out to us as particularly recommended/promising, to the point where we may be able to be confident in a charity without the sort of impact assessment we’ve sought in the past.

  • Orphans and vulnerable children: some charities provide homes, shelters, and other basic services for children who otherwise might be homeless, sleeping on the street, or even taken in by those who exploit them. By speaking with the right people, we may gain an understanding of where and when there are needs for these sorts of organizations to expand, resulting in more children having safe homes/shelters who would not otherwise.
  • Water: if we found a charity that was demonstrably improving access to clean water, in a way that (a) benefited communities with very poor previous access to water (b) lasted over time (we are very concerned about wells being put in and falling into disrepair), we might recommend such an organization without direct evidence of improved health outcomes.

Tactic 3: investigating other promising causes

As mentioned previously, we’re experimenting with a method for quickly getting a high-level picture of a charitable cause and how likely it seems that we could find top charities in this area. By investigating particularly promising causes, such as disease research funding and catastrophic risk mitigation (including but not limited to global warming), we might be able to find more outstanding opportunities for donors.

We will certainly be pursuing this tactic, but feel it is less likely to generate top charities in 2011 than the tactics above.

Tactic 4: project funding.

We have always aimed to find great organizations and recommend unrestricted donations to them, rather than funding particular projects. This is partly because we think traditional donation restricting is unreliable; partly because we think project-based funding adds harmful complications (particularly the fact that the donor’s and charity’s goals aren’t fully aligned); and partly because, in the past, we have had so little sense of how much money (if any) our top-rated charities could expect to raise.

But if we can’t find more charities that focus – at the overall organizational level – on proven, cost-effective, scalable programs, we will open the doors to large organizations offering promising projects, and potentially recommend that donors give to these organizations with specific designations (“Use this donation for project X”).

If we go down this path, it will become essential to have concrete expectations for what will be implemented – and what will be measured and reported – at different levels of funding. (Projects also ought to be based, to the maximum extent possible, on programs that have worked in the past.) The fact that we now have a track record of moving money to top-rated charities makes this option more feasible than it was before.

We’d like to avoid project-based funding, and even if we do implement it, we’ll be keeping an eye out for organizations that we can recommend for unrestricted funding. The latter will always take precedence.

We think this tactic is promising in the long run, but unlikely to generate “gold medal” opportunities in the short run because of the difficulties we’ve had (and expect to have) communicating with grantwriters.

Comments

  • Ian Turner on February 23, 2011 at 12:38 pm said:

    I am extremely surprised to see you list “Orphans and vulnerable children” as a “low burden of proof” approach, given the number of well-publicized problems with orphanages, such as child trafficking, adoption profiteering, orphan tourism, and the possibility that creating (or funding) orphanages can move orphans from a home environment with an extended family member into an institution.

    Saundra has written extensively about orphanages; perhaps the most important of these posts is “Does funding orphanages create orphans?“.

  • Jonah on February 23, 2011 at 6:33 pm said:

    I’m glad to see that GiveWell has several avenues to pursue for finding top charities in 2011.

    One point that one of you raised in your most recent board meeting is that there’s a question of why GiveWell has found only one VillageReach.

    I’ve been wondering about this for the past few months. A priori it seems unlikely that there would be exactly one charity meeting GiveWell’s criteria – one would either expect zero charities meeting the criteria or multiple charities meeting the criteria.

    One hypothetical narrative that could describe the situation here is that GiveWell is setting the bar for top rated charities based on the top charity that it sees. This is somewhat consistent with November 24th blog post:

    Part of what’s going on here is a gradual raising of our bar, as we have capacity for more analysis and a substantially higher opinion of our top-rated charity VillageReach than we had last year.

    But on the other hand there seems to be a pretty substantial gap between VillageReach and GiveWell’s silver medal rated international charities – I say this both based on my own impression having read this review and with a quote from Holden’s February 4th blog post

    When the 2010 holiday season was approaching, and we started thinking about our strategy for outreach, we realized that we only had one charity we could personally feel really good about aggressively raising money for. While we think all our recommended charities stand well above other donors’ options, it was only VillageReach for which I felt I could sit down with someone face-to-face about and say, “Give as much as you can to this one.”

    Why? Because not only does VillageReach have outstanding evidence of effectiveness; it has outstanding bang-for-the-buck (in absolute terms, not just relative terms) and most importantly, it has a concrete plan for additional funding.

    We personally don’t like raising money for groups that we can’t say all these things about. We’re happy to provide our recommendations for someone looking for the best microfinance charity, but we can’t honestly say that we think dollars given to it are accomplishing as much good as possible.

    Several possible interpretations here:

    1. VillageReach is a “once in a blue moon” type opportunity for casual donors to do good; it’s unrealistic to expect comparable opportunities for casual donors on a regular basis.

    2. GiveWell’s criteria are overly rigid in ways not necessarily correlated opportunities for casual donors doing the most good.

    3. Thus far, venues where GiveWell has looked for quality charities are suboptimal; there are other places to look that would yield more fruitful results.

    I have a poor sense of the relevance of each of the above mentioned factors. The above post seems to focus on (3), but maybe it’s worth considering (1) and (2) as well.

    ————-

    A question strongly related to that of why GiveWell has found only one VillageReach is the question of why VillageReach wasn’t able to secure adequate funding without GiveWell’s help. VillageReach’s pilot project was funded by the Gates Foundation. Why didn’t the Gates Foundation continue to fully fund VillageReach after the pilot project was successful?

    Also according to the linked websites for The Skoll Foundation and for The Mulago Foundation, both foundations funded VillageReach. Is the fact that they did not single-handedly bridge VillageReach’s funding gap an indication that they deemed their other grantees to be offer roughly comparable expected return on investment? Is the overlap between the values of the above mentioned foundations and GiveWell’s values sufficiently great so that a casual donor would do nearly as well to donate to one of the above mentioned foundations rather than to GiveWell’s top recommended charities?

    I have guesses of my own, and maybe one issue is that the foundations are not explicit about their reasons for funding the projects that they do; but it would be good to get a better picture of the reasons why VillageReach had difficulty bridging its funding gap in absence of GiveWell – it seems like thinking about this might point a way toward finding other opportunities to fund outstanding organizations with room for more funding.

  • Chuck S'r on February 25, 2011 at 2:10 am said:

    Your frustration with identifying top-rated or gold medal charities is palpable. I see your frustration as evidence of your integrity.

  • Holden on February 25, 2011 at 10:26 am said:

    Ian: “low burden of proof” does not mean that little information is needed overall; it means that once the cause and context are well understood, there may be little information needed from individual charities.

    Investigating the cause of orphans and vulnerable children would mean trying to understand (perhaps from, for example, speaking with government agencies that refer them to nonprofits) when and where there are orphans and vulnerable children who lack appropriate support services, and thus trying to understand where more orphanages or support services are needed. This is not easy or necessarily feasible, but if we could come to this understanding, we wouldn’t need much information from orphanages themselves, and that’s important because it’s usually at the stage of getting information directly from charities that things become most difficult.

    Jonah: I think all the factors you name are relevant, including the raising of our bar.

    • We’re always going to compare charities against the best we’ve found. So it isn’t that we’ve set a certain bar and only VillageReach cleared it; it’s that the gap between our #1 and the rest happens to be large.
    • VillageReach is unusual in a lot of ways, in terms of timing as well as in terms of activities/evaluation. It is a small organization that has had one well-demonstrated success but hasn’t yet scaled it up. That’s arguably a “sweet spot” for casual donors, because there is neither the uncertainty that comes with an entirely new project nor the uncertainty that comes with the size and bureaucracy of a scaled-up organization. And it’s my view that in charity, the effect of brand and peer pressure creates fast and uncontrolled jumps from “new and unestablished” to “dramatically overfunded to the point of departing far from the original vision.” So VillageReach is at a particular stage that most charities don’t seem to stay at for long, and it may have stayed at this stage longer than most because of struggles in storytelling – i.e., the unusual difficulty of fundraising for it may be a factor behind its unusually high appeal to us and our donors. Finally, it has an aptitude for meaningful evaluation and a commitment to transparency that we’ve found rare.
    • Since the beginning of the project, we’ve preferred “false negatives” (failing to recommend good charities) to “false positives” (recommending weak charities), which has led us to err on the side of having overly strict and limited criteria. The more we learn, the more different ways we come up with to identify a charity as outstanding, and the more we can broaden. So the fact that there’s only one charity we really feel good about is partly a statement about our own knowledge and experience (though it is also a statement about the sector).

    Taking these considerations together, the fact that we’ve only found one VillageReach to date doesn’t seem particularly shocking or puzzling to me.

    As for why VillageReach has had trouble getting funding from elsewhere, we’ve speculated that it has to do with the difficulty of storytelling for this kind of work (relevant for all donors but especially individuals) and the fact that it’s looking to scale up a project of its own (which makes it unattractive to most institutional funders). Note, however, that it does still receive funding from both Mulago and Skoll and that both of these organizations tend to keep their gifts to any particular organization within a pretty consistent range.

Comments are closed.