The GiveWell Blog

Are charities helping? We don’t know

In a recent debate, David Hunter’s article on the nonprofit sector has taken heat for its assertion that “While nonprofits work incredibly hard, with passion and dedication, and often in incredibly difficult circumstances to solve society’s most intractable problems, there is virtually no credible evidence that most nonprofit organizations actually produce any social value.”

We agree with the claim for the sectors we’ve examined, which we believe are similar to the sectors Mr. Hunter has examined: particularly thorny areas such as charities working to improve education and international charities addressing extreme poverty overseas. These are problems on which experts have struggled for decades to make any progress, and while we don’t necessarily agree that most charities are failing to produce value, we agree that most charities cannot produce any credible evidence that they are. This is different from the claim that Sean Stannard-Stockton attributes to Mr. Hunter (“most nonprofits and the social sector as a whole is not currently producing social value”), but it still means that it’s very hard for a donor to give with confidence.

The information we have

Our belief is based on two years of looking for this evidence; we’ve published the full details of our findings online, and you can see our summary of international charities (only 19 out of 320 examined publish any impact-related evaluation reports) and U.S. equality of opportunity charities (only 6 of 83 examined provide credible impact-related reports, and 2 of these show negative or no impact).

In addition, in a guest post on the GiveWell Blog, David Anderson of the Coalition for Evidence-Based Policy estimates that 75% of rigorous evaluations show weak effects, no effects, or negative effects.

More information needed

On the other hand, we also believe the criticism that Mr. Hunter doesn’t support his own claim with evidence has merit. We would like more clarity on which sectors Mr. Hunter has examined and is referring to, and information on where he has looked for evidence and what he has found.

In addition, we feel that examples of failing/harmful programs, such as “Well intentioned but ineptly run mentoring programs where failed matches reinforce in youngsters a sense of their low worth and poor prospects” (and the other items on the list on page 2 of the article) should be clearly referenced to summaries of evidence.

The truth is that we cannot have a very informed debate about how much value nonprofits create because we have so little evidence of any kind. Some people adamantly believe that nonprofits create enormous value; others are skeptical that they create any; and there is very little to go on, at least in the sectors under discussion.

Nonprofits that do have credible evidence of their social impact

The good news for donors is that they need not be in the dark if they give to the right charity. Our top-rated charities do produce credible evidence of their social impact. We encourage individual donors to expand and fund these charities until and unless others follow suit.

Comments

  • Sean Stannard-Stockton on October 20, 2009 at 8:44 am said:

    Holden, I know you are very careful in the words you choose. A word by word reading of David’s first truth…

    “While nonprofits work incredibly hard, with passion and dedication, and often in incredibly difficult circumstances to solve society’s most intractable problems, there is virtually no credible evidence that most nonprofit organizations actually produce any social value.”

    …does leave open the possibility that David actually believes that the social sector may or may not be producing social value, but he is unable to prove it. But no average person would read the statement that way.

    If I wrote on my blog “there is no credible evidence that Holden knows what he’s talking about!” ;^) It would be clear that I was expressing an opinion about your actual knowledge and could not with a straight face suggest that I was just commenting on the lack of evidence.

  • Holden on October 20, 2009 at 9:14 am said:

    Sean, I do intend to read the spirit rather than the letter of Mr. Hunter’s comments, and not to be literal about them. But I simply don’t have the same reading you do.

    My personal position that is that I’m very unsure about how much good charities are actually accomplishing, but believe that (a) the ones I’ve examined “easily could be” failing; (b) there isn’t evidence to settle whether they are; (c) I can’t be confident in them; (d) this is a huge problem that I would like to see addressed by funders’ placing more emphasis on evaluation.

    Maybe it’s because this is my personal position, but when I read Mr. Hunter’s piece, I see a very similar sentiment expressed.

  • Brian Slesinsky on October 21, 2009 at 3:19 am said:

    I wonder if we’re holding charities to a higher standard than we would for a for-profit business? If an anonymous local pizza joint is profitable then we’d probably say they’re doing something right, even though non-customers are not going hungry and they probably wouldn’t be missed if they went out of business. Similarly, if a local charity serves local customers who might otherwise get similar help elsewhere, it might not be as good a choice for donations as another charity, it doesn’t follow that they’re not doing any good.

  • Doug S. on October 21, 2009 at 4:06 pm said:

    If a business isn’t profitable, it (eventually) disappears. On the other hand, if a charity is fundamentally useless, it can go on being useless indefinitely as long as it gets money from somewhere. There’s basically no accountability.

  • Holden on October 22, 2009 at 5:30 pm said:

    Brian, we do think nonprofits should be held to a higher standard than for-profits. As we’ve discussed before, nonprofits need to put MORE effort into measurement because they don’t have the built-in metric of profits. And when you support a nonprofit, you need enough confidence in the organization to feel comfortable giving away money that you’ll never see the direct impact of; when you buy a slice of pizza, all you need confidence in is the pizza.

    A couple of other points of clarification:

    • We are not (and I personally don’t think David Hunter is) saying that charities accomplish no good. We are saying that because of the paucity of evidence, it’s unacceptably difficult to identify the good/best ones and to have confidence in a donation.
    • The main concern raised by Mr. Hunter and by us is not that charities are providing benefits that others might provide in their absence. It’s that charities may not be providing benefits at all, or at least nowhere near the level of benefits that they are “selling” to donors.

Comments are closed.