In a recent debate, David Hunter’s article on the nonprofit sector has taken heat for its assertion that “While nonprofits work incredibly hard, with passion and dedication, and often in incredibly difficult circumstances to solve society’s most intractable problems, there is virtually no credible evidence that most nonprofit organizations actually produce any social value.”
We agree with the claim for the sectors we’ve examined, which we believe are similar to the sectors Mr. Hunter has examined: particularly thorny areas such as charities working to improve education and international charities addressing extreme poverty overseas. These are problems on which experts have struggled for decades to make any progress, and while we don’t necessarily agree that most charities are failing to produce value, we agree that most charities cannot produce any credible evidence that they are. This is different from the claim that Sean Stannard-Stockton attributes to Mr. Hunter (“most nonprofits and the social sector as a whole is not currently producing social value”), but it still means that it’s very hard for a donor to give with confidence.
The information we have
Our belief is based on two years of looking for this evidence; we’ve published the full details of our findings online, and you can see our summary of international charities (only 19 out of 320 examined publish any impact-related evaluation reports) and U.S. equality of opportunity charities (only 6 of 83 examined provide credible impact-related reports, and 2 of these show negative or no impact).
In addition, in a guest post on the GiveWell Blog, David Anderson of the Coalition for Evidence-Based Policy estimates that 75% of rigorous evaluations show weak effects, no effects, or negative effects.
More information needed
On the other hand, we also believe the criticism that Mr. Hunter doesn’t support his own claim with evidence has merit. We would like more clarity on which sectors Mr. Hunter has examined and is referring to, and information on where he has looked for evidence and what he has found.
In addition, we feel that examples of failing/harmful programs, such as “Well intentioned but ineptly run mentoring programs where failed matches reinforce in youngsters a sense of their low worth and poor prospects” (and the other items on the list on page 2 of the article) should be clearly referenced to summaries of evidence.
The truth is that we cannot have a very informed debate about how much value nonprofits create because we have so little evidence of any kind. Some people adamantly believe that nonprofits create enormous value; others are skeptical that they create any; and there is very little to go on, at least in the sectors under discussion.
Nonprofits that do have credible evidence of their social impact
The good news for donors is that they need not be in the dark if they give to the right charity. Our top-rated charities do produce credible evidence of their social impact. We encourage individual donors to expand and fund these charities until and unless others follow suit.
Comments
Holden, I know you are very careful in the words you choose. A word by word reading of David’s first truth…
“While nonprofits work incredibly hard, with passion and dedication, and often in incredibly difficult circumstances to solve society’s most intractable problems, there is virtually no credible evidence that most nonprofit organizations actually produce any social value.”
…does leave open the possibility that David actually believes that the social sector may or may not be producing social value, but he is unable to prove it. But no average person would read the statement that way.
If I wrote on my blog “there is no credible evidence that Holden knows what he’s talking about!” ;^) It would be clear that I was expressing an opinion about your actual knowledge and could not with a straight face suggest that I was just commenting on the lack of evidence.
Sean, I do intend to read the spirit rather than the letter of Mr. Hunter’s comments, and not to be literal about them. But I simply don’t have the same reading you do.
My personal position that is that I’m very unsure about how much good charities are actually accomplishing, but believe that (a) the ones I’ve examined “easily could be” failing; (b) there isn’t evidence to settle whether they are; (c) I can’t be confident in them; (d) this is a huge problem that I would like to see addressed by funders’ placing more emphasis on evaluation.
Maybe it’s because this is my personal position, but when I read Mr. Hunter’s piece, I see a very similar sentiment expressed.
I wonder if we’re holding charities to a higher standard than we would for a for-profit business? If an anonymous local pizza joint is profitable then we’d probably say they’re doing something right, even though non-customers are not going hungry and they probably wouldn’t be missed if they went out of business. Similarly, if a local charity serves local customers who might otherwise get similar help elsewhere, it might not be as good a choice for donations as another charity, it doesn’t follow that they’re not doing any good.
If a business isn’t profitable, it (eventually) disappears. On the other hand, if a charity is fundamentally useless, it can go on being useless indefinitely as long as it gets money from somewhere. There’s basically no accountability.
Brian, we do think nonprofits should be held to a higher standard than for-profits. As we’ve discussed before, nonprofits need to put MORE effort into measurement because they don’t have the built-in metric of profits. And when you support a nonprofit, you need enough confidence in the organization to feel comfortable giving away money that you’ll never see the direct impact of; when you buy a slice of pizza, all you need confidence in is the pizza.
A couple of other points of clarification:
Comments are closed.