The GiveWell Blog

True Experts? Or old boys’ club?

I want to take the time to explain a couple views I hold that offend some people. These views are:

  • The label “philanthropy expert” means very little to me.
  • I see no strong reason to believe that charitable foundations are generally capable, competent, or effective at anything at all.

These aren’t insults aimed at particular people, and they aren’t hyperbole intended to generate controversy. They’re opinions I hold based on my observations about how philanthropy is structured. I can explain them. Both of these beliefs come from the fact that I have no way of determining whether the “respected experts” in philanthropy represent a set of True Experts, or an old boys’ club.

True Experts

True Experts have two qualities. (1) They have learned a lot from experience. (2) This translates to a demonstrable ability to do something.

Regarding (1): Experience can be educational … or it can not be. It’s educational when it consists of trying something, learning whether it worked, and trying something else, all in an environment where “what works” is fairly stable. A highly experienced glovemaker will know all kinds of tricks of the trade and have all kinds of instincts that a young but “naturally talented” glovemaker doesn’t.

In the world of investing, by contrast, it’s much less safe to bet on the more experienced person. The environment is so complicated and the results so random that people can spend 50 years day-trading without ever learning from it. Writing music is even more this way: I don’t think there’s any strong connection between a musician’s level of experience and their ability to write great music (if anything the relationship is negative).

Now imagine an activity that consists of investing without looking at your results. In other words, you buy a stock, but you never check whether the stock makes money or loses money. You never read the news about whether the company does well or does poorly. How much would you value someone with this sort of experience – buying and selling stocks without ever checking up on how they do? Because that’s what “experience in philanthropy” (or workforce development, or education) comes down to, if unaccompanied by outcomes evaluation.

The peculiar thing about philanthropy is that because you’re trying to help someone else – not yourself – you need the big expensive study, or else you literally have no way of knowing whether what you did worked. And so, no way of learning from experience. I wish I had an analogy for this but I don’t. There is no analogy. Philanthropy is the one sphere where you don’t get any info about your effects unless you explicitly study them.

Regarding (2): In most fields, “expert” doesn’t just mean “experienced” – it means “proven.” Tons of people have been investing their whole lives and are still dreadful at it, worse than a bright young 18-year-old would be. But Warren Buffett has a strong track record of investment success; he’s a True Expert in investing. On a smaller scale, your local doctor hopefully has a track record of helping people get better. OK, there’s no randomized-design evaluation of him, but we have a lot of independent information about how medical ailments progress if untreated, and the people your doctor treats get better instead. Unlike a Middle Ages leech supplier, your doctor is a True Expert in medicine.

As an aside, it’s pretty rare for a True Expert to refuse to share their knowledge. Even in the hypercompetitive finance industry, Warren Buffett basically tells you how he thinks because he knows you can’t replicate it. In more world-serving fields such as academia, every expert’s beliefs are peer reviewed, published, and available to any schmoe with a library card.

What am I to make of the statement that Person X is a respected expert in philanthropy? It could mean that they have a track record of solving social problems, and I just haven’t seen the evidence due to a lack of information sharing. Or it could mean …

Old boys’ clubs

We’ve all seen old boys’ clubs. They’re groups of people who think alike, act alike, and have some sort of power or privilege that they don’t have to continually earn.

I’ll use the example of a typical college fraternity. Nominally, the fraternity has a mission, something about “upholding honor and serving the community” or something. Nominally, the President is the person most qualified to help the fraternity promote this mission, the Treasurer is the best person to keep the books, etc. But really? The President is the most popular guy. The Treasurer is ~4th-most popular. The guy who slept with the President’s girlfriend is at the bottom of the totem pole. Etc.

These clubs’ members usually have a lot of weird, idiosyncratic beliefs about the world, because of how similar they are to each other and how little they check these beliefs with the outside. You can usually bet that the “accepted wisdom” at these clubs plus $2 is worth about one subway ride. You can usually not bet that the “top people” at these clubs are better in any way than the others. And it’s not clear that Louis XVI and his cronies did, in fact, run France better than a 26-year-old punk might have.

Foundations

The thought that much of the world’s well-meaning money is managed by old boys’ clubs is scary – scary enough to make one want to deny it, even without evidence. But the fact is, everything about the way foundations are structured is consistent with the idea that they may be old boys’ clubs. I have no evidence that they do the measuring necessary to learn from experience. (They might, and they might not.) I have no evidence that they interact meaningfully with people outside their bubble. (They might, or they might not.) I have no evidence that they have made the world better off, consistent with their mission. (They may have. They may not have.)

I have no evidence that their power and privilege are connected in any way with performance or merit. Unlike with a private company, the fact that they have money means only that someone gave them money decades ago.

The only thing I do know is that they don’t share their information as much as would be consistent with devotion to a better world. That’s why I have a negative view of them.

I’m not committed to this view. Maybe foundations do have ways of learning from their activities. Maybe they do have a track record of success. Maybe they know far more about philanthropy than GiveWell can ever hope to. That’s definitely possible.

So which is it? Foundations, are you an old boys’ club – or can you demonstrate that you’re True Experts?

The 2007 Holden Award for Excellence in Imperfection

I am proud to announce the winner of the 2007 Holden Award for Excellence in Imperfection. This award has a rich history dating back to 3 minutes ago. The rules are as follows:

The first charity to make a negative correction to its GiveWell review wins the award.

Since we’ve published our review drafts, we’ve gotten many corrections – “you’ve underestimated ___ advantage of our organization,” “you’ve left out how great it is that we ___,” etc. Most of these corrections have been correct or at least worth considering, and I’m glad we’ve gotten them. But, not until today did someone write in, “You’re overstating our lives saved per $.” For that, this charity wins the 2007 Holden Award for Excellence in Imperfection, in recognition of its honesty, humility, and recognition that getting the truth out there is more important than looking good.

The award comes with a $500 donation from Holden (not from The Clear Fund). This means a lot now that I no longer make a hedge fund salary.

Your winner: Population Services International, for sending us downward revisions of its own estimates of HIV/AIDS infections averted. Well there’s a shocker for you. The very same organization that has demonstrated the strongest commitment to monitoring and evaluating itself is also the first one to ask us to remove information that exaggerates its impact! Maybe monitoring and evaluation really do go hand in hand with humility. Would you guess that combination also goes hand in hand with effectiveness?

Getting the word out

It’s been about 5 days since we released our Africa report, and now we’re trying to get the word out. Some stuff:

  • This week’s Chronicle of Philanthropy features us, and in a true act of sadism puts me on the cover.
  • Smart Money mentions us in an article on intelligent giving. On the other hand, even after I desperately tried to explain to the reporter why the Straw Ratio is worthless, the article still spouts out the same old crap about “The important figure to look for is the spending ratio” blah blah blah. Even when they quote me pointing out that “That’s like looking at how much of a movie’s budget is going to the actors, and rating the end result based on just that,” the conclusion they come to is that you need to adjust the Straw Ratio for the sector – not that you need to scrap it altogether.

    Nobody in the nonprofit sector, except Charity Navigator itself, thinks these ratings are a good idea. How can we get this across to the mainstream media?

  • Tom Belford, with whom we’ve sparred before, made a post that I enjoyed quite a bit and completely captures the big picture of our project. He rightly points out that I’m a punk, and also (wrongly I think, but it’s debatable) calls our focus on cost-effectiveness “naive or chilling.” But this is what really matters: GiveWell is “truly putting to shame much larger foundations and major donors who haven’t shown any remotely proportionate determination to ask probing questions about the efficacy of the groups they support. Or to share the answers when they do ask … If Give Well sticks at it, they’ll make a huge contribution. Not so much via their own direct giving, but rather through their provocation and the public sharing of their analyses, successes and failures.” Right. Our decisions are of course going to be highly debatable; but the most important reason for our project is just to get the discussion going.
  • Want to read our press release? Check out Sean calling it the “best press release I’ve ever seen.” He must not have seen Charity Navigator’s … we can all only guess how they’ve been managing to get the media to listen to their perspective over the unanimous disagreement of everyone else who has ever thought about charity.
  • Finally … don’t forget to join the online chat with me at noon ET. Anyone who asks me a stumper gets a free GiveWell hat.

Wanna help spread the word further, and get as many people as possible to give well this holiday season? Here are some ideas:

  • Comment/email on your favorite blogs about us, especially when they bring up charity.
  • Set your away msg or status msg (Facebook, gchat, Twitter) to GiveWell.net .
  • Consider a GiveWell tattoo. What have you got to lose?

We’ll be back to ranting about Africa soon enough. Stay tuned.

2 things WAY more exciting than a blog post

1. Our research on saving lives in Africa is now public.

2. Online chat this coming Tuesday, via the Chronicle of Philanthropy, with a special guest who looks absolutely too good to miss. Please participate – I type 120wpm, I am beyond pumped to have an online chat with the nonprofit sector, I want to take on all comers, and if it ends up being an hour of softball questions like “Do you think the nonprofit sector would benefit from more transparency?” I’m going to blame you.

That’s all for today, and it’s more than enough. Please take a look at our Africa research and share your thoughts.

What a “life saved” means

The goal of Cause 1 is to save lives in Africa, and we estimate that a good strategy can save a human life for somewhere in the ballpark of $1000. Sounds like an unbelievable deal, right?

Not to everyone. I was recently talking to a Board member and mentioned how much cheaper it seems to be to change/save lives in Africa vs. NYC. He responded, “Yeah, but what kind of life are you saving in Africa? Is that person just going to die of something else the next year?”

I think it’s interesting how (a) completely fair, relevant and important this question is for a donor; (b) how rarely we see questions like this (“Sure I helped someone, but what kind of life did I enable?”) brought up and analyzed. Here’s what we know right now:

Read more

Welcome to your virtual site visit

Monday I visited Year Up, our #1 ranked charity in the cause of employment assistance. Although they quite understandably declined my request to videotape the visit, I’m going to share as much of my impressions as I can here. Although I think informal/intuitive evidence is generally overrated (relative to facts about life outcomes), I do still think it’s valuable.

Read more