6 months ago, GiveWell pledged to give a $25,000 grant to the best organization we found in each of our five causes, and we’re going to follow through. But, I wish we didn’t have to.

We recognized that donors are more likely to trust us for some decisions (comparing organizations with similar goals) than for others (for example, trying to compare the value of saving a child’s life in Africa to that of helping improve a child’s education in the New York City). That’s why we divided charities into different causes, with the aim of giving one grant in each cause.

Having learned what I have about organizations that aim to raise incomes in New York City and those that aim to save lives in Africa, I would not donate to any of our applicants aiming to raise incomes in New York. I’d donate instead to Population Services International, our top choice for saving lives in Africa. I have two reasons for this:

1. The relative cost of the programs.
2. The logic underlying the approach.

1. Relative cost. The organizations we recommend – the ones in which we have the most confidence – spend $10,000-50,000 to help an unemployed/under-employed person become self supporting. For $50,000, I’d expect to save at least 50 lives in Africa by donating to Population Services International. There are reasonable arguments in favor of improving lives in the developed world – there’s value in helping people “get over the hump” that’s preventing them from living a fullly enabled otherwise happy life – and our “lives saved” number is of course a simplification that leaves out a lot. But these are not in the same ballpark. I think that the ratio of people helped (10-50:1) is just too much.

2. The logic underlying the approach. I don’t understand the “theory of change” (or, the basic logic behind the intervention) that explains the outcomes that Employment Assistance charities achieve. Are some basic soft skills (resume writing, interviewing tips, etc.) preventing their clients from finding employment? Do their clients need specific vocational skills (computer technician, environmental remediation technician, etc.)? We’ve wondered if it makes more sense to run job-training programs or just give the people money to support their basic needs, or otherwise, provide directly for their needs. In Africa, I understand the logic behind the interventions much more clearly (for example, selling insecticide treated nets that prevent mosquitoes from biting children, and thus prevent the transmission of malaria).

I end up feeling like a donation can make a huge difference in many people’s lives, with high confidence (through our top Cause 1 organizations) – or might make a difference in someone’s life, with low confidence and an unclear understanding of the logic (through our top Cause 5 organizations).

Our aim at GiveWell is to help donors find great charities to donate to by identifying the best charities in a cause (in which we can compare apples to apples). Our goal isn’t directly to choose their causes for them because the choice of apples versus orgnges is often a question of personal taste. But we hope that as we show donors more concretely what they get for their dollar, that will affect what causes they choose too. Speaking very broadly and roughly, imagine that for $25,000 you could save 25 lives in Africa or help 1 person get a better job in New York City. I know how I’d choose. What would you do?

I want to take the time to explain a couple views I hold that offend some people. These views are:

• The label “philanthropy expert” means very little to me.
• I see no strong reason to believe that charitable foundations are generally capable, competent, or effective at anything at all.

These aren’t insults aimed at particular people, and they aren’t hyperbole intended to generate controversy. They’re opinions I hold based on my observations about how philanthropy is structured. I can explain them. Both of these beliefs come from the fact that I have no way of determining whether the “respected experts” in philanthropy represent a set of True Experts, or an old boys’ club.

True Experts

True Experts have two qualities. (1) They have learned a lot from experience. (2) This translates to a demonstrable ability to do something.

Regarding (1): Experience can be educational … or it can not be. It’s educational when it consists of trying something, learning whether it worked, and trying something else, all in an environment where “what works” is fairly stable. A highly experienced glovemaker will know all kinds of tricks of the trade and have all kinds of instincts that a young but “naturally talented” glovemaker doesn’t.

In the world of investing, by contrast, it’s much less safe to bet on the more experienced person. The environment is so complicated and the results so random that people can spend 50 years day-trading without ever learning from it. Writing music is even more this way: I don’t think there’s any strong connection between a musician’s level of experience and their ability to write great music (if anything the relationship is negative).

Now imagine an activity that consists of investing without looking at your results. In other words, you buy a stock, but you never check whether the stock makes money or loses money. You never read the news about whether the company does well or does poorly. How much would you value someone with this sort of experience – buying and selling stocks without ever checking up on how they do? Because that’s what “experience in philanthropy” (or workforce development, or education) comes down to, if unaccompanied by outcomes evaluation.

The peculiar thing about philanthropy is that because you’re trying to help someone else – not yourself – you need the big expensive study, or else you literally have no way of knowing whether what you did worked. And so, no way of learning from experience. I wish I had an analogy for this but I don’t. There is no analogy. Philanthropy is the one sphere where you don’t get any info about your effects unless you explicitly study them.

Regarding (2): In most fields, “expert” doesn’t just mean “experienced” – it means “proven.” Tons of people have been investing their whole lives and are still dreadful at it, worse than a bright young 18-year-old would be. But Warren Buffett has a strong track record of investment success; he’s a True Expert in investing. On a smaller scale, your local doctor hopefully has a track record of helping people get better. OK, there’s no randomized-design evaluation of him, but we have a lot of independent information about how medical ailments progress if untreated, and the people your doctor treats get better instead. Unlike a Middle Ages leech supplier, your doctor is a True Expert in medicine.

As an aside, it’s pretty rare for a True Expert to refuse to share their knowledge. Even in the hypercompetitive finance industry, Warren Buffett basically tells you how he thinks because he knows you can’t replicate it. In more world-serving fields such as academia, every expert’s beliefs are peer reviewed, published, and available to any schmoe with a library card.

What am I to make of the statement that Person X is a respected expert in philanthropy? It could mean that they have a track record of solving social problems, and I just haven’t seen the evidence due to a lack of information sharing. Or it could mean …

Old boys’ clubs

We’ve all seen old boys’ clubs. They’re groups of people who think alike, act alike, and have some sort of power or privilege that they don’t have to continually earn.

I’ll use the example of a typical college fraternity. Nominally, the fraternity has a mission, something about “upholding honor and serving the community” or something. Nominally, the President is the person most qualified to help the fraternity promote this mission, the Treasurer is the best person to keep the books, etc. But really? The President is the most popular guy. The Treasurer is ~4th-most popular. The guy who slept with the President’s girlfriend is at the bottom of the totem pole. Etc.

These clubs’ members usually have a lot of weird, idiosyncratic beliefs about the world, because of how similar they are to each other and how little they check these beliefs with the outside. You can usually bet that the “accepted wisdom” at these clubs plus $2 is worth about one subway ride. You can usually not bet that the “top people” at these clubs are better in any way than the others. And it’s not clear that Louis XVI and his cronies did, in fact, run France better than a 26-year-old punk might have.

Foundations

The thought that much of the world’s well-meaning money is managed by old boys’ clubs is scary – scary enough to make one want to deny it, even without evidence. But the fact is, everything about the way foundations are structured is consistent with the idea that they may be old boys’ clubs. I have no evidence that they do the measuring necessary to learn from experience. (They might, and they might not.) I have no evidence that they interact meaningfully with people outside their bubble. (They might, or they might not.) I have no evidence that they have made the world better off, consistent with their mission. (They may have. They may not have.)

I have no evidence that their power and privilege are connected in any way with performance or merit. Unlike with a private company, the fact that they have money means only that someone gave them money decades ago.

The only thing I do know is that they don’t share their information as much as would be consistent with devotion to a better world. That’s why I have a negative view of them.

I’m not committed to this view. Maybe foundations do have ways of learning from their activities. Maybe they do have a track record of success. Maybe they know far more about philanthropy than GiveWell can ever hope to. That’s definitely possible.

So which is it? Foundations, are you an old boys’ club – or can you demonstrate that you’re True Experts?

I am proud to announce the winner of the 2007 Holden Award for Excellence in Imperfection. This award has a rich history dating back to 3 minutes ago. The rules are as follows:

The first charity to make a negative correction to its GiveWell review wins the award.

Since we’ve published our review drafts, we’ve gotten many corrections – “you’ve underestimated ___ advantage of our organization,” “you’ve left out how great it is that we ___,” etc. Most of these corrections have been correct or at least worth considering, and I’m glad we’ve gotten them. But, not until today did someone write in, “You’re overstating our lives saved per $.” For that, this charity wins the 2007 Holden Award for Excellence in Imperfection, in recognition of its honesty, humility, and recognition that getting the truth out there is more important than looking good.

The award comes with a $500 donation from Holden (not from The Clear Fund). This means a lot now that I no longer make a hedge fund salary.

Your winner: Population Services International, for sending us downward revisions of its own estimates of HIV/AIDS infections averted. Well there’s a shocker for you. The very same organization that has demonstrated the strongest commitment to monitoring and evaluating itself is also the first one to ask us to remove information that exaggerates its impact! Maybe monitoring and evaluation really do go hand in hand with humility. Would you guess that combination also goes hand in hand with effectiveness?