I think most discussions of charity are way too fuzzy. Your standard fundraising proposal has several adorable pictures, a couple disturbing ones, and one or two numbers that seem to have been shouted in a moment of impassioned inspiration, with nary a source to be found. Then, though, there is the occasional economist/mathematician/cyborg who comes along and decides enough is enough – no more guessing and generalizing, we’re going to get everything down to one number come hell or high water. This annoys me even more than excessive fuzziness. At least when someone’s crying incoherently, I can tell they’re human.

There’s a common insistence on stating the cost of, say, malaria in terms of impact on GDP. You can see a great example here … with a footnote to an academic paper that I’m guessing put a sick amount of effort into coming up with these numbers. Frankly, I don’t want to think about this effort, let alone read about it, because this number is meaningless.

Allow me to explain. According to these guys, malaria costs Africa about $12 billion per year. … What does that tell you? Does that help you understand anything about the impact of malaria? Well, maybe you just need some context, so here, I’ll put it in context for you. It’s about half the cost per year we would incur by dismantling Citigroup, a little more than dismantling AIG (here’s my very questionable source for those claims), 1/1000 the annual output of the United States, 1/50 the annual output of Brazil, and half the annual cost of racism (OK, I made that last one up). The death of Alex Rodriguez would cost the world about $25 million next year, or the same as the death of 315.14 college professors. We’re totally clear now?

To whom does “$12 billion” mean more than “2 million people”?

Maybe the problem is that this number just isn’t complicated and counterintuitive enough. That must be what the World Health Organization was thinking when they put together Disability Adjusted Life-Years. Read that formula. Then tell me what “5 Disability Adjusted Life-Years” means to you.

Of course, the fact that something is complicated doesn’t mean it’s a bad idea. I’m a fan of the VORP metric in baseball, for example, because having read the description of how it’s calculated, I feel reasonably confident that if one player’s VORP is way higher than another’s, he’s probably a more valuable hitter. Putting all causes in the same terms tries to do for charity what VORP does for baseball: let us compare everything in the same terms.

The problem is, it just can’t be done. Deciding whether fixing education or malaria is more important involves a ton of philosophical judgment – enough that you can’t reasonably entrust that decision to an “expert” with a calculator. Once people start trying to make major philosophical decisions with formulas, they come out with numbers that no longer tell you what you want to know.

Our goal at GiveWell is to measure what we can measure, compare what we can compare, and acknowledge when we reach the point at which decisions become philosophical judgment calls. That way the donor doesn’t have to trust our reasoning … and that way we can make sure we’re still talking about people.

We met with New Visions for Public Schools on Friday afternoon. The contrast with the Children’s Aid Society meeting was instructive. CAS met us for lunch; New Visions met us in a conference room. (I like food, but I think we’re going to avoid lunch meetings in the future so we don’t end up with all our notes on soup-stained Post-its.) The CAS representatives were all smiles; the conversation with Bob, the President of New Visions, bordered at times on adversarial. When asked about other organizations, CAS stressed how much they talked and worked together, despite not sharing funding; Bob pretty much said the opposite, stressing the practical challenges of partnerships.

I came away pretty impressed with New Visions, and feeling that they did a better job of addressing my concerns in a head-on, all-business way (at least in the meeting – we’ve waiting on followup materials from both). The fact that New Visions generally has a “supporting” role in its projects, working with the Department of Education and with smaller teams rather than running programs itself, had been making it hard for us to understand where the money literally goes; now we have a much better sense of that, and I’ll incorporate it into the New Visions review. What’s so exciting about New Visions is that it is funding, supporting, and getting the government to cooperate in a variety of unconventional and “experimental” approaches, and placing a high priority on collecting evidence of these approaches’ effectiveness. The devotion to and quality of self-evaluation distinguish it from other education-centered organizations; another distinction is that by working in existing public schools rather than starting charter schools or “school choice” programs, New Visions is more likely to affect the students who need the most help.

My biggest concern about New Visions is that it doesn’t seem to be building a centralized “This is what we think works, and this is what we think doesn’t” list. I find its Ten Principles of Effective Small Schools vague; more importantly, this list has been around since its small school initiative started 3 years ago. Dialogue is presumably taking place, but what and where are the conclusions (even if tentative) being reached?

The common thread between New Visions, CAS, and Robin Hood (whom we finally got to start a conversation with today, thanks to GiveWell member Kerry’s greasing the wheels with a donation), is that none of them seem to have many other donors who are interested in the details of their activities and self-evaluations. You can see it from their concern about “overwhelming” us with material (which we keep trying to explain is not a concern) to their lack of detailed funding breakdowns to their lack of precedent for having the kinds of conversations we want (both of our in-person meetings ran over; one was over corn chowder; and the Robin Hood representative seemed genuinely surprised that someone wanted to “look under the hood”). Don’t tell me this is because people are “leaving the details to the experts” – that isn’t what you do when a car dealer, even a nice one, assures you that what you’re about to buy is “all fixed up,” and it isn’t what you do when your President assures you that our military plan is all checked out. Any time there’s high stakes, low information, and multiple options, leaving the details to the experts is a mistake. And though we hope that we’ll eventually become experts on charity-related issues ourselves, we also hope you’ll demand that everything that goes into our recommendations is available for you to check out.

I’ve been ranting about how ridiculous it is to judge a charity by the % of its funds that go to administrative expenses, which I’ve dubbed the “Straw Ratio” because I was too lazy to find a prominent advocate of it so I just invented a Straw Man.

Well, it turns out there quite a lot of human men (and women) who make this mistake. Specifically, according to a recent study that I found out about from Gift Hub, 75% of high-net-worth households say they would give more to charity if less were spent on administrative expenses.

This aspect of charity got by far the strongest response out of anything the households were surveyed on. High-net-worth households care more about their money going to administration than they do about having more access to research (34%), or being able to determine the impact of their gifts (60%). See page 7 of the actual study if you think I’m lying.

Meanwhile, frustrated with the incompetence and disorganization I’ve seen in the nonprofit sector, I’ve practically been begging these guys to spend more on administrative expenses.

If these survey numbers are to be trusted, we’ve got quite a conundrum here. How do we convince people that this seductively easy-to-measure number is no more meaningful for a charity than for a company? How can we make people care more about accomplishing something than about pinching pennies from executives?

My initial ideas:

• Yell
• Blog
• Insist on calling the program expenses / total expenses the “Straw Ratio,” banking on the negative connotation of straw
• Start a project to collect all the meaningful information about charities that we can find in one place
• Hold a protest! We could march around with straw hats on – it would be so symbolic!
• Social networking

Elie and I had lunch with three people representing the Children’s Aid Society (CAS) yesterday. At this point we aren’t used to having meetings without a wiki handy for note-taking – and dealing with corn chowder at the same time intensified the challenge – but here are the highlights of what I remember:

• The meeting started with a more or less total repudiation of the main self-evaluation study we’ve been sent so far, a review of test scores at CAS community schools. I had sent an email on Wednesday summarizing my problems with the study (basically, I think it shows nothing), and we were told that it had been put together on the fly for a donor request and that we shouldn’t focus on it. This is a little disconcerting because:
  • This study was originally sent to me a year ago, in response to my broad request for any and all evidence of CAS programs’ effectiveness.
  • When I repeated the request last October, I was sent basically a re-worded version of the same study (same data, same conclusions, different but equally unconvincing presentation).
  • I think this basically comes down to a disconnect between fundraising and analysis, which I’ll get to in a second.
• CAS claims that there is another, more rigorous study of community schools in the works, which they will send to us.
• They also told us that they have designed a much more ambitious, New Visions-like comprehensive study of how their community schools compare to other schools in the city, but they’ve been unable to get the necessary funding for it because most donors aren’t interested in spending a lot on evaluation. They are going to send us the design of this study as well.
• Jane, who oversees the community schools program, says there is a strong independent research case that the community school model (described here) is the most promising approach to improving education. She has agreed to send us some starting points for looking into this.
• We also talked about CAS’s relationship with other programs trying to do similar things, including New Visions for Public Schools and Harlem Children’s Zone. We came out of this discussion mostly confused – we were told that the people in these organizations know each other (which we believe) and that they work together (which we were unclear on the details of, especially since they don’t share funding and are generally going aboiut the same problems in radically different ways).

So, a lot of what comes out of this depends on what they email us for followup. The main thing I have to say at this point is that it’s been a long, hard slog through the fortress of fundraising (what CAS calls “development”) to meaningful conversations about why CAS chooses the strategies it does.

I’ve been hounding CAS for information longer than anyone else – they were my first ever real charitable donation back in July 2005 – and this is the third personal visit I’ve had with them. The first was a tour of one of their community centers; it was Halloween, and I was invited to participate in the “haunted house” for the little kids (I declined; I didn’t trust myself to draw the line between “fun, enthusiastic scaring the kids” and “causing heart attacks”). The second was a tour of a community school, where I got to interview little girls about the handbags they were making. From the beginning, I’ve been adamant about their sending me all the details they have of their programs’ impact, yet all I’ve seen is the study mentioned above and a longer community school study that is more encouraging but also is something like 10 years old.

It seems clear to me that the whole donor communication process is designed to deal with people who are basically the opposite of me. If what I was looking for was the opportunity to talk to charming people and look at adorable children, I would be absolutely thrilled because CAS has definitely provided that. But as someone who thinks 1000 words are often worth a million pictures, I’m still wondering why getting in touch with people who could more directly address my concerns (a) took so long to happen (b) had to happen over lunch.

And all that said, CAS is one of the best organizations I’ve seen in terms of being responsive to donor concerns and at least trying to answer the questions I’ve been asking.

If donors wanted meaningful information, donor relations would be set up to provide it. At this point in time, it seems that what most donors want is pleasant conversation, cuties, and asparagus-shrimp ravioli. GiveWell would love to help change that.