That makes sense to me.

I had a go at evaluating my favourite charity, the School of St Jude, using a three step approach.

* Is it appropriate? Does it it fill a real need?
* Is it effective? Does it get good results?
* Is it efficient? Does it makes maximum use of its resources?

It was easy to find evidence on the first two points, but harder on the third.
You can check out my attempt here.
http://schoolstjude.blogspot.com/2007/05/wheres-evidence.html

The third point is really what GiveWell is about — distinguishing those charities that are using their resources to the best advantage. I think you are probably well on the way towards developing a pro forma for gathering the relevant information.
Have you looked at the kinds of program evaluation done for international agencies like USAID? Have you talked with Evaluation specialists? The American Evaluation Association is the professional organisation for evaluators. They have all kinds of neat tools like the Hierarchy of Outcomes framework, and they distinguish between Process Evaluation (how well it does it), Outcome Evaluation (what results it gets) and so forth. Their website is www.eval.org/

Cheers
Gillian

What’s unclear to me is why you *don’t* think the Straw Ratio is irrelevant. We think it is, by and large (may be useful in weeding out flagrant crooks, but it’s no good in distinguishing mediocre charities from excellent ones). We think that trying to understand a charity using its Form 990 is trying to get blood from a stone, and that rather than insist on making something out of the information that’s easy to get, we need to get the more complex, difficult-to-obtain - but relevant - information about what charities actually do and what the evidence is that it works.

I’m not quite sure where this thought fits with your posts, but it feels important to me. It relates to the proportion of a charity’s money that is spent on admin and fundraising, and the proportion spent on programs. This is your straw ratio, I think.

My view is that the world needs charities that spend almost everything on programs (high straw ratio?) AND IT ALSO NEEDS charities that spend relatively more on fundraising (lower straw ratio).

The penny dropped for me when my husband said one more time that he liked to support MCC (a Mennonite charity) because everything went to programs. At the same time I read that some charities found it worthwhile to pay backpackers on a commission basis to intercept people in the street and sign them up. The charity got 50% of the amount donated. In my husband’s view this was an inefficient charity and he would not want to give to it. But, apparently these street intercepts are particularly successful at getting donations from young males - a group that is commonly under-represented amongst donors.

I realised that MCC doesn’t have to spend any money on fundraising because the money just funnels in through church networks.

We need a variety of charities, with a variety of fundraising methods, and a range of straw ratios, because different donors respond to different overtures. 50% of something is better than 100% of nothing.

Does this mean ’straw ratio’ is irrelevant? I don’t think so. There must be a zone of inefficiency below which it becomes less effective. I wonder how one would develop guidelines for that?!!

This suggests that your assessments may not be applicable to charities that put extra resources into gathering donations from hard-to-reach donors. Not many charities have a mainline connection to a community of givers like the Mennonite church.

How will your assessment process take account of this? Perhaps it should be recognised in your guidelines? Perhaps it already is.

Thanks for providing this open forum for thinking about these issues.
Gillian

We won’t be the only funder in the world, and we can’t and won’t do it all, especially in our first year. We’re focusing on adding value in a way that no one else is: finding what’s proven, effective, and scalable, and supporting it with total openness.