This week, The Chronicle of Philanthropy wrote an article about the creation of the Alliance for Effective Social Investing. We wholeheartedly applaud Steve Butz and other members of the Alliance for their efforts, and really hope they succeed. There are too few organizations focusing on the effectiveness of charitable programs, and we’re excited to see their first results.
I’ve briefly looked at the survey they plan to send to nonprofits and here are some quick thoughts. Ultimately, the survey focuses on procedures and processes as opposed to impact and results and therefore has two problems:
- Charities often say they track outcomes even if they don’t. This happened to us consistently last year. (You can view our Round 1 application we sent to international charities linked on this page along with all the materials we received. In particular, look at the answers to section III on the application. )
- There are no specifics about what each organization does and what effects it has. All the questions are abstract about whether or not each organization tracks their outcomes. Donors need to know what impact they can expect from their donations not whether or not a charity has a “process in place” to track outcomes.
[As an aside, I’d really appreciate a tool that simply lists all of an organization’s programs. If there’s one type of information I’d like for all charities, it’s a simple specification of what they do and where they work. This is not currently available anywhere. (Guidestar offers very brief summaries of a charity’s programs off its 990, but nowhere can I see the specifics of each of its activities.) We’re currently working on building such a tool for international aid, with the help of some great volunteers.]
Finally, I’m concerned that this survey won’t accomplish its most important goal: distinguishing between effective and ineffective organizations. We’ve analyzed the Children’s Scholarship Fund and the Nurse-Family Partnership. Both organizations collect a large amount of data about their clients, and I believe they would each answer the Alliance’s survey identically. Nevertheless, we believe that NFP is running an effective, impactful program and we strongly recommend them; we think that CSF’s strategy is marginally effective (i.e., not making a substantial difference) at best and ineffective at worst.
Any useful evaluation tool has to distinguish between two programs like CSF and NFP. If it doesn’t, it falls short in the most important way.
All of this criticism is offered in the hope of dialogue and improvement. We’re rooting hard for our competitor, and if they want any help or information from us, they’ll get it.