Elie and I had lunch with three people representing the Children’s Aid Society (CAS) yesterday. At this point we aren’t used to having meetings without a wiki handy for note-taking – and dealing with corn chowder at the same time intensified the challenge – but here are the highlights of what I remember:

• The meeting started with a more or less total repudiation of the main self-evaluation study we’ve been sent so far, a review of test scores at CAS community schools. I had sent an email on Wednesday summarizing my problems with the study (basically, I think it shows nothing), and we were told that it had been put together on the fly for a donor request and that we shouldn’t focus on it. This is a little disconcerting because:
  • This study was originally sent to me a year ago, in response to my broad request for any and all evidence of CAS programs’ effectiveness.
  • When I repeated the request last October, I was sent basically a re-worded version of the same study (same data, same conclusions, different but equally unconvincing presentation).
  • I think this basically comes down to a disconnect between fundraising and analysis, which I’ll get to in a second.
• CAS claims that there is another, more rigorous study of community schools in the works, which they will send to us.
• They also told us that they have designed a much more ambitious, New Visions-like comprehensive study of how their community schools compare to other schools in the city, but they’ve been unable to get the necessary funding for it because most donors aren’t interested in spending a lot on evaluation. They are going to send us the design of this study as well.
• Jane, who oversees the community schools program, says there is a strong independent research case that the community school model (described here) is the most promising approach to improving education. She has agreed to send us some starting points for looking into this.
• We also talked about CAS’s relationship with other programs trying to do similar things, including New Visions for Public Schools and Harlem Children’s Zone. We came out of this discussion mostly confused – we were told that the people in these organizations know each other (which we believe) and that they work together (which we were unclear on the details of, especially since they don’t share funding and are generally going aboiut the same problems in radically different ways).

So, a lot of what comes out of this depends on what they email us for followup. The main thing I have to say at this point is that it’s been a long, hard slog through the fortress of fundraising (what CAS calls “development”) to meaningful conversations about why CAS chooses the strategies it does.

I’ve been hounding CAS for information longer than anyone else – they were my first ever real charitable donation back in July 2005 – and this is the third personal visit I’ve had with them. The first was a tour of one of their community centers; it was Halloween, and I was invited to participate in the “haunted house” for the little kids (I declined; I didn’t trust myself to draw the line between “fun, enthusiastic scaring the kids” and “causing heart attacks”). The second was a tour of a community school, where I got to interview little girls about the handbags they were making. From the beginning, I’ve been adamant about their sending me all the details they have of their programs’ impact, yet all I’ve seen is the study mentioned above and a longer community school study that is more encouraging but also is something like 10 years old.

It seems clear to me that the whole donor communication process is designed to deal with people who are basically the opposite of me. If what I was looking for was the opportunity to talk to charming people and look at adorable children, I would be absolutely thrilled because CAS has definitely provided that. But as someone who thinks 1000 words are often worth a million pictures, I’m still wondering why getting in touch with people who could more directly address my concerns (a) took so long to happen (b) had to happen over lunch.

And all that said, CAS is one of the best organizations I’ve seen in terms of being responsive to donor concerns and at least trying to answer the questions I’ve been asking.

If donors wanted meaningful information, donor relations would be set up to provide it. At this point in time, it seems that what most donors want is pleasant conversation, cuties, and asparagus-shrimp ravioli. GiveWell would love to help change that.