Holden and I have been reviewing applications for Cause 5: help disadvantaged adults become economically self-supporting. This is what we’ve learned so far and what we’re just generally wondering about.

We initially envisioned this cause pretty broadly, but we’ve found that there are a critical mass of organizations that follow the same basic model: take disadvantaged (largely unemployed African-Americans and Latinos) people, train them in both a specific vocation and in how to get a job (e.g., resume writing and interviewing help), place them in a job, and follow up with them to track their progress and help them retain their opsition.

We want to know how much good each organization effects for the dollars that donors spend. For us, that means understanding where each individual would have been had Organization X not helped him and where he is because they helped him. Roughly, that equates to understanding:

• The population each organization serves. Although each organization follows the same basic model for improving economic opportunity, they differ on whose lives they try to improve (though all focus on low-income, largely-minorities). On one end of the spectrum, some recruit the best and the brightest, taking only 25% of applicants (who they put through a rigorous interviewing process testing their motivation) and requiring a high school diploma or the equivalent. On the other hand, some organizations attempt to help all those who walk in, trying to train homeless, unemployed, poorly educated (often those with only at fifth-grade level for reading and math) and many who have a criminal record and a history of substance abuse.
• The outcomes they achieve. We (and the organizations who’ve applied) have three ways of tracking outcomes: people who get jobs, how long they keep those jobs, and how much they make.
• The cause of the change. Understanding the population served and the outcomes achieved will allow us to better understand each organization’s effect on those it serves. For example, it seems likely an organization which selects only 25% of those who apply (and interviews particularly for motivation) and accepts only those who have graduated high school will likely have graduates with better outcomes (higher paying jobs held for longer periods of time) than those organizations which select previously homeless, poorly-educated clients. But, Organization 1 is not achieving those outcomes because it has a better program. It’s achieving better outcomes because it selects people who would likely have gotten there (or at least improved) without any help in the first place. How many of these people do you think would have been fine without any help? Or at least, with extremely minimal help – could these organizations serve a lot more people (and basically get the same results) by cutting way back on services?
• Cost. Well, last but not least, what does it cost each organization to effect that change?

Right now, we’re leaning toward passing the organizations that provided us with outcomes data (how many of their clients got jobs, and how long they stayed in them) to the next round – if an organization didn’t even tell us this (when we asked for it quite clearly in our application), evaluating their results is just going to be too much of an uphill battle. With these orgs, we’re going to get all the data we can get from them about what results they got and what population they served. Then comes the hard part: figuring out what the “benchmark” should be for that population (i.e., how they likely would have done without this help). NYC has some pretty detailed census data, so we hope we’ll be able to get at least some idea of what’s “normal” for people in different neighborhoods at different levels of education.

Well, that’s what’s on my mind. What do you think?