Regarding complete solutions: our metrics refer to outcomes (what ultimately happens due to a charity’s activities), not outputs (what it literally does). A charity that just distributes food will cause a certain number of people to have food, water and shelter who wouldn’t have otherwise (they would have had water and shelter but not food); a charity that just distributes water will also do this; and because having all three of these things is so qualitatively different from having just two, we think it’s appropriate to use this as the yardstick for comparing the food charity vs. the water charity.

I recognize how debatable that is: there is a theoretical imperfection because we aren’t counting more minor benefits at all. That’s because we don’t want to make a conversion factor between helping someone a little and helping them a lot. In deciding what a charity’s “ultimate goal” is, we’ve used philosophy and intuition - for example, deciding that the real reason you’d want to distribute food is ultimately that you’re trying to increase the # people who have ALL their basic needs met. There’s no ironclad way to do this, which is why we throw it open to discussion. If you disagree with any of the particular philosophical/intuitive choices we’ve made, please jump in.

One possibility is that to save lives or educate people, you are not expecting any novel solutions that don’t exist already in the for-profit sector, in which case, there is only funding as a reward for organizational effectiveness. Another is that there are enough charity dollars that.

RE: choice of causes, Africa and NYC, etc.: I didn’t explain that in this post, but I gave some coverage of it earlier . Briefly: we need an achievable goal, so we have narrowed the field in ways that are far from deductive. They are based on our personal values; on the structure of the sector (i.e., most organizations serving NYC focus on NYC, whereas many organizations serving Kenya have broad mandates of serving Africa); on what we think will appeal to a wide enough range of donors to make our site a good (if not comprehensive) resource for its first go-round; and on what we think our approach is best suited to (for various reasons, advocacy is not a very good fit). This doesn’t mean we think these are the only worthwhile causes, but they are where we’re focusing for now.

We have cut some causes since our earlier post. A more complete overview of our scope will be available in our business plan, which will be online soon.

RE counterfactuals: I, along with basically everyone else I’ve talked to, completely agree with you that it’s impossible to really say what would have happened if not for a charity’s activities. But conceptually, this IS what we want to know, and it is useful as a regulative ideal. When I see that a job training program led to X people getting jobs, I want to know if there is any data on a comparable set of people who didn’t take the program - so I can start to get an idea of the program’s actual impact.

As I say in the post, we know that we will never actually calculate any of our metrics. But it is still important to articulate them, because it helps us to figure out what information we’re looking for. When a charity says to us, “We’ve given training to over N people,” we know not to stop there because we know what we would ultimately and ideally like to measure.

I would think that if we agreed on this purpose for the metrics, we would agree that the counterfactuals are appropriate. The question is whether we have the same vision of what the metrics are and aren’t useful/relevant for. I appreciate your thoughts.

I’m curious: Did the decision to focus on an entire continent, rather than on the poorest countries in that continent, also flow from some kind of metrics? Africa has some of the poorest countries in the world, but East Timor (with a per capita GDP of less than $500) can give many of them a run for the money, in a manner of speaking. Why not fund in East Timor? Wouldn’t the decision to fund in Africa also need to depend on your ability to identify and work with good organizations on the front lines? to leverage your investments? to be given some assurance that public policies will not undermine the good work you manage to do? etc. etc.

I’ll assume then that your decision to fund in Africa is a given, based on a careful consideration of many factors — as were the decisions to fund causes 4 – 7, say. This is important because there’s simply no deductive argument that takes you from your desire to “remove barriers to opportunity in wealthy societies, focusing on New York City,” to these four causes. Why not put all your money into public policy advocacy, for example? Can any metrics show that these are the kinds of investments that will most effectively remove barriers to opportunity?

My biggest caution about your metrics would be not to have them depend too much on things that would not have happened had the charity not existed. These counterfactuals are very difficult to understand and assess. Why not simply measure what these charities in fact accomplished?

I’m reminded here of the story of Saint Rose who prayed fervently to God that the people of Lima be spared a devastating earthquake. The story goes that the earthquake did not in fact occur, and that Rose started being venerated as a saint shortly after her intercession. Would the absence of an earthquake not have happened had Rose not intervened? These are murky waters indeed.

I offer these thoughts in the spirit of one who, as you know, is fairly agnostic about many of the ways that metrics are used and appealed to. This doesn’t affect in the least my admiration for the work that you and your friends are doing, and the spirit of generosity that animates it. The latter trumps all other considerations, in my view.

Best wishes for your very worthy project.