Sean asks (via email):
What’s your view on whether funders should do research on techniques and then fund organizations that use those techniques or do research on organizations and let them decide on techniques? I was intrigued with your education research post, but was wondering if it might make more sense to find smart dynamic nonprofits who will figure out the best techniques to use and change strategy as more information becomes available.
My literal response is that it depends on the funder’s priorities and techniques – I don’t think there is much to be gained by debating the approach “funders” should take in the abstract. But I want to share how we deal with this question, as naive funders (i.e., not experts in the issues) aiming to serve more naive funders (i.e., individual donors), because we do have a specific philosophy on it and we’d appreciate feedback.
My ideal is to fund at the highest level I can have confidence in, i.e., delegate as many decisions as possible to to someone who I feel confident will make those decisions well.
So, my ideal would be to donate not to a charity, but to another funder. If a major foundation, such as the Gates Foundation, could convince me that they consistently make decisions using (a) a strong process, (b) good reasoning, and (c) subjective/philosophical values that are close to mine, I would give to them and let them do the rest (and get rid of our own, now redundant overhead). This was one of the first things we tried when GiveWell was still a part-time volunteer club. What stopped us was that we couldn’t find a single foundation that publicizes substantive information about how it makes its decisions, why it chooses to do A instead of B, and what evidence there is regarding its past and likely future impact. We couldn’t be confident in the institutions without such information; we couldn’t think of a way to get them to share information, since such institutions generally don’t have incentives that we can affect. So we moved on to trying to find great charities.
Again, the goal was ultimately to find a great organization – one that’s better at what it does than we could ever be, and can make its own compelling, evidence-based case for its effectiveness – and give with no strings attached. In some cases, we found exactly this: for example, the Nurse-Family Partnership‘s outcomes evaluation is available via peer-reviewed publications, its basic model is clearly described on its website, and it provided documents to fill in gaps in our understanding. PSI was a similar case: after some independent checks on its estimates, we felt we could trust its process as a whole, even for activities we haven’t researched.
In other causes, the strongest applicants could provide some pieces of the puzzle, but not the full top-down case for why their approach was the best available. That’s where we had to start looking on our own for information about what approaches are likely to work, and pick organizations that fit with what we had found. There’s a spectrum here. KIPP gave us about 60% of what we needed to have confidence in it, and after some independent analysis, we ended up feeling that it was our best bet. By contrast, our Cause 2 (global poverty) applicants gave us so little to go on that we ended up betting on an approach, more than an organization.
Between blind faith and micromanagement is conditional confidence: trusting an organization to make decisions because of an evidence-based case that they can make them well. That’s our ideal; when it isn’t available, some degree of micromanagement (i.e., picking an organization based on its approach) seems preferable to blind faith.
Comments
So you’re saying that as a former nonprofessional donor, you would rather have hired a “philanthropy manager” the way many people hire a money manager. But if you assumed that was unavailable (or that someone didn’t want to hire someone else), you would like to be able to pick great nonprofits and let them figure out how to run their business. But you also find examples where you are unable to identify great nonprofits and so must pick a technique to fund and they find a nonprofit that follows that technique. Correct?
The reason I ask is because I subscribe to PubHub and I really wish that all the foundation research being pushed through that channel was looking at great nonprofits rather than techniques. I would much prefer to read reports on great nonprofits that are working on education than reports that talk about how to fix education.
Right now, I’m more interested in techniques than specific nonprofits, because I think the range of potential effectiveness varies so widely across causes. i.e. You can have the best run art museum around, but compared with even a so-so organization focused on African health, the latter will, per my current understanding, likely have far more positive impact on measures that I care about.
I also want to spotlight this phrase of Holden’s:
“subjective/philosophical values that are close to mine”
It seems to me that this is quite important but often overlooked. You can boil things down to metrics, analysis of effectiveness, and so on, but ultimately, there are value judgements to be made, and it’s very difficult to be fully objective about these values. Just to toss out a few, how do we choose among causes that might have disparate effects upon:
1) Mortality now
2) Health now (i.e. conditions not generally fatal)
3) Economic well being now
4) Environment now
5) All of these things in the future (in some cases, there are tradeoffs between current and future impacts)
There are many more I could list. Individuals will have different opinions about these, which is why I think the *process* of charity evaluation is almost as important as any specific set of results. i.e. GiveWell, using its own set of values, might arrive at the conclusion that charities A, L, and Z are the most effective. I might read their analysis, but come to somewhat different conclusions based on my values – perhaps A, B, and Y are the most effective charities in my opinion.
Sean: yes, if I could find a “philanthropy manager” I had strong reason to believe would do an excellent job choosing a charity that fit with my philosophical priorities (and the price were right for my relatively small donation), I would opt for that. From what I know, it is generally difficult to assess such a manager without actually working with them (since they generally do not post their past research publicly); I also believe, from conversations I’ve had, that this option is rarely affordable for small-to-medium donors.
I too would like to see more research that is focused on specific nonprofits, instead of just techniques. However, it’s worth noting that we don’t believe a nonprofit can be assessed in isolation from what it does and what the evidence is that it works (more on this idea at this old post.
Phil, I fully agree with you about the importance of subjective/philosophical values. From the beginning, we’ve tried to conduct our research in a way that respects this issue. We divided charities into causes so that we could directly compare (and decide between) charities with similar philosophical goals, while structurally acknowledging that some goals (such as K-12 education vs. global health) are incommensurable. Even having done this, though, we know that there are subjective/philosophical decisions that must be made within causes, and that there are non-subjective arguments to be made between them; every decision is a mix.
We’ve taken the approach of trying to give donors the whole picture – “what you get for your dollar,” with all necessary caveats – rather than just presenting our recommendations. And we’ve had many donors giving to charities that we recommended, even without ranking them 1st in their cause – which to me confirms the importance of giving more than one option and spelling our reasoning rather than just declaring a “winner.” I think this is an important illustration of what GiveWell provides that Gates and other foundations, with their lists of grantees, do not.
We’ve discussed the question of examining nonprofits vs. examining techniques before, but briefly, I don’t believe it’s practical to examine techniques in the abstract, divorced from the specific context and details of their execution. This belief is based largely on our attempt to do so, particularly in the “saving lives” cause – the Disease Control Priorities Project and other sources we consulted don’t have enough information to decide between different approaches in the abstract.
Instead of starting with technique or organizations, why not start with looking at issues, the geography of where these issues are most severe.
For instance, poverty maps can show what areas of a big city, or the world, have high degrees of poverty. Overlays to these maps, such as poorly performing schools, or crime/violence incidents, can show other indicators of a need for social benefit organizations.
From this information donors could choose a cause, then pick what country, what state, what city and what neighborhood, they want to have an impact in. Once they do that there are few choices of where they might put their money. Some of these might be great organizations. Some might be not so great. However, rather than start a new organization in the identified area, it might be better to try to help the non profit doing needed work to improve their ability to do that work.
I’m piloting this concept in drawing volunteers and donors to volunteer-based tutor/mentor programs in Chicago. You can see examples of maps and a searchable database of programs in the Program Locator on the TutorMentorConnection dot org web site.
Using a map to show all of the places where a problem exists, forces us to think of strategies that would distribute resources to make good social services available in a growing number of these places.
Would this change the way donors approach their business?
Dan, I agree that the “top-down” approach you describe can be productive, and I’m familiar with the Tutor-Mentor Connection (and agree that it’s a potentially useful resource).
I don’t think the “top-down” approach can do the whole job, though, as I explain here.
Comments are closed.