Objections and concerns about our new direction

GiveWell has recently been taking on activities that may seem to represent a pretty substantial change of direction, especially for those who think of us as a “charity evaluator focused on saving the most lives per dollar spent.”

  • Within global health and nutrition, we’re considering restricted funding for specific projects, not just recommendations of particular charities.
  • We’re also exploring other causes that are extremely different from global health and may be far less amenable to measurement and “cost per life saved” type calculations, such as meta-research.

When discussing these activities, we’ve lately been encountering a couple of different objections and concerns; this post discusses the objections and our responses. In a nutshell:

  • Some are concerned that we’ll lose our objectivity if we get involved in providing restricted funding: we’ll be tempted to rank the groups following our plans ahead of the groups following their own plans, and we’ll thus lose the quality of being a disinterested third-party evaluator. We believe we can draw a meaningful line between “charities we recommend for unrestricted funding” and “plans we have designed,” leaving individual donors to decide whether they’d rather take our recommendation unconditionally or only follow our advice in the areas where we’re disinterested; we also believe that being open to providing restricted funding is necessary and important, and justifies the resources we’ll be investing. More
  • Some are concerned that by going into new causes, we’ll be spreading ourselves too thin. Understanding global health is already an ambitious and difficult goal; it’s been suggested that we should “stick to our knitting.” We feel that sticking to global health, when we see other causes as potentially more promising, would be out of line with our fundamental mission and value-added as an organization that seeks to help people do as much good as possible. More
  • Some are concerned specifically about new causes that don’t lend themselves to measurement and cost-effectiveness calculations (such as meta-research). It may be difficult to remain systematic and transparent about how we make decisions in these more speculative areas. We recognize this concern, but feel that we can remain systematic and transparent even where measurement is difficult or impossible; furthermore, we feel that we must find a way to do this if we are to have a strong case that philanthropy as a whole (not just sub-sectors of it) should be more systematic and transparent. More

Despite the concerns and risks above, we feel that the benefits of our new direction outweigh them. A major input into this view is the feeling that sticking to our old process would be extremely unlikely to result in finding more outstanding giving opportunities within a reasonable period of time; this is something we will be writing more about.

That said, we do recognize the concerns and risks, and we are interested in others’ thoughts on them.

The risk of losing our objectivity

To date, all of GiveWell’s recommendations have involved unrestricted support to existing organizations. Because of this, we can be pointed to as a “neutral third party” that recommends organizations based exclusively on impact-related criteria. But we’re now contemplating doing what a lot of major funders do and helping to set the agenda for a funded organization, through the mechanism of restricted funding. If we did this, we might have difficulty being neutral between (a) projects that we help design and (b) charities that are simply asking for unrestricted funds, not contracting with us. In fact, we might be tempted to eschew (b) entirely and focus exclusively on designing – rather than finding – giving opportunities.

One important principle here is that we will draw a clear line between organizations we recommend for unrestricted funding and projects designed by GiveWell. We don’t know exactly how the visual presentation will work yet, but we have agreed on the principle that there will be a clear distinction – including on our higher-level and frequently-accessed pages – between GiveWell-designed projects and recommended charities.

Of course, there is still a risk that recommendations for unrestricted funding will have “soft conditions” (i.e., that it will be clear to charities what activities they have to carry out in order to earn or maintain recommendations); this is something that has always been true, though I think the situation is somewhat mitigated by the nature of the room for more funding analysis we perform. (Our analysis asks for predicted charity activities based on total unrestricted funding, not based on GiveWell-specific funding. The expectation is that if GiveWell-directed funding falls short of expectations and the gap is made up by other funding, the activities will still be as outlined; this hopefully provides charities an incentive to project the activities they would most like to carry out, rather than projecting the activities they hope will most appeal to GiveWell specifically.)

Even with a clear distinction, there could still be a reasonable concern that GiveWell will over-allocate resources (in terms of investigative capacity) to designing its own projects, as opposed to finding great organizations. We recognize this concern, but wish to note that – philosophically – we greatly prefer unrestricted to restricted funding, and greatly prefer a “hands-off” to a “hands-on” approach. We don’t have the capacity to actively manage projects ourselves, and we believe projects are likely to work out better when they are run by people who fully buy into them (as opposed to people who are fulfilling the requirements of restricted funding).

It’s partly because of this philosophy that we’ve stayed away from restricted funding to date, and we remain highly cautious about it. We would prefer to stick to unrestricted funding and may never in fact deal in restricted funding.

Yet it is worth noting why we are considering restricted funding now in a way that we haven’t before. Our impression is that major funders frequently make extensive use of restricted funding; as a result, the existing landscape consists of many charities whose agendas are set partly or fully by external funders.

  • We’ve been surprised by the disconnect we’ve observed in which there is a large number of promising interventions but few charities that focus on these interventions (in a way such that additional dollars will mean additional execution).
  • More generally, we’ve been surprised that in the majority of conversations in which we ask an organization what it would do with more unrestricted funding, it has no clear answer, and prefers instead to tailor its answer to our priorities.

Practically speaking, charities have to focus on what they can fund; and in today’s world, it seems possible that agendas are largely set by funders. Our ideal role would be to “free” great organizations from restricted funding, allowing them to carry out promising projects that they can’t fund otherwise. However, it seems possible that there are too few charities for whom funding would make this sort of a difference, and there is thus some argument for our taking the sort of active role that other funders do.

Finally, by being open to restricted funding, we’ve come across some opportunities that are similar to “unrestricted funding” in most relevant ways, but that structurally involve restrictions and that we couldn’t have come across using our former approach. For example, we’re currently considering the idea of funding particular parts of UNICEF that work on particular interventions that we’re interested in. This wouldn’t involve laying out our own plan, and it would involve getting money to a specific team and leaving the use of the funds at their discretion; however, we could not find this sort of giving opportunity by talking to general UNICEF representatives and asking what they would do with more unrestricted funding. In some sense it may be appropriate to think of UNICEF (and other organizations like it) as a coalition of teams with their own priorities rather than as a single team with a single set of priorities; so in this case a gift that is formally restricted may have many of the desirable qualities of an unrestricted gift. To avoid confusion, we will still distinguish any recommendations along these lines from purely unrestricted gifts, as laid out above.

The risk of spreading ourselves too thin

We still have a lot to learn about global health and nutrition (as indicated by, among other things, our continued learning from VillageReach’s progress). It has been suggested that we should “stick to our knitting,” focusing on the areas of giving in which (a) we’ve built up the brand we have; (b) data and feedback loops tend to be unusually good for the nonprofit world, facilitating learning.

In response, I’d observe:

  • GiveWell is still a young organization. I believe we have attracted attention more for “bringing a different perspective and approach to giving” than for “being experts in global health” (the latter certainly does not describe us). We recognize that we’re taking some level of risk in moving into new areas, but we also believe that taking risks and staying open to new approaches is a major part of what makes GiveWell what it is and that part of “sticking to our knitting” is retaining this quality. We believe that GiveWell and the donors who use our research will be best served by our continuing to do whatever we believe will lead to the best giving opportunities, continuing to change course as much as necessary to facilitate this, and continuing to bring a different perspective and approach to giving – not continuing to focus on global health.
  • While we currently believe that global health is the most promising cause given the information available, we are not confident in this conclusion. We believe that other causes are potentially promising as well, and if we never investigate them, we will be failing in our mission of finding the best giving opportunities possible.
  • We are currently expanding our staff; we expect that we will invest at least as much time in global health over the next few years as over the last few (while also investing time in other causes).

The risk of losing transparency and systematicity as we move away from highly measurable interventions

We have written before that the cause of “global health and nutrition” seems unusually well-suited to meaningful measurement and metrics (by the standards of the nonprofit sector). When working within this cause, we have been able to be relatively clear about our process and about what distinguishes a recommended from a non-recommended charity. There is some risk that as we tackle other causes, such as meta-research, we will have less of an evidence base to go off of; our goals will be further out; we will have to use more intuition and may therefore become less systematic and transparent.

We believe this is a real risk. However, we also believe that (a) the best opportunities for good giving don’t necessarily lie in the domains with the highest measurability (though there is something to be said for measurability, all else equal); (b) we have reached the point where we feel we can explore causes such as meta-research in a way that – while not as systematic as our work on global health – will still include a great deal of public discussion of how we’re thinking, why we recommend what we do, what the key assumptions are in our thinking and recommendations, and how our projects progress over time.

We have long advocated that philanthropists should be more systematic and transparent in their work. If our own systematicity and transparency applies only to the cause where measurement is easiest, we won’t have a very strong case; if, however, we can consistently bring an unusually level of systematicity and transparency to every cause we examine (even those that are less prone to measurement), we will have much more potential to change philanthropy broadly rather than just a single sector of it.

The benefits of our new direction

The above discussion addresses potential concerns over our new direction. We have previously discussed the substantial benefits: finding the best giving opportunities possible and reaching the largest donors possible, both of which are core to our mission. Dealing with the above issues – keeping a focus on recommending unrestricted funding when possible, covering new causes without overly detracting from continued progress on the causes we know well, and remaining systematic and transparent – will be a challenge, but we feel that it is well worth it, especially because we feel we are reaching the limits (for the moment) of our old approach. (We went through a large number of charities in 2011 and are skeptical that we will find new contenders for our top charities, using that basic methodology, anytime in the near future.)

We welcome further comments and criticisms regarding our new approach.

Comments

Objections and concerns about our new direction — 6 Comments

  1. Well stated. I think you should continue with your plans. The whole notion of “objectivity” is very over-emphasised. It is normally just the appearance of “objectivity” that is complained about. The thing that makes you good is you take a principled stand. I don’t really know how objective you were before, and I don’t really care. You provided well reasoned cases based on your beliefs.

    I understand the argument for you becoming more biased. I am fine with that.

    We would benefit from a great deal more study of charity outcomes. I’m not really all that concerned about the objectivity of those doing it. Those that just make excuses for whatever they like won’t be of much value. Those that have a strong bias but put in good analysis and are not afraid to be open with outcomes (not just present those that support there cause) are great.

    Keep up the good work.

  2. Tell you what, it happens again and again that you chaps make a judgement on something, and I think “well, they’ve made the wrong call there”, and then you explain your reasoning and I think “well, they’ve made the the right call there”.

    After reading this post I’ve completely backflipped on my previous thoughts and I can now see why it’s necessary for you to go in this direction, and that it’s a good idea; particularly if you can maintain your previous activity with the present workforce. I’m with you.

    But to add a thought about this new direction Holden, your very first blog post read that the reason people would read this was that in your experience, you “know that no matter how pressed for time I am, I still read random crap online. Now I want to be your crap.” So that has worked great for me, and congrats on becoming my and many other people’s crap, but you’re right that it is ‘crap’ that people read online. It’s snappy crap, it’s humorous crap, and while it’s always been more thought provoking than typical internet crap, for a long time this blog was snappy and humorous; it was full of short snippets of things that happen in charity today, with pretty interesting perspectives on them. That was tops for getting people interested, for making them feel like they know you, and ultimately for getting people to follow your work. But have a look at the length of any of the later blog posts, and it’s pretty clear that they’ve changed to being pretty solidly in depth, they take effort to process, and they certainly don’t fit into what I’d think of as typical, internet crap. I get the reasons that it’s gone this way, and people who have been reading this blog for years will get that too and will be happy to come along for the ride. I suppose I just wonder if I would have begun reading this regularly if the first thing I came across here felt more intellectually heavy than the neuro lecture I had just come out of.

    Thanks for explaining your new direction, thanks for the longer posts that we get now, please keep it going, but I’m sure that you and your staff still have those interesting epiphanies which used to constitute blog posts – and I for one would love to hear them.

    Cheers, Brenton.

  3. “we’re considering restricted funding for specific projects, not just recommendations of particular charities….When discussing these activities, we’ve lately been encountering a couple of different objections and concerns; ”

    Givewell funding projects is a very different kind of work than what you are known for and what is currently described on your webpage. It really is a different direction…

    Givewell’s unique advantage in advising donors, that there is not the chance of, or even the appearance of, a conflict of interest, is going to be lost….

    I wonder where you can point us to where the discussions took place as to the objections and concerns that you mentioned….and there been any discussions online regarding this issue that we can see…..

  4. Thanks for the thoughts, all.

    Brenton: we recognize that there’s a tradeoff when it comes to the blog tone. Given what GiveWell is today, we think it’s important to emphasize credibility above entertainment/opportunities to get attention (some discussion of this our February post on high-level priorities).

    Dean: we are very early in the process of the shift described and do not yet plan any specific recommendations of restricted funding. So when you ask whether there are any online discussions of the issue, the answer is that this post is the potential beginning of such a discussion (it isn’t an announcement of a finalized shift). We’ve laid out the reasons we think the benefits outweigh the costs and would be happy to consider responses.

  5. I’m concerned about this remark:
    ” especially because we feel we are reaching the limits (for the moment) of our old approach. (We went through a large number of charities in 2011 and are skeptical that we will find new contenders for our top charities, using that basic methodology, anytime in the near future.)”
    This sounds like you are turning away from what you have established your reputation by doing. It’s been a big gap in your work, in my view, that you have not been able to evaluate the major aid organizations. I understand the difficulties, but I thought you were now starting to look at specific projects within major charities, rather than than trying to evaluate the entire organization. That sounds like something very worthwhile, and to do it properly would require considerable time and effort. I hope you’ll keep doing it, and expand your efforts in that direction.

  6. Peter, thanks for the comment. We are indeed beginning to investigate major organizations at the project level (the example of UNICEF is given above). Investigating specific projects means being open to restricted funding – this is one of the two prongs of what this post means when it refers to our “new direction.”