We previously listed our five chief criteria for GiveWell Labs, an arm of our research process that will be open to any giving opportunity, no matter what form and what sector. This post further discusses the third of these criteria: “accountability.”

We’re OK with funding a project that might fail, but it’s very important to us that we be able to recognize, document, publicly discuss, and learn from such a failure if it happens.

This is the area in which we feel most strongly that current philanthropists are coming up short: they’re failing to learn (or at least, to help others learn) from their track records. For a simple example, take the issue of sustainability in developing-world aid.

• A common goal of a philanthropic program is to see the government - or another funder - take over at some point, leading to lasting impact that doesn’t depend on continued funding.
• We’ve seen many different approaches to accomplishing this. For example, VillageReach initially paid entirely for its own project, with the hope that the government would switch over to its model once the proof of concept had been established; now that that idea has failed to pan out, VillageReach is asking for more cost-sharing from the government up front as it re-implements its model.
• Yet there appears to be so little evidence on what sorts of projects have and haven’t achieved sustainability in the past that one paper by prominent scholars argues that the whole idea of sustainability is an “illusion” (an argument endorsed by William Easterly).

Most of the funders we’ve talked to don’t seem to have very clear senses even of their own organizations’ track records (both good and bad). Even if funders are learning internally and informally from their own failures, they aren’t learning from each others’.

We believe that we have an unusual commitment to public and documented discussion of whether our giving ends up accomplishing what we hope. We’ve been releasing regular updates on VillageReach, the organization we directed the most money to in 2010, and we intend to do the same with our current top charities. These updates are specific and honest about both good and bad news (and there has been a fair amount of the latter).

Read the rest of this entry »

We previously listed our five chief criteria for GiveWell Labs (a new arm of our research process that will be open to any giving opportunity, no matter what form and what sector). This post further discusses the first two of these criteria - “upside” and “high likelihood of success” - and the tradeoff between them.

Read the rest of this entry »

We’re starting a new initiative, GiveWell Labs, an arm of our research process that will be open to any giving opportunity, no matter what form and what sector.

This post lays out, very broadly, what qualities we are looking for in giving opportunities. Future posts will elaborate on each of these criteria, and we will also discuss how we think these criteria apply to specific areas of philanthropy. Readers will hopefully be left with a strong sense of our beliefs and biases and what we’re looking for.

The main things we’re looking for in a giving opportunity are:

1. Upside: we’d prefer to fund projects that have the potential to go extremely well. Projects aiming to demonstrate a model that can be scaled up, generate new scientific knowledge that can be used by many others, or put a program in place that eventually becomes self-sustaining independent of philanthropic support all have “upside.” Simply aiming to deliver insecticide-treated nets using established delivery methods does not have much “upside” (though it may score well on many of these other criteria).
2. High likelihood of success: we’d prefer to fund projects that are very likely to do a respectable amount of good per dollar. The “evidence base” of a project - i.e., the set of past well-understood events that can be used to put its likelihood of success in context - is key here. Obviously this criterion will often be in tension with the “upside” criterion; the ideal for us is a project that has both, i.e., a project that’s both very likely to do some good and has some possibility of doing enormous amounts of good (we think that giving to VillageReach in 2010 fit into this category).
3. Accountability. We’re OK with funding a project that might fail, but it’s very important to us that we be able to recognize, document, publicly discuss, and learn from such a failure if it happens. We thus have a strong preference to fund projects with specific and meaningful deliverables that will give us a strong sense of whether things are going as hoped (as well as permission to publish updates on these deliverables).

   We are relatively new to giving and plan to be doing a lot more of it in the future, so making sure that early projects are learning opportunities is crucial.

4. People we’re confident in. We prefer to fund projects where we are impressed by and confident in the people involved. However, our take on how to evaluate people seems to be different from that of some other funders; we’ll elaborate in a future post.
5. Room for more funding. We prefer to fund projects that would not happen without our funding. This means that we aren’t actually looking for the “best ways to spend philanthropic funds”; we’re looking for the “best ways to spend philanthropic funds that aren’t already on the agendas of other funders.”

We don’t have an explicit formula for weighing the above criteria above against each other. Broadly speaking, we’d prefer to fund an opportunity that is strong on all of the following: (a) at least one of #1 and #2; (b) at least one of #3 and #4; (c) #5. (Note that we do not feel the approach of estimating ‘expected good accomplished’ for each project, and simply ranking by this metric, is a good way to maximize actual expected good accomplished; for more, see the body and comments of a recent post on expected-value calculations.)

One more consideration is leverage: we prefer projects where our funding mobilizes more funding from other givers as well, thus multiplying the impact of our funds in some sense. However, we think this is far less important than the criteria listed above. We’d rather fund a great project all on our own, and leave other funders to spend on their own projects, than get a 5:1 or 100:1 funding match from others on a project that is weak on the above criteria.

If you think we’re missing any important impact-related criteria, please let us know.

We’re starting a new initiative, GiveWell Labs, an arm of our research process that will be open to any giving opportunity, no matter what form and what sector.

One of the major challenges of this initiative (as mentioned in the previous post) will be remaining systematic and transparent despite the very broad mandate of GiveWell labs. It’s core to GiveWell that the thinking behind our recommendations

• Comes from reasoning and principles that are applied consistently, not from whims.
• Is transparent, i.e., interested people can read up on why we made the decisions we did and judge our thinking for themselves, with as little need as possible to have trust in us.

This post lays out our plan for accomplishing these.

While we reserve the right to change plans mid-stream, our basic planned process is:

1. Sourcing general ideas. We plan to cast a wide net initially, looking pretty much anywhere we can for general funding ideas - and/or organizations - that might be promising. Key sources will include:
   • Our existing research, particularly our list of the health interventions with outstanding track records and cost-effectiveness.
   • Conversations with anyone who might be a particularly good source of these kinds of ideas - particularly officers at major foundations and academics who do work related to philanthropy - as well as the referrals we get from these people.
   • Going over the work of anyone else who seems to similarly have a goal of identifying top opportunities for doing good. Examples we’ve found so far: Copenhagen Consensus, Back of the Envelope Guide, and the winners of sector-agnostic competitions for limited funds (Echoing Green, Ashoka, Skoll, etc.)
2. Going from general ideas to specific proposals. We will maintain a ranked list of the most promising ideas from step 1, and for high-ranked ideas, we will attempt to find the people and/or organizations who can make (and, potentially, execute on) specific proposals. At this stage we’ll just be looking, in each proposal, for rough ideas of (a) costs; (b) what people/organizations will do the execution; (c) what the basic plan is.
3. Detailed investigation of proposals. We will maintain a ranked list of the most promising proposals from step 2, and for the most promising ones, we will conduct in-depth investigations similar to those we’ve always conducted for promising charities. These will include in-depth conversations with the relevant people/organizations; conversations with others in their space, particularly those who have funded them or chosen not to fund them; site visits when applicable; and requesting technical reports, budgets, and other materials when applicable.
4. Recommending and funding proposals. We will attempt to get any outstanding submissions from step 3 funded. We will have $1 million to spend at our discretion if we can raise no other funds, but we expect to be able to raise more if we succeed in finding great giving opportunities.

At this point we are most interested in funding others’ ideas, and have a preference for cases where the implementing organization is the same as the organization that hatched the idea and strategy. We have the impression that much philanthropy works differently, as foundation staff design their own strategies and treat grantees to some extent as “contractors” for carrying it out; this model does not currently appeal to us, but we plan on further investigating the history of philanthropy (particularly success stories) to see whether there is more promise in this approach than we’d guess.

With a project as broad and open-ended as GiveWell Labs, we expect to make a lot of guesses and judgment calls regarding promising areas/ideas/projects, and we expect to use heuristics and take shortcuts in narrowing the field. We don’t commit to detailed reviews of every idea or every proposal, or to the use of objective formulas to decide between them. (The same applies, and always has applied, to our existing research on top charities.)

However, a core value of ours is that interested parties - no matter who they are - ought to be able to understand as much as possible of (a) which options we considered; (b) why we chose the ones we chose. To this end, we plan on publishing:

• Extensive discussion of the values and beliefs that are relevant to which sorts of sources we use and which areas we focus on investigating. This discussion will take place via future blog posts. We hope that anyone who reads these posts will understand why we look at the areas and sources we do, and if we aren’t accomplishing this we hope our readers will comment to let us know.
• A list of the sources we use to generate ideas (step 1), along with detailed notes from particularly noteworthy conversations. Our goal here is to cast the net wide, so if you know of promising sources of ideas that fit with the values/principles we’re expressing and you don’t see us using them, we encourage you to comment.
• Discussion of the general beliefs (and relevant facts) that lead us to discard certain ideas from step #1 while moving forward to step #2 on other ideas, again via the blog.
• A full list of the proposals we consider (step #2), along with notes from discussing these proposals.
• Discussion of the general beliefs (and relevant facts) that guide our choice of particular proposals (step #2) to move to the “deep investigation” phase (step #2), again via the blog.
• Full details of the materials we collect via deep investigation (step #3) and our notes on the strengths and weaknesses of each giving option that makes it to this stage.

We will withhold information when necessary to respect confidentiality agreements. However, we will make our best effort to obtain clearance for - and share - all important/relevant information. This is the same policy we’ve used in charity investigations, and while some information remains confidential, we’ve still published the vast majority of the information we have (enough so that our views generally don’t need to be taken on trust).

GiveWell Labs is different in substantial ways from our existing research (which continues). However, we feel that we will be able to preserve the most important aspects of GiveWell:

• A focus on finding the best giving opportunities in terms of positive impact, rather than in terms of telling compelling stories or making donors feel good.
• Recommendations that are transparent enough to allow outsiders to draw their own conclusions and give meaningful feedback.

If we can preserve these things while working in a more open-ended way, we’ll be able to find better giving opportunities and to demonstrate our principles’ broad applicability. This means there will be fewer reasons than ever for other large givers to be keeping their own processes opaque.

We previously announced GiveWell Labs, a new arm of our research process that will be open to any giving opportunity, no matter what form and what sector. Here I share a bit more of the thinking behind why we’re doing this.

Our goals are twofold:

Find better giving opportunities. When we laid out our main goals for 2011, #1 was finding more great giving opportunities, and our possible strategies for doing so involved (a) broadening our scope (b) considering project-based funding. With GiveWell Labs, we are doing both simultaneously.

• We’ve previously come across groups that might have been able to offer great giving opportunities, if we had selected a specific project and provided all the funding necessary to carry it out. However, we couldn’t recommend them to individual donors, not knowing whether $1000 or $1 million would come in as a result. Now, we’ll plan to go back to these groups, open to anything. If we do end up wanting to raise specific amounts of money, this will be a more complex endeavor than simply publishing a recommendation on our website and saying “Give here,” but we now have enough connections to major donors and enough sense of our audience of smaller donors that we think it will be worthwhile to try.
• We’ve previously come across interesting funding opportunities that didn’t fit neatly into the causes we had chosen to focus on. This won’t be an issue for GiveWell Labs.
• Examining opportunities with the above qualities (project-based and/or outside the sectors we’re experienced in) will be hard to do systematically, and will be by nature a bit experimental. That’s why we’re allocating only 25% of our research time to GiveWell Labs, with the remainder allocated to carrying out our existing research process (which has some restrictions but is more established and systemized). However, we expect the things we learn through GiveWell Labs to eventually shape the evolution of our more systemized research process.

Position ourselves to advise seven-figure donors.

When analyzing our own impact, we’ve noted that it comes disproportionately from large donors. (We influence more $100 donors than $10,000 donors, but the ratio is far under 100:1, so the $10,000 donors end up accounting for the lion’s share of our money moved.)

This seems logical to us, when considering that GiveWell is a “niche product” - we don’t appeal to large interconnected groups of people, but the rare people who do resonate with our work resonate very strongly with it, and give a lot based on it. The logical implication is that our greatest potential for impact may come from very large donors - and we need to be positioned to be useful to these donors.

The research we’ve done to date - recommending direct-aid charities that can absorb arbitrary amounts of funding - seems best suited to those giving under $1 million per year. When we encounter people who give more, they generally are interested in funding whole projects at once, which gives them options that simply aren’t open to our standard research process. That means our current product is a poor fit with the people who may represent our most potentially impactful audience.

We need to address this issue, and GiveWell Labs will allow us to do so. The $1 million in pre-committed funding is coming from large donors who will be able to give more if we find them great opportunities. More importantly, GiveWell Labs will allow us to move closer to having the same universe of options that seven-figure donors have, which will hopefully improve our ability to connect with and influence seven-figure donors.

GiveWell Labs is issue-agnostic, i.e., we are not restricting our work to particular areas of philanthropy (such as international aid, climate change, etc.) We will focus on what we consider the most promising areas, but we will be potentially open to anything.

There are clear disadvantages to issue-agnostic giving:

• The more different sorts of projects we allow ourselves to consider, the greater the challenge of sorting through them in a coherent, principled, systematic way. It will be particularly challenging to make sure we are applying principles consistently, rather than giving based on whims.
• There are conceptual advantages to “specializing” in particular sectors over time. Doing so means having the ability to
  • Learn from past successes and failures in an area.
  • Make contacts in an area.
  • Gather evidence about the most promising approaches in an area, particularly informal/qualitative evidence (e.g., site visits).

However, issue-agnostic giving has advantages as well.

• First and foremost is that when you’re new to giving, you can’t tell where the best opportunities are going to be. Picking a “sector” could be the dominant determinant of how effective your giving will be; taking a guess and sticking with it, therefore, seems very dangerous for a donor seeking to maximize impact. (Note that I’ve changed my view of the most promising cause as I’ve learned more about the different causes we’ve studied.)
• Even when you’re not new to giving, the highest-impact sectors can change rapidly and chaotically as new philanthropists come on the scene. Choosing to focus on developing-world-oriented medical research may have been a great idea before the Gates Foundation came along, but I’m guessing that opportunities in this area have fallen drastically since, as the Gates Foundation has attempted to fund the best ones.
• There may be outstanding opportunities that get overlooked by other funders because they don’t fit neatly into a particular “sector.” I think this is possible in today’s environment, simply because issue-agnostic giving is so rare.
• Regarding the above-mentioned advantages of specialization:
  • We are hoping for a relatively “low-touch” approach to funding: we seek people with ideas but not funding, and we seek to provide funding and not other kinds of support. We hope this approach will diminish the need for us to become “experts” in any given sector.
  • We hope to be very communicative with other funders and people with relevant expertise/experience. We won’t recommend a funding opportunity without getting as many relevant opinions as we can. If people with different specialties are open and communicative with each other, it mitigates the need for funders to have all the expertise themselves.
  • In practice, we will probably find ourselves focusing on certain sectors - not out of a pre-commitment, but because these sectors appear particularly promising to us. This is especially true in light of the fact that we prefer (as we always have) to fund things we can understand well - our past experience and established knowledge do matter. So being issue-agnostic doesn’t actually preclude specializing; it just means that any specialization will happen gradually and out of a desire to maximize impact, rather than being driven by up-front choices of particular sectors.

Bottom line - at this point in our development, we think the advantages of issue-agnostic giving outweigh the disadvantages for us.

The research we’ve been doing for the last couple of years has been constrained in a couple of key ways:

• We’ve pre-declared areas of focus (based on our guesses as to where the most promising charities would be found), and disqualified charities for recommendations on the basis of their being “out of scope” (though we’ve been gradually broadening our scope).
• We’ve needed to decide which organizations to recommend without being able to say in advance how much money would go to them as a result. This has led to challenges with the question of “room for more funding.” We’ve had to find charities that could essentially use any amount of funding (large or small) productively, and this has drastically narrowed our options.

We’re now launching a new initiative within GiveWell that will not be subject to either of these constraints. We plan to invest about 25% of our research time in what we’re calling GiveWell Labs: an arm of our research process that will be open to any giving opportunity, no matter what form and what sector.

Through GiveWell Labs, we will try to identify outstanding giving opportunities (whether they’re organizations or specific projects), publish rankings of these giving opportunities (separate from the top charities list we maintain using our existing research process) and try to raise money for these opportunities. Donors have pre-committed a minimum of $1 million to the GiveWell Labs initiative, meaning that we will have at least $1 million to commit to our choice of projects even if we are able to raise nothing else. (We expect to raise more if and when we find great giving opportunities; the $1 million has been committed based on donors’ trust in our ability to find such opportunities.)

Our existing work of finding outstanding international aid charities - using a more systematic process - continues. Over the coming year, we expect to spend about 75% of our research time on our existing work of finding outstanding international aid charities, and 25% of our research time on GiveWell Labs. Note that our “standard” process continues to gradually evolve and broaden its scope, and hopefully will come to incorporate insights gained through the work on GiveWell Labs. The distinction between the two may even dissolve over time. But at this time, GiveWell Labs is the arm of our process that is open to any giving opportunity, no matter what form and what sector.

In future blog posts, we’ll be giving a lot more information about this project, including:

• More on why we’re moving in this direction at this time, and why we think a less-constrained, exploratory arm of our research process will help us find better giving opportunities.
• Our planned process for finding great giving opportunities through GiveWell Labs, and what you can expect from us in terms of transparency.
• The main qualities we’re looking for in a funding opportunity (when unconstrained by the form or sector of the opportunity), and why we’re looking for them.
• The areas we think are most likely to yield great giving opportunities, and why.

In the meantime, if you know of any giving opportunities that are (a) not already funded or likely to be funded by others; (b) outstanding opportunities to have a large positive impact, please let us know.