This post is the first in a multi-part series, covering how GiveWell works and what we fund. Through these posts, we hope to give a better understanding of our research and decision-making. Links to the other posts:

• How we work, #2: We look at specific opportunities, not just general interventions
• How we work, #3: Our analyses involve judgment calls
• What we fund, #1: We fund many opportunities outside our top charities

Why cost-effectiveness matters

The core question we try to answer in our research is: How much good can you do by giving money to a certain program?

Consider how much good your donation could do if you give to a program that costs $50,000 to save a life versus one that costs $5,000 to save a life (which is roughly what we estimate for our top charities). Giving to the latter would have 10 times more impact. While in an ideal world both programs would receive funding, we focus on identifying the most cost-effective programs so that the limited amount of funding available can make the greatest difference.

The basics

We’ve written in detail here about our approach to cost-effectiveness analysis and its limitations. Our bottom-line estimates are always uncertain, and we don’t expect them to be literally true. At the same time, they help us compare programs to each other so that we can direct funding where we believe it will have the greatest impact.

At a very high level, assessing cost-effectiveness generally involves looking at:

• The cost per person reached. For example, how much does it cost to treat one child with vitamin A supplementation for one year?
• The outcomes of the program. Determining these outcomes often involves examining two factors:
  • The overall burden of a problem. For instance, how many kids who will be reached with vitamin A supplementation would otherwise have died?[1]
  • The effect the program has. For example, how much does vitamin A supplementation reduce mortality rates relative to that baseline, and are there other benefits to providing vitamin A?

We use unconditional cash transfers as a benchmark for comparing opportunities, such that a program is estimated to be “12x cash” if we believe it’s 12 times more impactful per dollar than giving money directly to people living in poverty. In other words, if we estimate that a program is 12x cash, we think donating $100 to that program does as much good as donating $1,200 to a program that delivers unconditional cash transfers.

More detail

We aim to come to an all-things-considered view that involves a variety of complex factors and judgment calls. The evidence we rely on varies in strength but is always imperfect, and our conclusions represent best guesses rather than truth. (More on judgment calls and uncertainty in a later post.)

For our top charities, our cost-effectiveness analyses generally involve dozens of parameters and assumptions. In constructing these analyses, we ask questions like the following:

• How well do the study results generalize to a different time and setting? (That is, what is the evidence’s external validity?) For example, what are current levels of insecticide resistance relative to study conditions, and how do they affect the efficacy of antimalarial nets?[2]
• Are people already receiving the relevant program even in the absence of our funding? For example, how many kids served by a recent grant to Helen Keller International would receive vitamin A supplementation outside of its program?[3]
• If we fund an intervention, such as insecticide-treated nets, how would that affect what other funders spend on it? Would they have funded this program if we didn’t? If we’re influencing them to spend more on this program, what would they otherwise have spent their funding on?[4]
• How would GiveWell funding affect the actions of governments in the countries where grants are funded? For example, will Evidence Action’s work scaling up syphilis testing during pregnancy be successfully transitioned to the government? Would the government take on that program in the absence of Evidence Action’s work, and if so, on what timescale?[5]
• What spillover effects might an intervention have? For example, as New Incentives increases immunization rates, what impact does that have on disease transmission throughout a population?[6]
• How does the impact of averting a death compare to the impact of doubling someone’s income?[7]

The amount of time we spend on a research question depends both on how much progress we believe we can make and how important the answer is to the bottom line. Many considerations are built into the main model, and we make supplementary adjustments for other factors to yield a final estimate.

We aren’t inherently risk-averse in our assessments—we fund the global health and development programs we think are best. That said, we do look for reasonably compelling evidence in order to fund something. Not everything we fund needs to be backed by a randomized controlled trial, but our goal is to help our donors give with confidence. And for our top charities, where we want to give donors an even higher level of confidence in the impact of their donations, we have additional criteria. More on those here.

Qualitative considerations

Qualitative factors that aren’t part of our formal quantitative model can influence our final recommendations, especially when we’re deciding between several programs with similar apparent cost-effectiveness. We bake as much as we reasonably can into our quantitative analyses, but still ask ourselves questions like these: Does this organization have a strong track record? Are there risks to program implementation that we haven’t fully captured? Does this bottom-line estimate match our true beliefs, which might involve relevant information and intuitions that might be hard to quantify? We might recommend grants that we estimate to be slightly below our cost-effectiveness threshold if we think they’re especially strong qualitatively, and vice versa.

A small portion of our grantmaking relies on qualitative reasoning rather than substantial quantitative modeling. For two examples, see this grant for research on cash spillovers and this grant to the Agency Fund. In those cases, the grant investigator believed that the expected impact of the grant justified its cost, but may not have conducted a formal analysis.

Further reading

For a deeper dive into GiveWell’s cost-effectiveness work, see the following materials:

• The cost-effectiveness analyses for our top charities
• Our high-level page on cost-effectiveness
• Examples and discussion on specific grant pages, such as this grant to Nutrition International for vitamin A supplementation in Chad
• This walkthrough of the cost to save a life, using an antimalarial net distribution in Guinea by the Against Malaria Foundation as an example

Putting it all together

Our cost-effectiveness models aim to incorporate as many of the important considerations as we reasonably can, and much of our research effort goes toward building these models and refining the key parameters.

Cost-effectiveness matters because we aim to direct funding to the highest-impact opportunities we can identify. With our current cost-effectiveness threshold, this means recommending programs we believe are at least 10 times as impactful per dollar as unconditional cash transfers to people living in extreme poverty.

In future posts, we’ll share more about the details of our research and the types of opportunities we fund. In the meantime, please comment or email us at info@givewell.org with any questions!