The GiveWell Blog

AMF and Population Ethics

One important input we consider in our top charity recommendations is our quantitative cost-effectiveness estimate of each charity: an estimate of how much good the charity accomplishes per dollar. We have written before about the many challenges of constructing and interpreting such cost-effectiveness estimates. One such challenge is the problem of how to assign a numerical “score” to various good outcomes such as averted deaths or increased incomes, so that they can all be compared on the same scale. This involves answering questions such as:

  • What would I do if I had a choice between doubling the income of three individuals in extreme poverty for a year and averting the death of a child under the age of 5?
  • How many deaths of children under the age of 5 would I need to avert to accomplish as much good as averting the death of one adult?

GiveWell staff members enter their estimates for such values-based comparisons in our cost-effectiveness analysis (CEA). Along with empirical estimates of program costs and outcomes, this forms the basis for the numerical comparisons we make between our top charities.

Recently, a post on the Effective Altruism Forum raised concerns about our recommendation of AMF specifically. It argues that the value of averting deaths for children under five depends on one’s view of population ethics – a branch of philosophy that asks questions like “Is it good to avert a death if it has no long-run impact on the total population?” – and that some approaches to population ethics would imply a substantial discount to our cost-effectiveness estimates. We’ve chosen to respond to this argument at length because we think it is interesting, serves as a good example of the thorny issues we grapple with in estimating cost-effectiveness, and gives the opportunity to explain some aspects of our 2016 cost-effectiveness model that are not widely understood.

In this post, I will

  • Explain basic concepts in population ethics and how they inform the way people think about the value of averting death
  • Summarize arguments from the post mentioned above, which argues that people with certain views on population ethics should substantially discount our cost-effectiveness estimate of the Against Malaria Foundation (AMF) to better reflect their values
  • Walk through the reasoning behind our current estimate of AMF’s cost-effectiveness and explain why I believe it’s compatible with most plausible accounts of population ethics, so discounts as aggressive as those suggested in the linked post are likely inappropriate.

Population ethics and the value of averting death

Population ethics is a branch of philosophy which outlines some major considerations that influence how people value averting deaths. It is defined in the link as “the theory of when one state of affairs is better than another, where the states of affairs may differ over the number of people who ever live.” Population ethics deals with questions such as:

  • Can it be morally good or bad to create new people who would not have otherwise lived, or is creating people always morally neutral (assuming they do not affect people who are already living)?
  • Is the badness of death lessened if someone else will be born to “replace” the person who died?
  • What makes (premature) death bad? Is it because the individual misses out on the years of happy life they would have otherwise lived? Is it because they had a strong preference to live that was violated? Is it because surviving loved ones will grieve? Is it some combination of these?
  • Is death worse at some ages than at others?

A population ethics stance is a set of beliefs that inform how to compare different states of the world by answering these and similar questions. Staff members’ implicit or explicit stance on population ethics guides the way they quantify the value of averting death in the CEA. This in turn can greatly affect how the cost-effectiveness of life-saving charities such as the Against Malaria Foundation (AMF) and Malaria Consortium’s seasonal malaria chemoprevention program compares to the cost-effectiveness of life-improving charities such as GiveDirectly and charities implementing mass deworming programs.

In a recent post on the Effective Altruism Forum, Michael Plant argues that AMF is unlikely to be highly cost-effective under four plausible population ethics stances:

  • Total hedonic utilitarianism: The stance that the best state of the world is the one with the most total happiness or fulfillment. Under total hedonic utilitarianism, averting someone’s death is only good to the extent that it results in more overall happiness experienced in the world (regardless of who experiences it). Thus, if a child dies, and as a result the family has another child they would not otherwise have had, the badness of the child’s death comes only from the gap between their death and the new child’s birth and the grief and other negative effects experienced by family members. (This is assuming the child who died and the new child who is born as a result would live similarly happy lives.) This counterintuitive result is referred to as “the replacement problem” in this post. Michael argues that a total utilitarian should substantially discount GiveWell’s cost-effectiveness estimate for AMF to account for the replacement problem.
  • The deprivation stance: According to the deprivation stance, the badness of death is equal to the number of years of life that the individual who died misses out on as a result of their death: in other words, their life expectancy at the time of death. Unlike in total hedonic utilitarianism, this badness is not lessened even if someone else is immediately born as a direct result of the death. Michael argues that this is the point of view which is most favorable to AMF, but those who hold this view should still be more interested in other interventions, such as life extension.
  • The time-relative interest stance: Quoting from Michael’s original post,

    [The time-relative interest account] holds the badness of death depends, roughly, on the extent to which it frustrates the person’s interests in continuing to live. This captures the [intuition] many people have that it’s much more important to save a 20-year old than a 1-minute old foetus because, in essence, that 1-minute old foetus hasn’t developed enough to miss out on life.

    Note: the time-relative interest stance, unlike the previous two stances on population ethics, doesn’t suggest a simple formula that would capture most of the “badness” of death if all the empirical facts were known. Michael argues that those who subscribe to the time-relative interest stance should “reduce [GiveWell’s] estimate of AMF’s effectiveness by however much [they] discount child deaths compared to adult ones.”

  • Epicureanism: This stance holds that death in itself cannot be bad for the individual who died, because as soon as they die, they cease to have morally relevant interests. Thus, a given death is only bad due to suffering in the process of death, and the grief and other negative effects it has on survivors. Epicureanism also does not immediately suggest a simple formula that would estimate the badness of death given all the empirical facts.

I believe Michael suggests discounts on our cost-effectiveness estimate of AMF that would be too aggressive for most people’s value systems. I explain this further in the rest of the post, but I want to first make a general point: if you are concerned your stance on population ethics does not align with the GiveWell median, I believe it would be better to download an editable copy of GiveWell’s CEA and input your own values for rows 7, 53, 63, and 64 on the “Parameters” sheet than to attempt to multiply GiveWell’s bottom-line cost-effectiveness estimate by some factor to account for expected differences in population ethics. This is because the cost-effectiveness model is complex and in parts unintuitive, and I don’t believe it will be straightforward to guess how your stance on population ethics differs from the implicit stance of the median GiveWell staff member.

Summary of considerations against discounting our cost-effectiveness estimate

Here are the key reasons why I believe most people should likely not reduce GiveWell’s stated cost-effectiveness estimate of AMF as much as Michael suggests:

  • GiveWell’s 2016 cost-effectiveness model suggests that the median GiveWell staff member believes that averting the death of a young child averts ~8 disability-adjusted life-years, or DALYs (“Parameters”, C14), whereas last year the GiveWell median opinion was ~35 DALYs averted, as Michael states in the post. While GiveWell does not take an official stance on population ethics as an organization, I believe this change is a result of some staff members leaning away from an explicit “discounted age-weighted expected years of life lost” model of value to a more complex and less precise model of value that does not map intuitively to the concept of DALYs as used in health economics. (More.)
  • GiveWell’s current cost-effectiveness estimate does not predict that a $3500 donation to AMF will, on expectation, prevent the death of one young child. It predicts that a $3500 donation to AMF will on expectation cause a combination of outcomes that are equivalent in value to saving the life of one young child, according to the median staff member. (More.) According to staff median estimates, the benefits of AMF are driven
    • ~37% by preventing deaths of children under the age of 5
    • ~36% by preventing deaths of people age 5 and over
    • ~27% by providing future financial benefits due to improved young-childhood development, similar to deworming charities.
  • It is not clear that the “replacement problem” described in the original post fully applies to AMF, and if it does, it’s not clear that conservative staff estimates of value (see A) don’t already account for it in the 2016 model. (More). Because of this, a total utilitarian may end up concluding that AMF is more cost-effective than we suggest. (More.)
  • I don’t believe it’s obvious than even an Epicurean would consider AMF to be dramatically less cost-effective than the GiveWell CEA suggests — although it is likely that they would discount our cost-effectiveness estimate somewhat. (More.)

I provide more detail on these four points in the rest of the post. I’m going to reference specific cell numbers and sheets in our November 2016 CEA throughout.

DALYs are a flexible, non-literal measure in our CEA

From Michael’s original post:

[GiveWell claims that saving a child’s life is worth] 35 ‘QALYs’ (Quality-Adjusted Life Years), which is [a] more technical way of saying it creates 35 years of healthy life for the beneficiary.

As mentioned above, the new median value is 8 DALYs per under-5 life saved. (See the “YLL per Death” sheet in our CEA for examples of calculations that take a more explicit deprivation or time-relative interest stance, which were more common in the past.)

More generally, in the context of our CEA, I don’t think it makes sense to treat “DALYs” as straightforwardly denoting “years of life lost by the person with the disease + years lived with disability due to the disease.” Particularly in 2016, staff members interpreted the “DALYs averted per death of an under-5 averted—AMF” parameter (“Parameters”, B63) as an opportunity to quantify how holistically “bad” the death of a young child is. This interpretation can take into account broader considerations such as:

  • For total utilitarians, concerns of replacement
  • Harm caused by parental grief
  • Potential economic harm caused by sickness and death
  • Time-relative interest considerations which weigh young children less highly than older people
  • Considerations around the loss of the child’s individual identity and the child’s desire for life

Note: I’m not sure what considerations different staff members actually did take into account; I’m just observing that our interpretation of the DALY unit is broad enough to allow for a range of such considerations.

Because DALYs have been somewhat divorced from their rigid, concrete meaning, I think comparing how a given staff member filled in row 7, row 53, row 63, and row 64 in the “Parameters” sheet (all of which are value inputs) is more informative for understanding their values than looking at any one of those inputs in isolation. For example, staff member Sophie included comments on her inputs in O7, O63, and O64.

Also because of this ambiguity, if you want to make adjustments for your beliefs on population ethics, I believe it would be better to download an editable copy of GiveWell’s CEA and input your own values for rows 7, 53, 63, and 64 in the “Parameters” sheet than to attempt to discount GiveWell’s bottom-line cost-effectiveness estimate by some factor to account for expected differences in population ethics.

In particular, I think that with certain reasonable assumptions about replacement rate (calculated here), a total utilitarian or someone with a time-relative interest account of value may end up concluding that AMF is more cost-effective than our estimates suggest (see below).

Over 60% of AMF’s expected benefits are not driven by saving young children

From Michael’s original post:

GiveWell estimate[s], although this is not to be taken too seriously, [that] $3,500 to AMF saves a child’s life.

However, the cost-effectiveness estimate for AMF produced by our staff median parameters (B74) is around $3400 per young life saved-equivalent, not $3400 per young life saved.

That is, our cost-effectiveness estimate does not predict that if you give $3400 to AMF you will, on expectation, prevent the death of one young child. It predicts that if you give $3400 to AMF, you will on expectation cause a combination of outcomes that, according to the values of the median GiveWell staff member, are morally equivalent to saving the life of one young child. Specifically, the expected benefits of AMF are split between:

  • Preventing the deaths of children under 5
  • Preventing the deaths of people 5 or older
  • Improving the expected future income of young children, similar to deworming

Note that benefits in the 2015 model were also not exclusively driven by preventing deaths of children under the age of 5: cell M20 in the “GW medians” sheet of the 2015-2016 CEA implies that the median staff member believed that one year of bed net coverage was almost as effective as one year of deworming for improving expected future income of children. The primary new addition this year is accounting for the deaths of older people.

Example calculation

This section walks through the math of how we achieved our “cost per young life saved”-equivalent figure; please consider this section especially optional.

According to our median cost-effectiveness estimates (all cell numbers taken from the “Bed Nets” sheet of our CEA):

  • For every ~$9,161 spent, on expectation one marginal death of a child under the age of 5 is averted (cell B55). Each under-5 death prevented gets a weight of one “young life equivalent” unit. According to the median GiveWell staff member, averting the death of a child under 5 averts about 8 DALYs (“Bed Nets”, B57).
  • For every ~$37,391 spent, on expectation one marginal death of a person age 5 or over is averted (multiply cells B61 and B63 to get the ratio of 5-or-over deaths averted / under-5 death averted, and then divide the cost per under-5 death above by this value). According to the median GiveWell staff member, each 5-or-over death prevented gets a weight of 4 “young life equivalent” units (“Bed Nets”, B62).
  • For every ~$500 spent, on expectation you have produced financial utility equivalent to increasing an individual’s ln(consumption) by one unit for one year (“Bed Nets”, B69, multiplied by 500)—which means ~doubling their income for a year. According to the median GiveWell staff member, averting 1 DALY is equivalent to increasing ln(consumption) by one unit for three years (“Bed Nets”, B72). Combining that with the value of 8 DALYs per death of a young child above, this means that each unit increase of ln(income) gets a weight of 1 / 24 “young life equivalent” units.

This means that a $37,391 donation to AMF would, in expectation:

  • Prevent the deaths of ~4.08 children under the age of 5, i.e. ~4.08 young-life equivalent units
  • Prevent the death of ~1 person over the age of 5, i.e. ~4 young-life equivalent units.
  • Have a financial benefit equivalent to increasing the ln(income) of 37.391 * 2 = 74.782 people by one unit, i.e. 3.12 young-life equivalent units.

Putting that together we have $37,391 / (4.08 + 4 + 3.12) = $3338 per young life saved-equivalent. (Note: this is not exactly equivalent to the value in “Results”, R4, likely because of a combination of rounding and small adjustments that are not accounted for here.)

Do parents have additional children to replace those who die at a young age?

In 2014, GiveWell commissioned David Roodman to write a report on the possible causal link between mortality and fertility, which is linked in Michael’s post:

By GiveWell’s own estimates, the effect of AMF is that it leaves total population numbers largely unchanged. I call this the ‘replacement problem’ for total utilitarians because, in these replacement cases, they can’t say there’s much (or any) value in saving lives apart from the effects of bereavement on the parents.

However, I think the picture presented in David Roodman’s analysis is more nuanced than this. From the conclusion of that report:

As mentioned at the outset, we should expect that where fertility is most controlled, typically indicated by total fertility of about 2 births/woman or less, that the volitional replacement effect is large—that for every child’s life saved, parents avert one birth. That births/woman averaged 2.7 in developing countries as a whole in 2005–10, and that the number has probably fallen more since, suggest that most couples today are engaging in family planning. Meanwhile, where the fertility transition does not yet appear to have occurred the replacement effect is likely much smaller. The studies I find most informative tend to corroborate this theory, indicating near-full replacement among a group of relatively affluent countries; partial replacement in a context where fertility had begun to decline but still had far to go (Uttar Pradesh); and no replacement in an area of continuing high fertility (Northern Ghana).

I spent a little bit of time trying to come up with a reasonable range of estimates for the fertility replacement rate in the areas AMF operates in, primarily informed by whether it seems like those areas are converging to a fertility rate of 2 births per woman. This should definitely not be considered the final word on the complicated topic of replacement rates, nor should it be considered an official GiveWell estimate. However, I found it to be helpful as a personal exercise, and it may be valuable to some people to see my reasoning:

According to GiveWell’s latest update on top charities, marginal funds from GiveWell-influenced donors will go toward AMF’s Execution Level 1 gap. We believe at Execution Level 1, AMF will likely use marginal funds to do more work in the places it had worked previously: primarily Malawi, Ghana, and Uganda. According to this tool, estimates for fertility 2010-2015 for these countries were:

  • ~5.25 lifetime births per woman in Malawi
  • ~5.91 births per woman in Uganda
  • ~4.25 births per woman in Ghana

(To get the above estimates, I chose the following options in the linked tool, in order: “Total fertility (children per woman)”, “Malawi, Ghana, Uganda”, “2010-2015”.)

I also calculated the average fertility rate across these three countries (weighted by population) to be approximately 5.2 in this spreadsheet (see sheet “AMF Countries”).

Furthermore, according to the United Nations’ probabilistic predictions as shown in this tool, it seems none of these three countries are likely to converge to two births per woman until past 2050. Based on David’s analysis, this would imply that family planning is not pervasive in the countries where AMF will likely operate over the next couple of years, and so the death of a child would lead to significantly less than one additional expected birth in the family. After a very quick scan of the table in the Conclusions section of the mortality-fertility report, one study struck out as potentially most informative for estimating replacement rates in such countries: Bhalotra and van Soest 2008.

Bhalotra and van Soest 2008 was a study in Uttar Pradesh, India, which used data from 1963-1999 to estimate that the death of a child under one month old is followed by 0.37-0.52 extra births. In the “India” sheet of my spreadsheet, I calculate that average fertility over the period of 1960-2000 was approximately 4.78. Assuming fertility in Uttar Pradesh 1963-1999 was similar, it seems the average fertility today in Ghana, Malawi, and Uganda is slightly higher than in the population studied in Bhalotra and van Soest 2008 (5.2 vs 4.78). This implies the expected replacement rate should potentially be even lower than 0.37-0.52.

I want to emphasize that these calculations are extremely rough, but they support my general impression that it is not clear that replacement is close to 1 in countries where AMF is likely to use its marginal funds in 2016.

A total utilitarian may consider AMF more cost-effective than GiveWell does

From Michael’s original post:

I should note it’s not particularly important what the exact replacement ratio is. If it turns out AMF causes parents to have 0.5 fewer children for every 1 life it saves, the total utilitarian should still [halve] AMF’s effectiveness.

This doesn’t seem obvious to me; there are a couple of flavors of total utilitarian I could imagine, and in general they don’t seem to assign a substantially lower value of DALYs averted per death of a young child averted than the GiveWell median, even assuming a 75% replacement rate (which is significantly higher than my best guess). The values used below come from the results of modifying the following values on the “YLL per death” sheet of our CEA: “Discount rate” (C6), “Age-weight parameter beta” (C7), and life expectancy at age 5 (I8 and J8).

  • You could count additional years of life created without discounting over time or weighting by age. After the deprivation stance, this is perhaps the view of population ethics that is most favorable to AMF. Assuming a life expectancy of ~65 years at age 5, preventing the death of a five-year-old in Malawi, Ghana, or Uganda would avert 65 * 0.25 = ~16 DALYs if there is a 75% replacement rate.
  • You could discount by time and weight by age. This is the sub-type of the total utilitarian viewpoint that is least favorable to AMF. Suppose the “Discount rate” is set to 0.03 and the “Age-weight parameter beta” is set to 0.04 (giving the highest weight to a year of experience as a 25 year old and a lower weight to a year of experience as a young child). If life expectancy at age five is still 65, then causing an additional five-year-old to exist would produce the equivalent ~36 years of healthy life for a 25 year old. Adjusting for replacement, preventing the death of a five-year-old in Malawi, Ghana, or Uganda would avert 36 * 0.25 = 9 DALYs, similar to the GiveWell median of 8.
  • You could have an in-between view (for example, weighting by age without discounting or vice versa), which would produce in-between estimates for DALYs per young death prevented.

It seems that because GiveWell staff members’ median estimates for the value of averting the death of a young child are already relatively conservative from a pure total utilitarian standpoint, it may be too aggressive for most total utilitarians to discount further due to the replacement problem, unless they believe replacement rates to be close to 1.

I believe broadly similar considerations would apply to certain kinds of time-relative interest viewpoints.

An Epicurean may still believe AMF produces substantial value per dollar

From Michael’s original post:

The fourth option is the Epicurean view, named after Greek philosopher Epicurus. It holds that there’s nothing good about creating someone and that death doesn’t harm anyone: once someone is dead, there is no them for anything to be bad. Obviously the process of dying can be painful. The point Epicureans make is that nothing is good or bad for you once you’re dead. On this account, the badness of death consists only in the suffering felt by the living.

For Epicureans, the value of their $3,500 donation to AMF is that it stops a family from having to grieve for a lost child.

I think there are significant costs for survivors beyond the emotional cost of grief associated with the deaths prevented by AMF:

  • For the deaths of children under the age of 5: if parents “replace” their lost child by having another, we must include in the DALY burden estimate:
    • The monetary costs of having a new baby and raising that baby to the age the deceased child was at the time of death, which may significantly impact the consumption or savings of a poor family
    • The strain on the mother’s health and productivity associated with the course of a normal pregnancy
    • The expected harm due to the possibility of serious complications in pregnancy — if this results in the mother’s death, it could result in serious permanent harm to her surviving dependents (see the next bullet point)

    The above costs are higher the higher you believe the replacement rate is.

  • For the deaths of people over the age of 5, particularly if they are parents or heads of households, we must account for:
    • The loss of the productive income they provide to the family
    • Other harm caused to dependents due to the loss of their care and guidance — it’s plausible this is very long-lasting
  • For all malaria deaths, we must account for the costs of seeking medical care to attempt to prevent the death and the costs of funeral rites, both of which might be a large strain on a poor family’s income

In the comments section of the original post, Michael also suggests that it’s implausible that grief alone could impose a significant welfare cost here:

[Replacement] doesn’t say anything about the parents. Total utils should account for that too, but note how much of the value of the intervention replacement removes. You thought you were giving a child 35+ years of life and preventing parental suffering, but now you’re just (in effect) doing the [latter]. If parental suffering is equivalent to taking away 1 year of happy life away from each parent (IMO, v unlikely), then AMF is equivalent to 2 happy years rather than 37+.

I run through some calculations here http://www.plantinghappiness.co.uk/the-questionable-importance-of-saving-lives/

It’s not clear to me that the non-tangible costs to the parents should be assigned a value of less than 1 DALY total, particularly under the broad and flexible conception of DALYs that is used in the GiveWell CEA. I think there are two plausible ways we could try to quantify this from the parents’ perspective:

  • Preference utilitarianism: How many years of their lives would parents trade away to prevent their young child from dying?
  • Hedonic utilitarianism: How long do parents grieve after the death of their young child, and how intensely on average do they experience this grief?

If you have a preference utilitarian theory of value, then it seems plausible that averting the death of a child could be equivalent to averting multiple DALYs to a parent. One parent has replied to the linked comment saying they would likely trade multiple years of their life to prevent the death of their two-year-old child, and this doesn’t strike me as a very unusual sentiment. Parents have also been known to sacrifice their life or take large risks to save their child.

However, Michael’s theory of value appears to be hedonic utilitarianism, as explained in this comment:

I’m thinking hedonically and am leaning on the literature on hedonic adaptation….few [life] events have a long term impact on happiness, either positive or negative.

I am not familiar enough with the literature on bereavement, subjective well-being and hedonic adaptation to have an informed view on how long parents typically grieve for the death of a young child, or a good sense of how intense the subjective experience of grief is. I find it plausible that the negative effects on parents’ subjective well-being could be relatively moderate and short-lived, and I also find it plausible that these effects could be extreme and long-lasting.

My colleague Luke Muehlhauser studied the literature on subjective well-being several months ago, so I asked him for his impressions. He replied:

It’s hard to say. First, the strongest designs used for studies of subjective well-being (SWB) and life events are panel studies (for a review see Luhmann et al. 2012), which makes causal inference quite tricky, even given recent econometric innovations. Second, the outcome measures typically used in SWB studies are not as well-validated as (e.g.) patient-reported outcomes used in health care (PROMIS) or the measures typically used in educational testing, especially for use across cultures and over long periods of time (as in studies of SWB and life events).

That said, if we cross our fingers and hope that the available panel studies are very roughly capturing what’s going on, we can make some guesses. I haven’t seen panel studies on SWB and the loss of a child, but perhaps we should expect the SWB effects to be similar as with the loss of a spouse, or perhaps somewhat smaller than that, especially in areas with a high rate of under-5 mortality. The Luhmann et al. meta-analysis of prospective panel studies on SWB and the loss of a spouse says (p. 605), roughly, that loss of a spouse is indeed quite bad for SWB initially, that pre-event levels of “cognitive well-being” (cognitively-assessed life satisfaction) are typically achieved within a couple years, and that adaptation is surprisingly rapid for “affective well-being” (feelings of happiness/sadness), with pre-event levels achieved within a couple months. So loss of a spouse is bad, but (according to Luhmann et al.) less bad than (e.g.) unemployment. That said, I should add that I don’t personally trust these underlying studies, nor Luhmann et al.’s method of combining them.

All told, I would guess an Epicurean would likely choose a lower value for “DALYs averted per death of an under-5 averted” than the GiveWell median of 8 (“Parameters”, C63), but I am unsure whether it would be a dramatic downward adjustment, particularly if the Epicurean places relatively more weight on parents’ stated or revealed preferences compared to their subjective experiences, or if they are relatively skeptical of academic research on subjective well-being. For example, my value of 4 DALYs per under-5 death averted (“Parameters”, E63) seems within the realm of plausibility for an Epicurean. This along with my other inputs suggests that I should believe AMF is approximately as cost-effective as GiveDirectly (“Results”, D12).

Conclusion

If you have strong beliefs about population ethics, and are interested in donating to organizations serving the global poor that meet our criteria, I think it would be valuable to download an editable copy of our CEA, override cells C7, C53, C63, and C64 in the “Parameters” sheet with your own values, and then view column R in the “Results” sheet to see what charity cost-effectiveness estimates and rankings that would imply. I’ve outlined some considerations that may apply to total utilitarians and Epicureans above.

If you don’t have a strong stance on population ethics and are wondering what the original critique should imply about whether your giving decisions, two main things are worth keeping in mind:

  • According to the median GiveWell staff member, over half of the benefits of AMF are driven by saving the life of people aged 5 and older or by improving future incomes for young children, rather than saving the lives of children under 5 (“Bed Nets”, B78-80). If you otherwise agreed with the median staff member’s values but believed that averting a young child’s death averts ~0 DALYs, AMF’s cost-effectiveness would be reduced ~37%. If you believed that averting death in general has ~0 value but agreed with the median staff member’s empirical and moral beliefs about improving future incomes, AMF’s cost-effectiveness would be reduced ~73%.
  • The median staff member’s estimate of 8 DALYs averted per young death averted appears to be within the realm of plausibility for multiple stances on population ethics, including total utilitarianism (where it seems to be more on the low end) and Epicureanism (where it seems to be on the high end). I would guess that it is also within the range of many interpretations of the time-relative interest theory of value.

Comments

  • Kelly Witwicki on December 14, 2016 at 10:39 am said:

    If what a person cares about is suffering (if you are an “Epicurean” and “total hedonistic utilitarian”), AMF comes out to something like 0.8 to 2.7 days of a continuous malaria attack spared[https://www.facebook.com/notes/kelly-witwicki/comparing-the-impacts-of-robust-interventions-for-farmed-animals-and-the-global-/151673711956989]. (The note linked uses the $3,500 figure, so that is adjusted for what I have only on reading this post learned is the actual amount per death averted, $7,358 if I’m correct — $37,391/(1 person over five + 37,391/9,161 people under five).)

    I for one find DALYs useless, as they assume being dead is bad, so making complicated calculations with them is unhelpful. All I care about is suffering (or stated otherwise, happiness, welfare, or well-being), and we can get more directly to that, as in the note linked above.

    I find it odd that a lot of people like to bring up the costs to the parents of the lost child, as though this means that AMF is still cost-effective for people who care about reducing suffering, without even making a back-of-the-envelope calculation for that suffering, which is very simple:

    Grief for a parent in the region (who is likely more prepared for the loss than a parent in ours) + suffering caused by additional financial burden (if we think they will only have another child to replace a lost one) + suffering caused by additional pregnancy (if we think they will only have another child to replace a lost one) / $9,161.

    Those speculative effects seem far to un-measurable to meet GiveWell’s bar of robustness, and including them at all makes me wonder why we would do so while ignoring similarly (or even less) speculative concerns in other cause areas on account of that weakness of evidence and low measurability. But in any case, say the grief, combined with that other suffering, is as bad as one person having a continuous malaria attack for an entire year — which anyone who has both lost a family member and had a severe flu would probably argue against, so this is charitable — then we’d have an additional 365 days of a malaria attack/$9,161 = 0.04 days of a malaria attack spared for a dollar.

    So the suffering reduced per dollar is hardly affected by those considerations, and remains extremely small compared to e.g. top interventions for nonhuman animals.

    As an aside, it would be much more transparent and rigorous of GiveWell to more clearly indicated that the ~$3,400 does not actually mean ~$3,400 per life saved, which is how I have always seen people use it. The text on the AMF review page no longer has the number so I am unsure of whether it even said “equivalent” previously, though I believe it either did not or that it at least did it did not link to an explanation of what that means, as this is the first I am learning of this. Essentially, people using that number are thinking GiveWell is ~2-3x as effective as it is, which is unfortunate.

    I would also encourage GiveWell to stop using misleading terms like “life saved” or “death averted”, which may emotionally mislead viewers who may not be appreciating that the person will still die — it is rather a death “delayed”.

  • Ajeya Cotra on December 15, 2016 at 1:20 pm said:

    Kelly: Total utilitarianism isn’t a person-affecting view (it’s not a stance which says that only people who would have existed anyway can be helped or harmed by actions), so I don’t think that a total utilitarian would only count suffering due to malaria sickness and suffering endured by the survivors in their calculus. If no one else would have been born to “replace” the person who died, then a total utilitarian would also factor in the loss of all the years of happy life the deceased person would have experienced (ie, it would look identical to the deprivation stance). This is tempered by replacement rate: if on average the death of a child results in 0.5 additional births in the family, the value from the deprivation stance is multiplied by 0.5.

    I agree that an Epicurean who focused on suffering would likely discount our cost-effectiveness estimate significantly, but I’m a lot more uncertain than you are about how severe those effects will be. It’s not obvious to me that grief is short-lived and mild; in fact, my personal instincts point in the opposite direction (though I recognize they could easily be mistaken). On top of that, the possible effects on children if parents or other caretakers die of malaria could easily be very long-lasting, and that could change the calculation significantly. It gets more complicated if you bring people’s stated or revealed preferences into the mix: would a parent give up five years of their own life to save their two-year-old child? What sort of risks or harms are parents willing to endure to protect their young children, in practice? If we get this information, should we take it at face value or worry that it’s being influenced by biases?

    On transparency: When we previously wrote that ~$3,500 to AMF saved a life in expectation, we were referring to the cost per life saved, not the cost per life saved equivalent. In our revised November 2016 model, that number has risen to about $7,500, while the cost per life saved equivalent is, at about $3,200, coincidentally close to the previous value for cost per life saved. You can see all past versions of our AMF review here. (All of our top charity reviews link to previous versions of those reviews.)

  • Thanks for this very interesting and in depth post. I in particularly enjoy access the the editable CEA that givewell uses, exceited to show this off to people and play around with it.

    Will updated editable versions of the givewell CEA be continued to be made available, and if so where would we access them?

  • My belief that AMF is a good donation opportunity are based on the following (moderately confidently-held) beliefs:
    – Deaths, especially of children, cause some sadness
    – Where parents have lots of children, there is less capacity to invest in any of them, so those children tend to be less likely to have a basic level of education
    – To the extent that malaria contributes to adult death, it (somewhat) leads to a society with a surfeit of young men, who are especially prone to be under-educated relative to their potential (see previous point) – this leads to a higher probability of violence and war.
    – To the extent that malaria contributes to adult death, it stops people from fulfilling long-term life plans to build things of value for society (e.g. companies, civil society)
    – (Linked to the previous point) Malaria slows economic growth, and economic growth is probably a good thing for the poorest societies
    – I would be worried about an argument against AMF’s work if I thought it would lead to explosive population growth that was too fast for infrastructure development to keep up – to a certain extent I think there is an element of valid worry here, but there is at least a partial self-regulating element (albeit with a lag) – this is what David Roodman’s post told us (or which we might have guessed by reading, e.g., work by Jeffrey Sachs)

    I have reviewed GiveWell’s past CEA analyses with interest, but always imagined that measures like lives saved or QALYs were simply a simplification/proxy to get at the things we really care about – namely the sorts of things I’ve listed above. There is a nod to this sort of thinking in this post, but it got less emphasis than I thought, and makes me wonder whether GiveWell thinks of QALYs/lives saved as more of an end in themselves than I thought?

  • Ajeya Cotra on December 19, 2016 at 1:16 pm said:

    Chris — Thank you! I’m glad you found it useful. All of our current and past cost-effectiveness analyses will be available on this page. The page also contains a video walk-through of our CEA.

    Sanjay — There isn’t a single coherent set of assumptions underlying GiveWell’s cost-effectiveness calculations. The way we arrive at the cost-effectiveness estimates in column R of the “Results” sheet is by using the median staff inputs (column C in the “Parameters” sheet). Some staff members may have been trying to take into account their best guess about the kind of long-term flow-through effects you mentioned when entering estimates for parameters such as rows 6, 7, 53, 63, and 64 in the “Parameters” sheet.

    I personally didn’t, because I felt that trying to account for such high-uncertainty, big-picture considerations in the direct cost-effectiveness estimate was too difficult and unhelpful for me. Instead, I use the cost-effectiveness estimate calculated from my parameters as a jumping-off point, and reason about long-term effects separately, and more qualitatively.

    On QALYs/DALYs: I can’t speak for everyone on staff, but I believe that giving people more years of fulfilling life and/or making the years they are going to live more fulfilling is an end in itself, and that’s approximately what I mean when I say “averting DALYs.” I’d guess that’s a pretty common view among GiveWell staff. This doesn’t mean I’m unconcerned about long-term outcomes such as economic development — it just means that I would ultimately conceptualize the benefits of economic development in terms of something like DALYs (at least, under the flexible interpretation of DALYs we use in our CEA).

  • I’m concerned that your spreadsheet may not be functioning properly. When I decrease my value for row 63 (“DALYs averted per death of an under-5 averted — AMF”), that winds up *reducing* AMF’s cost per life saved on the results tab (and the cost per life saved of all the other interventions!). Isn’t that backwards, or am I misunderstanding something? If I input that averting child death is worth 0 DALYs, the results imply that all programs are infinitely effective.

  • Ajeya Cotra on January 1, 2017 at 1:25 pm said:

    Jon, that is a little bit counterintuitive, but it’s accurate. Suppose that in expectation, giving $1000 to Program A averts one young child’s death and doubles one adult’s income for one year. If you left “Parameters”, C7 alone in the CEA, then doubling one adult’s income for one year is fixed at approximately 1/3 of a DALY.

    Now let’s see what happens with different values of “Parameters”, C63, “DALYs averted per death of an under-5 averted.” If it’s 10 DALYs, then doubling an adult’s income for one year is worth (1/3) / 10 = 1/30 as much as saving one young life. Then in young-life-saved-equivalent units, Program A is worth $1000 / (1 + 1/30) = $967 per young-life-saved-equivalent. On the other hand, if averting a young child’s death is worth much less, say 1 DALY, then the relative value of increasing income is much higher: doubling one adult’s income for one year is now worth 1/3 / 1 = 1/3 as much as saving one young life. Then Program A is worth $1000 / (1 + 1/3) = $750 per young-life-saved-equivalent. If we assume averting the death of a young child is worth 0 DALYs, then doubling an adult’s income will be worth (1/3) / 0 = infinity, which is how you get the result that everything is infinitely valuable.

    This behavior is strange because you’re changing the value of the reference point of the cost-effectiveness calculation. Consider another example: for $1000, Program B will on expectation avert the death of one young child, and have no other effects. In that case, no matter what DALY value you assign to that outcome, Program B will always have a cost-effectiveness of $1000 per young-life-saved-equivalent — it’s just how much you value the young-life-saved-equivalent unit that will change.

    In the “Results” sheet, rows 12-24 provide relative cost-effectiveness estimates for all of the charities (for example, B12 shows that the median staff member believed AMF was 2.2 times as cost-effective as GiveDirectly). These outputs will behave more intuitively: holding other inputs constant, as you reduce the value of averting a young child’s death, AMF and SMC will appear relatively worse compared to GiveDirectly and the deworming charities.

  • Pablo Stafforini on March 13, 2017 at 8:54 pm said:

    Hi Ajeya,

    In the post, you write that GiveWell’s current estimate of AMF’s cost per life saved-equivalent is $3,400, but in a later comment you give a figure of $3,200 instead. Is this a mistake, or are you reporting different estimates? Thanks.

  • Ajeya Cotra on March 14, 2017 at 6:28 pm said:

    Hi Pablo,

    Thanks for raising this! We reviewed the revision history in the CEA. It turns out that when I was drafting the post, the cost-effectiveness estimate generated by combining staff members’ median parameters (“Results”, R4) was around $3400. The value dropped by ~$100 shortly after, when an error that accidentally excluded one staff member’s inputs from the median was corrected.

Comments are closed.