The GiveWell Blog

DALYs and disagreement

Elie thinks that fistula is worse than death. jsalvati disagrees.

I’d rather bring someone to full health than save an infant’s life. Ryan agrees, but Basti does not and Ron Noble emphatically does not.

It’s possible that we would all agree if we knew more about the lives of people in the developing world, or if we just had a long enough to argue about our values. It’s also possible that we wouldn’t. And as long as we disagree, we’ll have different opinions on what the most “cost-effective” interventions are. For example, if it’s true that fistulas can be repaired for $450 each, is this a better or worse use of donations than preventing children’s deaths for $200 each through vaccinations? My answer would be “It depends on the donor.”

Converting disease burdens and intervention benefits into DALYs doesn’t resolve questions like this. Rather, it obfuscates them, by converting the two interventions into the same terms using a single set of philosophical values. If the numbers above ($200/death averted for vaccinations, $450/surgery for fistula) are accurate, they allow different donors to make their own judgment calls, while being informed about their options. But these aren’t the numbers you’ll find in the Disease Control Priorities Project’s summary tables; instead, you’ll see only that surgical services cost an average of $136 per DALY averted (Jamison et al. 2006, Pg 75) and that the vaccinations interventions costs an average of $7 per DALY averted (Jamison et al. 2006, Pg 77).

Some simplification and information loss is necessary in order to compare different options, but reducing everything to a single unit means being able to serve only a single kind of donor. I’d prefer to estimate the effect of different interventions on a variety of “life outcomes” that different donors might value differently. We will discuss this variety more in a future post, but here’s a quick list:

  • Total life-years saved.
  • Adult lives saved (as it is common to value adult lives more than children’s lives).
  • Cases of extreme misery, such as fistula or perhaps severe elephantiasis, averted.
  • People brought to a “normal” level of health, i.e., without any debilitating nutritional or other conditions.

All of these things need to be separately estimated to produce DALY estimates. The DCP report did so with admirable thoroughness and far more people than we have (Jamison et al. 2006, Pgs xxiii-xxxiv). Yet because they published only their DALY estimates (not, with some exceptions, the estimates of different health problems that went into them), they buried a great deal of this work, and produce cost-effective estimates that are useful only if you’re completely on board with all of their values (from how bad each disability is to how to value different years of life). We’re currently trying to get in touch with the authors so we can get access to more of the details; if we don’t, we’ll have to repeat much of their work (with less capacity to do so).

Sources

Previously in series:

Comments

  • Ron Noble on August 27, 2008 at 9:49 am said:

    I think with so much controversy and so much being genuinely debatable about the DALY measure, it makes since to use a variety of measures. I think we are a long way away from being able to measure the amount “good” done in one measure. It may be an insurmountable problem. And as you point out, using a measure to encourage effective donations won’t do much if people balk at the metrics. That said, I’d encourage you to include DALY estimates along with the other measures you mention. Besides the fact that they are in my own opinion the best attempt to date to measure the good done by health-related interventions, the fact the WHO uses them gives them some prestige, which will appeal to some donors.

    But measuring DALYs is hard. And for all the various different aspects of measuring DALYs that are discussed in the DCP report, they don’t as far as I’ve seen go into any detail about how the DALY estimates for various illness are derived. As you note, people have wildly different intuitions about these things.

    Probably though, the DALY weights were derived by first gathering individual estimates from large numbers of people, and then averaging them. This is a defensible approach, particularly if you ask people who actually suffer from the illnesses in question. If you can’t do that, the best proxy is a very accurate description of what the illness is like. Even for people who actually have, let’s say, severe arthritis, the individual impact on them may very greatly. Some will have a higher tolerance for the physical pain than others. Some with jobs requiring physical labor may have their livelihoods taken away, while those who do other types of jobs may be able to continue working. At its best, this type of measurement is better than individual opinions of how bad the disease would be for you if you had it.

  • Holden on August 27, 2008 at 10:25 am said:

    Ron, I think you’re right to observe that the kind of studies used to come up with disability weights could provide useful information to people, beyond their intuitions, about how to value different disabilities.

    We appear to be in agreement that:

    • Using DALYs as the sole measure of cost-effectiveness is a mistake; having access only to DALY estimates and not their components is unfortunate.
    • The DALY is a measure that has some advantages and has value to some donors. We will include DALY estimates as one of our metrics whenever possible.
  • Ron Noble on August 27, 2008 at 2:24 pm said:

    Holden,

    yes, we are in agreement on those things.

    Often researchers do report some sensitivity analyses, which gives the DALYs figured a few different ways, which is somewhat helpful. Whether you can figure them other ways based on what is reported varies, and always requires some work (provided you know how to do the work.)

Comments are closed.