The Brookings Institution is hosting a conference this week called “What works in development?” including an interesting paper by Simon Johnson (International Monetary Fund) and Peter Boone (London School of Economics) titled, “Do Health Interventions Work? Which and in What Sense?”

Johnson and Boone review the existing literature and conclude that there is very little knowledge about the most effective methods for reducing child mortality in the developing world, and that without improved knowledge, aid organizations may fail to reduce child mortality as much as they hope.

Knowledge is limited in the following ways:

• We know what works clinically, but know far less about the most effective ways to fully implement an intervention.

  This rings true to me and makes me think of bednets. We know that insecticide treated bednets, when used properly and consistently prevent deaths from malaria (at least in the short term), but we don’t know the most effective way to ensure proper use, a critical component of the intervention. Evidence for the effectiveness of bednets comes from aid experience in very specific contexts (e.g., the way in which the nets are distributed, the education level of those who receive them, etc) which means that the distributing bednets may not be as effective when implemented in a context different from that of the initial evaluations.

• When we have multiple, proven interventions, we generally don’t know which to implement where or how they’d work as a package.

  Keeping the point above in mind, there’s strong evidence that both insecticide treated bednets and artemisinin-based therapies reduce child mortality when implemented properly. However, little is known about how they work together (as a package) or which situations are best suited to one or the other. (It doesn’t make sense to implement both everywhere because a) the more the treatment is used the more quickly resistant strands of malaria will likely develop and b) the cost of implementing both everywhere will obviously exceed an approach that implements only what is necessary).

• Evaluations of interventions’ effectiveness often stop at measuring reduction in incidence as opposed to total mortality.

  Often, evaluations of interventions focus on an intervention’s effect on disease incidence (e.g., the reduction in cases of diarrhea caused by building improved water and sanitation infrastructure). This is a problem because many of the causes of death in the developing world are interrelated – i.e., one problem increases the likelihood of death from another. UNICEF estimates that malnutrition is a contributing factor to 50% of child deaths (from malria, diarrhea, etc.), and the WHO finds that measles contributes deaths from pneumonia and diarrhea. Because of these interrelationships, evaluations that only asses an intervention’s effect on disease incidence may not accurately identify the effect on mortality.

  This problem is illustrated in a recent paper cited by Johnson and Boone that finds that while water and sanitation projects reduce incidence of diarrhea, they have a minimal impact on child mortality. Johnson and Boone hypothesize that:

  It seems plausible that the much wider coverage of water and sanitation today, along with the advent of vaccines and treatments for the main causes of death from infectious disease, mean that further improvements in water and sanitation are no longer necessary or very significant to eliminate remaining deaths (pg 21).

Johnson and Boone have their own view on the best approach: targeting parental knowledge rather than distribution of materials. They observe that:

• Many interventions are extremely inexpensive (e.g., Oral rehydration therapy costs $.10/packet and malaria treatment costs $.50 cents/dose – Pg 17), and are not beyond the means of many people in the developing-world.
• There is good evidence that parents are not very knowledgeable about health (Pg 25), and parents’ education is highly correlated with child mortality (Pg 23).

It’s a plausible hypothesis, but could easily be flawed, as Johnson and Boone point out themselves. For example, it’s possible that the observed correlation between parental education and child health is a simple consequence of the fact that more educated parents also tend to be wealthier, and that wealth is in fact the primary factor here.

Knowing that their hypothesis could be right or wrong, Johnson and Boone have set out to test it. Working with Effective Intervention, a UK-based charity, they’re planning to implement a series of randomized controlled trials of comprehensive aid programs focusing on a) educating parents and b) providing access to necessary health products. In some areas, they’ll also include education for children as part of the intervention, planning to follow the children at least 10 years after the completion of the trial. Eventually, they plan to run trials in 600 villages in Africa and India covering 500,000 children. They say it will take three years for the first findings.

This is the first we’ve heard of Effective Intervention, but they are taking exactly the approach we identify with most: starting with a systematic review of what we do know, pinpointing what it is we want to know next, and then focusing on producing that knowledge rather than on scaling up a program with unknown effectiveness. We’re looking forward to their results.