In addition to choosing straightforward, cost-effective program-based interventions, we’re also planning to identify two regions to study in detail.

I’ve focused on finding priority region 1: an area that has both high and drastic disease burdens and relatively stable, low-corruption environments in which aid can likely be productive. This post details the criteria I used to narrow the field to a few finalists. (Note: We recently spoke to a private philanthropic advisor about this, and our methodology is quite similar to theirs.)

Corruption and poor governance

I used the latest versions of the Failed State Index (from the Fund for Peace and Foreign Policy Magazine) and Corruptions Perception Index (from Transparency International) to exclude countries where governance would be a large issue. In both cases, I removed countries with the worst rating on either index. (The FSI ranks 177 countries; 31 received the worst rank. The CPI ranks 180 countries; 38 received the worst rank.)

Disease burden and poor development

I used the 2007-08 United Nations Human Development Index, considering only countries that are marked “Low human development” (received by 22 of the 177 countries rated). Of those 22, only 9 passed the corruption/governance screen.

The UNHDI considers life expectancy (to assess health burden) but doesn’t consider morbidity. So, I also looked at DALY burden per 100,000 people (data from the WHO’s Burden of Disease Project) and include countries in the top decile of health burdens. This criterion excluded Senegal, Benin, and Eritrea.

That left me with 6 countries:

• Zambia
• Mozambique
• Mali
• Rwanda
• Tanzania
• Burkina Faso

Charitable activity in each country

We need to find a region that not only has both high and drastic disease burdens and relatively stable, low-corruption environments in which aid can likely be productive but also a region with a significant amount of aid being given (since our role is to identify outstanding charities rather than to create new ones, we’d rather work somewhere where we’ll have more options). One proxy for this idea is the total aid given by OECD countries to each of our six finalists (data from OECD). This measure is far from perfect since it reflects government aid, not private philanthropy. We also conducted a Guidestar search for the name of each country, and use the number of results as an approximation to the number of US-registered NGOs working there. The data is below.

Notes:

1. Population data comes from Wikipedia’s List of Countries by Population page.
2. OECD aid is the average annual aid over the last 5 years

These criteria would seem to indicate Zambia, Mozambique and Rwanda as our top contenders. We are also strongly considering Tanzania, as there may be superior availability of data on health and living conditions through the Living Standards Measurement Study.

Our plan now is to talk with our advisors for a) their thoughts on our process thus far and b) their ideas for picking a focus region from this list. We’re also planning to examine the Poverty Strategy Reduction Paper for each of these countries to get a better sense of the major challenges and available information there).

(Note: for the intrepid, you can access an Excel file with all the data used for this post here.)

Below is our basic outline for identifying the best options for donors within developing-world health. (Economic empowerment and possibly other developing-world causes will come after we have completed our report on direct health interventions.)

Focus on two priority regions and several priority interventions

As this post explains, we are simultaneously taking two approaches: a region-based approach (studying a particular region in depth, and aiming to fund interventions targeted to the specific needs of the region) and a program-based approach (identifying particularly promising interventions and funding their scale-up throughout the world).

Priority interventions: focus on the most proven, scalable, and cost-effective

Our general criteria for selecting program-based interventions are discussed here. To start, we are prioritizing what we call straightforward interventions, or interventions that can be carried out infrequently and for which the burden of monitoring and evaluation is relatively low. As discussed here, these interventions are also rated among the most cost-effective interventions by existing studies of cost-effectiveness. They are:

1. Vitamin supplementation programs, providing nutrients such as vitamin A (which reduces the risks of infant mortality, development of blindness, and deaths in childbirth – reference here).
2. Mass drug administration programs, particularly those aiming to treat school-age children who suffer from intestinal parasites. (Such interventions have been shown to improve school attendance and likely nutritional status.)
3. Vaccination campaigns to cost-effectively save lives.
4. Vitamin fortification programs such as the iodization of salt.
5. Surgical programs to correct deformities such as cleft palate and fistula.

We may add other priority interventions, particularly if we see particularly strong donor demand for interventions with fundamentally different goals.

Priority regions: help those in great need and those with great potential

Our region-based research will be less focused on strict definitions of “cost-effectiveness” and more on finding programs that address a region’s needs broadly and holistically, somewhat along the lines of Partners in Health (and more in line with the vision of the WHO commission on macroeconomics and health than that of the Copenhagen Consensus). The first step will be picking two priority regions.

Priority region 1: one priority region will reflect a goal of helping those in the most need. We are currently in the process of identifying countries that have both high and drastic disease burdens and relatively stable, low-corruption environments in which aid can likely be productive. Preliminarily, our strongest candidates are:

• Zambia
• Mozambique
• Mali
• Rwanda
• Tanzania
• Burkina Faso

The process for identifying these six countries is detailed here.

Priority region 2: the other priority region will reflect more of a triage approach: identifying an area where health conditions are poor and help is needed but strong economic opportunities exist for those at a reasonable level of personal health and productivity. Preliminarily, we are guessing that our work in this area will likely focus on somewhere in India, but we have not yet completed this analysis.

We know of two large-scale, systematic projects devoted to rating the cost-effectiveness of different health interventions. Both use an approach centered on disability-adjusted life-years (DALYs), which is not the only measure of cost-effectiveness we intend to use, but they’re a start.

One is the Disease Control Priorites report. We are currently in the process of constructing a full summary of the interventions and cost-effectiveness estimates discussed in the report, as the summary provided by the report itself appears incomplete. From looking at the average cost-per-DALY-averted of the existing list, however, it appears that three of the five most cost-effective interventions (including the top two) fall under what we call straightforward interventions: vaccination campaigns and mass drug administration (deworming).

The other is the 2008 Copenhagen Consensus, which explicitly ranks interventions by the desirability of funding them. Four of the top six (including #1) fit within our straightforward interventions: vitamin supplementation, vitamin fortification, vaccination campaigns and mass drug administration (deworming).

From our scan of the Disease Control Priorites Report, we also believe that our straightforward interventions will prove top candidates for cost effectiveness by other criteria. In particular, we noted vaccination campaigns as having the lowest “cost per death averted” (#4) in the report, which likely corresponds to the lowest “cost per life-year saved” (#3) as well since most deaths averted are infant deaths. We also feel that surgery may be the strongest intervention we’ve seen in terms of averting extreme suffering.

We therefore consider the “straightforward interventions” to be priority interventions, and are planning on looking into them actively. They will not necessarily be the only priority interventions we name, but they stand out given what we know.

Again, these interventions are:

1. Vaccination campaigns.
2. Mass drug administration programs, including albendazole for treating helminths (i.e., deworming).
3. Vitamin supplementation programs, with nutrients that need be taken only infrequently (particularly vitamin A).
4. Vitamin fortification programs, such as the iodization of salt.
5. One-time surgery programs, along the lines of Interplast’s surgical team trips.

As discussed previously, we are looking for (program-based) interventions that are both proven and scalable. The Disease Control Priorities report lists many interventions that are “proven,” in the sense that one or more studies have been done indicating that the program has improved health outcomes in the region. However, showing that a program has worked once – or even several times – is far from showing that it will work again. Not only could the context, circumstances, and clients change, but the people running the program (and the scrutiny they’re under) most likely do change from a studied experiment to a less thoroughly tracked replication.

In examining the case for different interventions (see the list of interventions here), we have noted that some types of programs are inherently more “straightforward” than others, in the sense that a lower burden of proof is necessary to extrapolate from their past effects to their future effects. For example:

• Many programs focus on educating clients, and their success thus depends on sustained and difficult-to-track behavior change from locals. Even if a sanitation program has successfully reduced diarrhea incidence rates in one region, bringing it to a new region means dealing with a new group of people, who may respond completely differently to the same education techniques. Without thorough and ongoing monitoring and evaluation, we feel it is difficult to be confident that a program along these lines is continuing to work.
• Other programs, such as DOTS for tuberculosis, focus on improving medical care. They may depend on sustained behavior change from local medical professionals, but not from the population in general. The success of such programs will depend heavily on the existing health care infrastructure, qualifications of health professionals, and quality of training. We feel it is difficult to have confidence in such programs without continually tracking patient outcomes.
• Other programs, such as vaccinations, do not rely on sustained behavior change from anyone in the local population. A vaccination campaign in a given region can be carried out fairly infrequently (i.e., once a year) as long as the vaccines administered are legitimate and a large number of people can be gathered and treated. Both of these factors are relatively straightforward to track and report.

Below are all the interventions we’re aware of that fit in the last category: interventions that can be carried out relatively infrequently (i.e., every 6-12 months), and for which all necessary behavior change can be directly observed and reported by the people carrying them out. For each intervention, we briefly characterize what information would need to be reported in order to reasonably extrapolate from the intervention’s past, studied effects on health outcomes.

1. Intervention: vaccination campaigns. Information needed: number and age of people vaccinated; disease incidence/prevalence estimates for the region.
2. Intervention: Mass drug administration programs, including albendazole for treating helminths and ivermectin for treating onchocerciasis and lymphatic filariasis (more on these conditions in future posts). Information needed: number and age of people treated; disease incidence/prevalence estimates for the region.
3. Intervention: Vitamin supplementation programs, with nutrients that need be taken only infrequently (particularly vitamin A). Information needed: number and age of people treated; estimates of vitamin A deficiency prevalence for the region.
4. Intervention: Vitamin fortification programs, such as the iodization of salt. Information needed: amount and circulation of food fortified; estimates of iodine and other deficiency prevalence for the region.
5. Intervention: One-time surgery programs, along the lines of Interplast’s surgical team trips. Information needed: Condition treated for each client; completeness of surgery (before-and-after pictures would capture this information).

In labeling interventions “straightforward,” we are not claiming that they are easy to carry out or that they will always work. However, all else equal, we find them more promising for our purposes than other program-based interventions. This is largely because our experience to date with developing-world aid has shown that thorough, high-cost monitoring and evaluation (of the sort that could track sustained behavior change, for example) is relatively rare. We believe we are most likely to be able to confidently recommend interventions along the lines of those listed above, for which the necessary burden of monitoring and evaluation is lower.

In our analysis of bednet distribution programs, we considered the likelihood that a distributed bednet was ultimately used for its intended purpose: to protect against malaria.

The Malaria Matters blog recently posted links to numerous examples of nets used for other purposes:

Net stories include use for fishing in Zambia, as bridal veils in Zambia and other countries and trapping edible ants in Uganda. These problems arise when LLIN distribution programs focus on the wrong numbers. It is not enough to say how many hundreds of nets have been distributed in a community. The real concern is whether they are used correctly and for the intended purpose.

The post also notes that:

The three most popular reasons for using bednets to dry fish were: fish dry faster on these nets, they don’t stick and not surprisingly, these nets are cheaper.

On the one hand, it’s important for donors to know what their donations accomplish, and if donors’ aim is to prevent malaria, the above stories provide evidence of the types of problems that can occur with unmonitored aid. On the other hand, if people are buying nets and using them in other ways that improve their lives – even if it’s not malaria prevention – it’s not clear to me that that impact isn’t worth considering as well.