The GiveWell Blog

Donor impact vs. donor attribution (or, does your $120 really buy a sheep?)

One of the questions we struggle with a lot is the question of what impact a donation has – i.e., what happens because of your donation that wouldn’t have happened otherwise? In other words, what do you get for your dollar?

It’s a tricky question, especially for relatively small donations going to relatively large organizations. A future post will discuss a couple of approaches we’ve taken to answering this question; for now I want to focus on an approach we haven’t taken, that of literally attributing each donation to a “set of purchased items.”

A good example is this GlobalGiving project, which promises to educate 1 woman for every $45 you give. Organizations like GlobalGiving, Kiva, DonorsChoose and Heifer International make similar offers.

We’ve never been sure what to make of projects like these. For example, I see two interpretations of the “teach 4200 women” project:

  1. The pitch is literally true. For every $45 that comes in, one woman will be educated. Therefore, if the project is underfunded by $45, one qualified and interested woman will be turned away from the class.
  2. Funds are fungible. The organization conducting the class (Women for Women International, a large international aid organization) has already drawn up a full budget for the project, and will fund it with “general” funds if necessary. Therefore, for every $45 you give, the organization moves $45 of its “general” funds back into the “unrestricted” pool, meaning that your $45 is effectively an unrestricted donation to the organization running the class.

I’m guessing that #1 is the one that donors subconsciously picture when they are drawn to the “tangibility” of a project. It’s the scenario under which a donation is literally matched to a discrete person or item. Yet if #1 were the case, this would seem a horribly inefficient way to run projects – spending all the overhead to pay staff, set up the class, etc., and leaving one woman out because of a $45 shortfall. (Or in the case of Kiva, setting up a micro-bank and then turning away a screened, qualified borrower because of a missing $850.)

Yet if #2 is correct, your donation is really an unrestricted donation to a large organization; to understand your impact, you need to understand the entire organization.

GlobalGiving’s FAQ suggests a mix of the two scenarios: sometimes a funding shortfall is covered by other funding, and sometimes a project is scaled back or canceled when funding falls far short. As a side note, I would guess that when a project is scaled back, the “cost per person” generally increases (since the program’s overhead can’t fall fully in proportion to people served).

Other organizations are explicit that #2 is the case (see small print on this “buy a sheep” page).

All in all, I’m skeptical of any claim that says “your $1000 buys X.” It’s a good way to make things feel tangible, but a donor truly trying to understand his or her impact should take a different approach. (Some possibilities will be discussed in a future post.)

Eat your salt

Iodine deficiency has been linked with child mortality as well as permanent cognitive debilitation. Some references are on page 25 of the 2004 Copenhagen Consensus report on malnutrition; a WHO report paints a more vivid qualitative picture (page 6):

In areas of iodine deficiency, where thyroid hormone levels are low, brain development is impaired. In its most extreme form, this results in cretinism, but of much greater public health importance are the more subtle degrees of brain damage and reduced cognitive capacity … the mental ability of ostensibly normal children and adults living in areas of iodine deficiency is reduced … there is little chance of achievement and underdevelopment is perpetuated. Indeed, in an iodine-deficient population, everybody may seem to be slow and rather sleepy.

How do you and I avoid these disorders? The answer isn’t as simple as you might think. Within the last century, iodine deficiency was common in the U.S., and it has been combated using an explicit and sustained public health effort. The following is taken from the Micronutrient initiative’s global progress report:

The element iodine was discovered in 1811, but almost a century passed before it was established that lack of iodine caused the swelling of the thyroid gland commonly known as goitre … In the United States, the alarm was first raised in Michigan in 1918 when it was revealed that over 30% of men medically examined for war service had been found to have an enlarged thyroid. Many were declared unfit for service. By 1923 an Iodised Salt Committee had been formed, including physicians and representatives of the Salt Producers Association …. Later that same year, the Morton salt company began marketing iodised table salt nation-wide … by 1932 iodised salt accounted for 90% to 95% of all sales.

Iodine deficiency disorder is now extremely rare in the U.S., but it is still common in many other parts of the world (see the WHO 2004 report on iodine status worldwide). The effects on cognition and economic growth are widely unknown but potentially disastrous – and this is a problem that isn’t necessarily going away by itself, or going away as soon as wealth increases. (Especially if there’s a direct link between iodine deficiency and productivity.)

Iodizing salt may not have the same visceral appeal as water-related programs, but it’s a proven way to solve a truly debilitating problem and reduce both death and poverty. And it’s an area where there’s arguably no substitute for large-scale public health programs. Note that the Copenhagen Consensus rates it as the 3rd most cost-effective use of funds.

Transparency done right

We were referred to the Against Malaria Foundation today. It’s the most transparent developing-world aid organization we’ve seen to date.

AMF grants funds to NGOs that distribute insecticide-treated nets (ITNs) to fight malaria. ITN distribution is a well-established method for saving lives and lowering the burden of malaria (see the “Insecticide-treated nets” section of the malaria chapter in the Disease Control Priorities Project).

But what sets AMF apart from other organizations carrying out similar programs is that they publish all the details for all of their projects.

This map lists all programs, by country, date, partners, and status (whether the nets are en route, have arrived, are being distributed, etc.) Click on an individual program and you can see the full proposal that was submitted and funded (and the proposals are quite concrete and specific), and, when applicable, other materials – including the post-project report and even shipping records. Pictures and videos are also available. Here’s an example of a project in progress; here’s a completed project.

Finally, you can see which donors’ gifts were used to fund the project. There’s been a lot of excitement over organizations such as Kiva and DonorsChoose because of their ability to link donors’ money directly with people served. This organization does something very similar, but it’s distinguished in that the types of projects it’s linking donors up with (bednet distribution projects) are larger-scale projects serving many people at once, and are well documented (see link above) in terms of their impact on human lives.

We’re going to have to spend more time with all of this information to check how the actual cost-effectiveness is living up to the top-line numbers. What we know now is that if all charities were as transparent as AMF, our job would be a lot easier.

Priority region 1 – finalists

In addition to choosing straightforward, cost-effective program-based interventions, we’re also planning to identify two regions to study in detail.

I’ve focused on finding priority region 1: an area that has both high and drastic disease burdens and relatively stable, low-corruption environments in which aid can likely be productive. This post details the criteria I used to narrow the field to a few finalists. (Note: We recently spoke to a private philanthropic advisor about this, and our methodology is quite similar to theirs.)

Corruption and poor governance

I used the latest versions of the Failed State Index (from the Fund for Peace and Foreign Policy Magazine) and Corruptions Perception Index (from Transparency International) to exclude countries where governance would be a large issue. In both cases, I removed countries with the worst rating on either index. (The FSI ranks 177 countries; 31 received the worst rank. The CPI ranks 180 countries; 38 received the worst rank.)

Disease burden and poor development

I used the 2007-08 United Nations Human Development Index, considering only countries that are marked “Low human development” (received by 22 of the 177 countries rated). Of those 22, only 9 passed the corruption/governance screen.

The UNHDI considers life expectancy (to assess health burden) but doesn’t consider morbidity. So, I also looked at DALY burden per 100,000 people (data from the WHO’s Burden of Disease Project) and include countries in the top decile of health burdens. This criterion excluded Senegal, Benin, and Eritrea.

That left me with 6 countries:

  • Zambia
  • Mozambique
  • Mali
  • Rwanda
  • Tanzania
  • Burkina Faso

Charitable activity in each country

We need to find a region that not only has both high and drastic disease burdens and relatively stable, low-corruption environments in which aid can likely be productive but also a region with a significant amount of aid being given (since our role is to identify outstanding charities rather than to create new ones, we’d rather work somewhere where we’ll have more options). One proxy for this idea is the total aid given by OECD countries to each of our six finalists (data from OECD). This measure is far from perfect since it reflects government aid, not private philanthropy. We also conducted a Guidestar search for the name of each country, and use the number of results as an approximation to the number of US-registered NGOs working there. The data is below.

Country Population (m) OECD aid (USD m) Guidestar results OECD aid/capita NGOs (per million ppl)
Zambia 12.3 727 210 59 17
Mozambique 21.4 958 132 45 6
Mali 12.4 326 86 26 7
Rwanda 9.8 247 187 25 19
Tanzania 44.6 951 354 21 8
Burkina Faso 15.4 310 57 20 4

Notes:

  1. Population data comes from Wikipedia’s List of Countries by Population page.
  2. OECD aid is the average annual aid over the last 5 years

These criteria would seem to indicate Zambia, Mozambique and Rwanda as our top contenders. We are also strongly considering Tanzania, as there may be superior availability of data on health and living conditions through the Living Standards Measurement Study.

Our plan now is to talk with our advisors for a) their thoughts on our process thus far and b) their ideas for picking a focus region from this list. We’re also planning to examine the Poverty Strategy Reduction Paper for each of these countries to get a better sense of the major challenges and available information there).

(Note: for the intrepid, you can access an Excel file with all the data used for this post here.)