The GiveWell Blog

Aid is not the only thing that reduces poverty

| by Holden

Earlier this month, a Gates Foundation representative (Mark Suzman, Acting President of the Global Development Program) made a post epitomizing what I feel is a key fallacy in the world of giving: that any and all progress in struggling countries can be attributed to aid.

The thesis of his post is that “A greater focus on results and accountability means that overall aid spending has been getting smarter, more focused and more effective. Increasingly, taxpayer dollars are spent on proven interventions that are saving and improving lives.” And the only support given for this thesis is observations about aggregate improvements in health, wealth and education (drops in childhood deaths, drops in the number of people living under a given poverty line, etc.)

Mr. Suzman concedes that “Economic growth in China and India has been the primary engine of this improvement,” but in his discussion of other regions including Africa, there is a strong implication (difficult to convey in an excerpt, but clear if you read the piece) that the mere fact of improvement points to the effectiveness of aid.

There is no mention of possible non-aid-related factors behind the improvement in these regions, such as:

  • Improvements in government programs and government accountability, which could happen because of aid to governments or for other reasons
  • Improvements in technology
  • Local people making progress on their own problems, even if such progress isn’t visible in national-level GDP statistics

This fallacy is one that we see often. People often ask how we can recommend that donors not support certain areas, such as water or education, saying things like “If we don’t support these areas, who will?”

The answer may be that no one has to. It’s worth reminding ourselves that the first countries to emerge from poverty didn’t receive any aid from wealthier countries, and there is no easily discernible influence of aid in many of those that have emerged since.

People can and do solve their own problems. Rather than giving ourselves responsibility for everything they’re struggling with, we should focus on the areas in which we’re most likely to be able to help.

Singularity Summit

| by Holden

Among those who follow GiveWell, there is some interest in the Singularity Institute for Artificial Intelligence and its mission of lowering the risks associated with the creation of artificial intelligence that “[leaves] human abilities far behind.” We have been asked several times to share our views on its work and the value of a donation given to it.

My only knowledge of this issue, as of now, comes from reading Less Wrong and Overcoming Bias and speaking with the Institute’s President, Michael Vassar. I’m interested in this community partly because it has a lot of people (including Mr. Vassar) who think critically and analytically about how to accomplish as much good as possible, considering all options and putting positive impact above other considerations; in that sense their values overlap strongly with ours.

At this point

  • I believe that there are enormous risks and upsides associated with artificial intelligence. Managing these deserves serious discussion, and it’s a shame that many laugh off such discussion.
  • I do not feel that the Institute is a strong opportunity for a donor to accomplish good. I sketched my reasons in this comment and will eventually lay out my thoughts more thoroughly.
  • I do intend to learn more about this area and am open to changing my mind in either direction.

Consistent with the last point, I will be attending the Singularity Summit this year and encourage others interested in this topic to consider doing the same.

GiveWell will be moving to Mumbai (India) for 3 months

(updated on: ) | by Holden

GiveWell’s 3 full-time staff (myself, Elie Hassenfeld and Natalie Stone) will be living and working in Mumbai from mid-August through late November.

Developing-world aid has become a major focus for us, and we hope to have more opportunities to see aid work up close (along the lines of my trip to Africa earlier this year).

Please let us know if you have suggestions for charities we should visit, contacts at such charities, or any other advice/suggestions/contacts.

Microfinance’s “failure” to reach the poorest

(updated on: ) | by Holden

USAID’s most recent report on microfinance and microenterprise development tells an interesting story and, in my view, shows just how widely microfinance has been (and continues to be) misunderstood. While many advocate that microfinance institutions focus on people under the global “extreme poverty line”, USAID’s report implies that actually doing so is rare and even unrealistic.

Background: the myth of targeting the poorest

The international “extreme poverty line” is around the equivalent of US$1.25 per day, and around 1.4 billion people worldwide (and over half of those in sub-Saharan Africa) are estimated to live below this line (see the discussion in our international aid report).

Many seem to believe that people in this category are appropriate – even ideal – as clients. For example, see Opportunity International and Grameen Foundation stressing the need to reach the “poorest” and “most vulnerable.” Both Accion and CGAP cite the entire 3-billion-strong set of people under the US$2/day line as potential microfinance clients (upwards of 50% of this set falls below the “extreme poverty line”).

U.S. official aid seems to have taken this idea particularly far. For the past several years, USAID has been required by law to target the “very poor,” defined partly with reference to this “extreme poverty line”:

Both the Microenterprise for Self Reliance Act of 2000 (henceforth, the 2000 Act) and the MRAA mandate that at least half of all USAID funding for microenterprise development directly benefit the very poor. The 2000 Act initially defined the “very poor” as the bottom [poorest] half of those living below each country’s national poverty line … Subsequent amendments to the 2000 Act mandated a second, much more ambitious approach … the amended law created a second definition of the “very poor” — those living on less than the equivalent of $1 per day, calculated using purchasing power parity (PPP) exchange rates. The law made clear that, for any given country, the applicable definition of the very poor would be the more inclusive one.

(Note that “$1/day” may be a reference to the $1.25/day “extreme poverty line” discussed here – see note 6.)

Investigating actual poverty levels of clients

To its credit, USAID put significant effort into tracking whether it was actually meeting this goal, developing poverty assessment tools for assessing clients’ poverty levels and requiring certain grantees to use them. The results:

Among the eight microfinance institutions that applied and reported on the Poverty Assessment Tools, the average share of Funds Benefiting the Very Poor (FVP) is estimated at 28.5 percent, up from 16.3 percent in FY 2007. … For the 14 enterprise development programs that applied and reported on the Poverty Assessment Tools, average FVP is estimated at 26.0 percent, up from 20.5 percent in FY 2007 …

USAID did not come close to its target of 50% “extremely poor” clients. Furthermore, it concluded that continuing to push for this target would be unwise:

As matters stand, USAID sees no promising options for meeting the FVP target. It cannot do so by reallocating funds among its existing partners, because with the exception of one small program, none had more than 50 percent “very poor” clients. It cannot do so by shifting funds to established microenterprise organizations that are not already receiving USAID funding, because few if any such organizations are voluntarily applying the USAID-certified poverty assessment tools, and no such organization has offered solid evidence that it has more than 50 percent “very poor” clients …

Misguided target?

USAID does not conclude that microfinance/microenterprise projects should be de-emphasized (it observes that “the great majority of clients … are very poor, at least in commonly used terms”). Instead, it concludes that the idea of serving the poorest was unrealistic/inappropriate in the first place.

the overall pattern of results lend further weight to the point that USAID raised in last year’s Annual Report – that current law imposes too low a threshold for being “very poor.” This very narrow definition makes it impossible for USAID to allocate its microfinance and microenterprise funding so as to reach the legislative target of directing 50 percent of the benefits of microenterprise funding to the “very poor,” without undermining other goals emphasized in the same legislation, such as sustainability and support for broad-based economic growth.

Unfortunately, this definition of being “very poor” was adopted without any evidence that a 50 percent FVP target based on this definition could be reached. Two years of results using the poverty assessment tools strongly suggest that the target cannot be reached without inflicting undesirable side effects on sustainability and economic development. In short, USAID sees no realistic prospect of reaching the target contained in the law, and urges prompt and serious consideration of changes in the law. (Bold mine; italics in original)

I’m inclined to agree with USAID’s conclusion. I agree that people with incomes well above the “extreme poverty line” can still be very poor, certainly poor enough that I’m interested in donating to help them. So my point is not that microfinance is being carried out inappropriately, or is failing to reach the very poor.

Rather, I’m noting yet another way in which microfinance seems to have been badly misunderstood by its biggest funders and proponents. USAID, and by implication its grantees, seem to have thought that they were serving the world’s poorest – to the point of legislating it – without any data, and wrongly. It’s another debunked myth, and another sign that the funding and the stories have gotten ahead of the facts.

New evidence that cleaner water -> less diarrhea

(updated on: ) | by Natalie Crispin

Providing clean water to people living in developing countries is a cause that many donors are interested in. Among other hardships, unclean water can lead to diseases such as diarrhea, which is responsible for millions of child deaths annually. Unfortunately, we have found little evidence that charities’ efforts to improve water infrastructure in the developing world have resulted in decreased incidence of diarrhea and we have not identified a water charity we can confidently recommend to donors.

A recent randomized controlled trial (RCT) Spring cleaning: A Randomized Evaluation of Source Water Quality Improvement, by J-PAL researchers, however, provides the first strong evidence we’ve seen for the impact of a water infrastructure project. Unprotected springs in western Kenya were randomly assigned to receive “protection” at the beginning or the end of the study period. “Protection” involved encasing the spring in concrete and directing water through a pipe so that water did not come into contact with contaminants on the ground. The researchers found that children under the age of 3 living in households that used protected springs were 25% less likely to have had an episode of diarrhea in the last week than children in households that used unprotected springs.

Note that an earlier draft of this study found no statistically significant impact on diarrhea incidence, so the update is a new and somewhat surprising piece of evidence.

So the project appears to have worked; what about the cost-effectiveness? This is tricky to assess:

  • Any calculation of the cost-effectiveness of spring protection will be highly dependent on how many households with young children use the spring and how long the spring remains useable.
  • The study’s authors estimate that “the cost per DALY averted for this intervention is $16.75.” Using a simple conversion calculation, we estimate that this is equivalent to about $0.25 per case of diarrhea averted and $534 per death adverted, which is well within the range we consider highly cost-effective.
  • However, the authors provide no information on how they reached this figure, and our own attempts to reconstruct the calculation (found in this spreadsheet) yield estimates ranging from $0.32 to $1.88 per diarrhea case averted ($21.63-$126.04 per DALY averted; $689.52-$4018.34 per death averted) depending on which assumptions are used. The more reasonable of our estimates (incorporating “discounting” of future diarrhea cases to mirror discounting of costs, and assuming that 75% or fewer of children under 12 are also under 3) imply costs of at least $0.61 per case / $40.66 per DALY / $1,296.24 per death averted – several times more expensive than we estimate it takes to prevent a death from a vaccine-preventable disease (~$200), from tuberculosis (~$150-$750), or from malaria (~$182-$1126). Update: We made an error. The authors do provide information about how they reached their estimate.

    They used a different approach than ours, and having considered it, we defer to their estimate as likely more accurate.

    We believe the source of the difference in our cost-effectiveness estimates is due to a difference in the conversion calculation used to translate between cases, DALYs, and deaths averted. The study’s authors used a conversion factor of 864 cases averted = 32 DALYs averted = 1 death averted, based on data on child deaths from diarrhea in sub-Saharan Africa from from the Global Burden of Disease report (2006, data from 2001) and data on cases of diarrhea per child in sub-Saharan Africa from Kirkwood (1991) (Pg 21). We used a conversion factor of 2136 cases averted = 67 DALYs averted = 1 death averted, based on data on total deaths, DALYs, and cases lost due to diarrhea from the World Health Organization for 2004. As we now believe that the study’s authors made more reasonable assumptions than we did, we defer to their estimate and conclude that, under conditions similar to the ones in the study, spring protection is may be a highly cost-effective way to save lives in sub-Saharan Africa.

Regardless, this study is good news for donors interested in improving health through water infrastructure. Independent evidence of effectiveness lowers the burden of proof somewhat for a charity conducting spring protection. Still, we feel the burden of proof for such a charity remains higher than for one delivering simple, proven medical interventions, such as vaccines or treatments for tuberculosis, malaria, or intestinal worms. Here’s why:

  • Medical interventions are generally subjected to extensive, highly rigorous testing before being approved for use. There is currently only one rigorous study (that we know of) of the impact of spring protection.
  • We would guess that there is more variation in the protection of springs than in the administration of proven medical interventions. While some medical interventions may be complex, many, such as vaccines and tuberculosis treatment, involve providing a chemical compound in a standardized way. How spring protection is carried out, and therefore how closely it resembles the intervention studied in the RCT, is likely dependent on such variable characteristics as the physical features of the spring, locally available building materials, and the particular design chosen by the charity.
  • The effect of spring protection is also likely more variable than the effect of medical interventions. Local practices such as how water is stored, whether water is boiled before use, what percentage of each household’s water comes from the spring, and whether defecation takes place near springs are likely to affect how large of a determinant of diarrhea incidence spring protection is.

Four out of the eleven charities listed at our overview of water programs appear to do at least some spring protection, though none appear to focus on spring protection.* In investigating a charity working on spring protection, we would want to see evidence that the springs are monitored over time to determine whether springs remain in use and in good condition. Ideally, we would also like to see that disease rates have declined among users of protected springs in relation to a suitable comparison group. From what we’ve seen to date, this is a bar that water charities have yet to reach.

*Specifically:

  • Water for People notes that projects in Guatemala, Hounduras, Nicaragua, and Bolivia generally involve spring protection, while spring protection isn’t mentioned for the other 7 countries it works in.
  • WaterCan/EauVive mentions spring protection for one of the four countries it works in.
  • WaterAid lists spring protection as one of seven methods it uses to increase water supply. It also provides technical information on how springs are protected.
  • Ryan’s Well Foundation reports protecting springs in one of its nine projects in progress.

Neglected tropical disease charities: Schistosomiasis Control Initiative, Deworm The World

(updated on: ) | by Holden

There are a lot of reasons to be interested in charities focused on neglected tropical diseases (NTDs), and particularly on deworming schoolchildren.

We are positive on deworming as an intervention. But we feel that for individual donors, it is necessary to form confidence in charities, not just interventions. This is where our interest in NTD control hits major obstacles. This post goes through our questions and the difficulty we’ve had answering them, and focuses on charities focusing partly or fully on deworming.

Key questions

As far as we can tell (details below), the major organizations focused on deworming work heavily with developing-world governments. They spend money on both advocacy and subsidies for government control programs. We therefore have the following questions:

  • How do the charity’s funds break down between subsidies and advocacy?
  • For subsidies:
    • How much of the total spending on control programs has been covered by the charity, as opposed to the government?
    • How has the government been audited to ensure compliance with terms and conditions? Note that the Stop Tuberculosis Partnership serves in a similar role to NTD charities, subsidizing (through drug grants) government tuberculosis control programs, and it has an extremely thorough auditing process to ensure that drugs are used appropriately, that reported statistics are meaningful, etc. It has in the past discontinued funding for noncompliance. How do NTD control charities compare?
    • Does the charity intend to keep subsidies at the same level indefinitely, or does it intend for the government to take over activities? If the latter, what is its track record in accomplishing this very difficult task? Note that while we consider “sustainability” optional for some programs, we believe it is crucial for deworming, for reasons discussed previously.
  • For advocacy activities:
    • How much has been spent, what programs (and where) have been advocated, and to what results? What does the future advocacy plan and budget look like, and what might be expected from it? Cost-effectiveness of control programs is one question; the cost-effectiveness of advocacy funds could be another question entirely.
    • To the extent that advocacy has succeeded, has it resulted in (a) more government funding for NTD control with no offsetting cuts, i.e., increases in total medical budgets; (b) more government funding for NTD control at the expense of other health spending; (c) more funding for NTD control, contingent on more donor subsidies?
    • If (a), how has the increase in funding been financed? If (b), where have cuts been made, and to what extent has funding shifted from other worthy health spending? If (c), what are the answers to the key questions about subsidies (above)?

Schistosomiasis Control Initiative (SCI)

A key roadblock to evaluating SCI is that it does not share any information about its budget by program (past or planned). Last year, after failing to find this information on its website, we contacted Prof. Alan Fenwick, and after significant back-and-forth we were told that no budget information could be shared. This makes it impossible to answer key questions about the role of advocacy vs. subsidies, and where future donations are likely to go.

SCI also declined to answer our questions about impact of past work directly, instead suggesting that we search PubMed. We did so, and have also recently reviewed SCI’s updated website, summarizing what we found in this XLS file. Our conclusions:

  • There is serious evidence of the impact of past projects, a credit to SCI and something that is rare among charities in general.
  • SCI appears to have provided both funding and technical assistance in the past (and from phone conversations we also understand that it has done advocacy work). Without budget information, we do not know how its funds have broken down between these activities or the specifics of what has been paid for by SCI vs. governments.
  • SCI appears to have exited at least 3 countries (of 8 that we have information on), and the extent to which its programming has been sustained – both in terms of finances and quality – by the government is unclear.
  • We do not have information on how SCI audits government data and practices (though the direct evidence of impact is encouraging), or on the specifics of its advocacy work.

Deworm The World

We have essentially no information about Deworm the World.

We consider Poverty Action Lab itself to be a model of transparency, posting exhaustive information on studies both completed and in progress. But Deworm the World is on the opposite end of the spectrum, providing no substantive public information as far as we can tell.

We have in the past been tempted to recommend Deworm the World simply on the strength of our respect for Poverty Action Lab. But ultimately, conducting research projects is a different enough challenge from working with governments on large-scale programs that we think that doing so would be the wrong move – both in terms of the incentives it would provide and the good it would accomplish.

*At the time this post was drafted, Poverty Action Lab discussed Deworm the World in various places on its site. However, the links have since broken as Poverty Action Lab redid its website. We have not been able to find any mention of Deworm the World on Poverty Action Lab’s new website.