The GiveWell Blog

More Than Good Intentions by Dean Karlan and Jacob Appel: Our review

(updated on: ) | by Holden

A few disclosures regarding this book:

  • Innovations for Poverty Action is a client of Sona Partners, and Tim Ogden of Sona Partners sits on our Board.
  • We are currently considering Innovations for Poverty Action for a recommendation.
  • We sent the authors of the book an early draft of this review and have made modifications after some back-and-forth.

I recently read More than Good Intentions, a book by Dean Karlan and Jacob Appel of Innovations for Poverty Action (which we’ve written positive things about before).

  • I thought the book was excellent, and I highly recommend it to anyone interested in effective giving and in development economics. I especially recommend it to anyone interested in microfinance – it has the best extended discussion of microfinance that I’ve seen.
  • I am a bit concerned that the book doesn’t adequately capture the distinction between “exciting ideas with some preliminary evidence behind them” and “the most proven interventions in aid.” I feel that programs falling in the latter category are largely not discussed in the book, and that this fact may be unclear to donors unfamiliar with the area.
  • The closing chapter of the book points readers to the Proven Impact Fund, which we understand is still a work in progress. After some discussion with Innovations for Poverty Action, we have determined that the Proven Impact Fund is unlikely to receive a GiveWell recommendation until and unless it becomes more established and has a track record. (We will be publishing the details of our views on the Proven Impact Fund at another time.) We are, however, evaluating Innovations for Poverty Action itself as a vehicle for donors to contribute to more high-quality research on aid.

Excellent discussion of microfinance and other topics

The book opens with a multi-chapter discussion of microfinance. Dean Karlan has been a leader in high-quality studies of microfinance, many of which have debunked the myths surrounding it, and I find his discussion of it to be the best I know of: engaging, educational, and sensible in its interpretation of the information we have.

The discussion centers around the idea – often lost in the stylized stories around microfinance – that different people have different needs, and that loans can be helpful to some and harmful to others. Karlan and Zinman’s study on consumer loans in the Philippines makes a strong case that when the timing is right, a loan can be very helpful even at a high interest rate. But the book also discusses people like Erlyn, whose business had a limited need for capital, such that the large long-term loans associated with her microfinance institution largely served to put her deeper into debt. The work of Karlan and others has shown how harmful many of the restrictions and rules of traditional microfinance – often based on romanticized notions of poor entrepreneurs – can be.

After discussing microfinance, More than Good Intentions goes through research on how to help the poor in other ways, from improving access to health services to raising attendance in schools. I was familiar with most of these studies prior to reading the book, but the book adds helpful context on the background and motivation for the various research projects.

One of the consistent themes of the research is the tradeoff between (a) giving clients the freedom and power to make their own choices; (b) finding “nudges” that can help people act more strongly in their self-interests. Much of the microlending research suggests that more freedom and discretion is better, but in other areas, clients seem to benefit from some restrictions – and in some cases the restrictions are even self-imposed (as with the SEED program, in which clients open savings accounts that can’t be withdrawn from until they save a certain amount).

It does seem to me that the set of investigations discussed in the book is missing an important piece. The studies cited generally compare one version of a charity’s program to another, or they compare it to “no intervention”, rather than compare a program to simply handing out cash equivalent to the program’s costs. To us this seems the most appropriate way to ethically and reasonably evaluate the value of programs: put them up against the simplest, most direct, and most empowering/discretionary of all interventions, a cash gift.

We understand that there have been studies in this field that do investigate unconditional cash transfers vs. other interventions; we also understand that funders are not always willing to finance a comparison to cash transfers as opposed to “no treatment.” Still, the fact that microfinance programs, free school uniform programs, etc. have always been compared to “no treatment” rather than “cash” seems suboptimal to us.

I also want to highlight the following anecdote from the beginning of the book, as a reminder of what low-quality evaluation (even at the largest and best-known organizations) can look like:

The first “impact evaluation” I ever saw of microcredit made my stomach ache. It was clearly intended to generate pretty numbers for a brochure to donors – not to determine whether something was really working. It asked clients something like, ‘You are eating better now, compared to before you joined FINCA, yes?’ … What I have since learned is that FINCA was doing just as much to measure its impact as anyone else. Which is to say, very little.

“Proven” vs. “exciting”

The book overall covers a lot of ground but doesn’t cover all the ground we would have liked it to, given its ambitions to encourage people to give to proven, effective interventions. This is best illustrated by the final section of the book, which aims to “leave you with seven ideas that have me [authors] particularly excited.”

The seven ideas are microsavings, reminders to save, prepaid fertilizer sales, deworming, remedial education in small groups, chlorine dispensers for clean water, and commitment devices. (The list of highlighted ideas has since been updated, as noted below.) I feel very strongly that this list (and its update) do not include the programs with the best track records in international aid. In my view, several proven health interventions – most notably immunization campaigns, distribution of insecticide-treated nets, and tuberculosis case finding and treatment – all have far better and more robust track records than any of the seven. The reasons I feel this way have to do with the distinction between micro evidence and macro evidence: the ideas in More than Good Intentions are all primarily supported by micro evidence (programs that took place on a small scale and were monitored and evaluated by academics, with the ultimate proof of effectiveness consisting in numerical differences between treatment and control groups) whereas the ideas I have listed are supported by both (not just randomized controlled trials, but also country-level success stories where a scaled-up program achieved an impact that is easy to see outside the confines of a study). I may elaborate on why this distinction is important to me in a future post.

I’m not sure that the authors of More than Good Intentions would disagree. So why does the book focus on relatively new ideas and give less attention to programs such as traditional immunization campaigns? I think there is a perfectly reasonable explanation:the authors are particularly excited about interventions that they and their colleagues have helped prove the promise of, as opposed to interventions like immunization campaigns that have been recognized as effective for decades.

There’s nothing wrong with being particularly excited about insights that are particularly new, counterintuitive, or simply a source of personal pride. However, donors need to recognize the difference between “exciting new ideas worth learning more about” and “the programs with the best track records,” and I am not sure that they will come away from More than Good Intentions with this distinction clear to them.

The closing chapter of the book points readers to the Proven Impact Fund, which we understand is still a work in progress (and unlike the list of ideas in the book, it includes interventions related to bednets and immunizations – though not the same interventions that I feel are most strongly associated with macro-level successes). After some discussion with Innovations for Poverty Action, we have determined that the Proven Impact Fund is unlikely to receive a GiveWell recommendation until and unless it becomes more established and has a track record. (We will be publishing the details of our views on the Proven Impact Fund at another time.) We are, however, evaluating Innovations for Poverty Action itself as a vehicle for donors to contribute to more high-quality research on aid.

In the meantime, we highly recommend More than Good Intentions as one of the most sophisticated discussions we’ve seen of microfinance and other fronts in the fight against poverty.

Three Cups of Tea scandal: Why we had the right bottom line on the Central Asia Institute

(updated on: ) | by Holden

There has been a lot of coverage of the scandals around the Central Asia Institute. The founder has been accused of fabricating inspiring stories, as well as of spending less than half of the millions of dollars he’s raised on building schools.

The Central Asia Institute receives four stars from Charity Navigator (archived) and had perfect ratings from Great Nonprofits prior to the scandal (archived), but GiveWell has declined to give this organization a recommendation or distinction (note that that page was published in mid-2009).

In a sense this doesn’t indicate impressive foresight on our part: nearly all charities we examine do not receive recommendations or distinctions, so it’s not as though we spotted the fabrications and financial mismanagement ourselves. Yet in the bigger picture, I see this incident as a vindication of our approach to giving: it’s a reminder that you shouldn’t give charities the benefit of the doubt.

Central Asia Institute is an education charity. Our questions for education charities are:

  • Is the goal to improve attendance or to improve school quality?
    • If the goal is to improve school attendance, what sorts of schools will beneficiaries be attending? What information is available on teacher attendance and quality of instruction? What evidence is available regarding the program’s effect on attendance?
    • If the goal is to improve school quality, is there evidence that similar activities have causally led to improved schools (in terms of attendance, test scores, graduation rates) in the past? Are indicators of school performance tracked over time, both before and after the interventions?
  • What evidence is available regarding the likely impact of any improved performance/attendance on later life outcomes? (For example – what are the economic opportunities that will be available to students and how do education levels relate to them?)
  • How would activities change if more revenue than expected was received? Would more revenue translate into more students served, and up to what point?

 

When we looked at Central Asia Institute’s website, we saw lots of stories but not a word to answer any of these questions substantively. So, as we usually do, we declined to give it a recommendation or distinction. As it turns out, the Central Asia Institute couldn’t have answered these questions well, because (a) it does appear to focus on school building rather than school quality, thus falling prey to one of our main concerns about education charities; (b) it apparently does not have strong monitoring and evaluation of its own schools. From the CBS story:

The IRS tax return Central Asia Institute filed last year included a list of 141 schools that it claimed to have built or supported in Pakistan and Afghanistan. Over the past six months, we visited or looked into nearly 30 of them. Some were performing well, but roughly half were empty, built by somebody else, or not receiving support at all. Some were being used to store spinach, or hay for livestock; others had not received any money from Mortenson’s charity in years.

If the Central Asia Institute had had anything on its website to indicate promise (such as an example study from a single school showing high-quality education), we would have opened a more in-depth investigation, including in-depth examination of the “What do they do?” and “Is there room for more funding?” questions, which would certainly have led us to the observation that the Central Asia Institute spends less than half its money on actually building and maintaining schools. (This point, raised by the CBS story, is easy to see from page 2 of the organization’s most recent tax return (PDF).) The fact that it still technically has a low “administrative expense ratio” is just a reminder of how easily manipulated, and ultimately meaningless, this metric is.

People frequently object to the fact that we recommend so few charities, and to our equating of “not enough evidence to evaluate” with “not recommended.” But the fact is that if a charity isn’t disclosing substantive information to answer the tough questions about its activities, you have no way of knowing whether it has the same kind of problems that the Central Asia Institute has. Certainly you can’t know from looking at its rating from other agencies.

We recommend very few charities, but we are able to stand behind those we do recommend. If you’re looking to do as much good as possible with your donation, consider choosing from a smaller menu to ensure that you get quality.

PSI rating change

(updated on: ) | by Elie

We recently published an updated review of PSI based on conversations over the past year with PSI staff and additional analysis. We now rate PSI as a “Notable” organization, and no longer list it among our top-rated charities. (Note that PSI is not the only organization whose rating changed in 2010: we similarly “downgraded” both Partners in Health and the Global Fund to “Notable” and the Stop TB Partnership to “Silver” from “Gold.”)

PSI is the organization GiveWell has followed the longest:

  • I chose PSI as the recipient of my first substantial charitable donation in December 2006 (the outcome of my GiveWell research when GiveWell was just a part-time project).
  • In GiveWell’s first round of full-time research in the fall of 2007, PSI was our top-rated charity in our “Saving Lives in Africa” cause, and GiveWell sent PSI a $25,000 grant. We believed that PSI’s cost-effectiveness was roughly in-line with its own $/DALY estimates.
  • In mid-2009, we conducted a more thorough and in-depth analysis of international aid charities reviewing hundreds of organizations (instead of merely reviewing the ones that applied for a grant as we had in 2007). In that process, PSI maintained its “Recommended” status and we ranked it fourth out of the hundreds of charities we had considered. In that report’s review of PSI’s impact, we questioned whether PSI’s programs cause behavior change (such as condom use by individuals who did not previously use condoms).

What changed?

In 2010, we returned to all our previously completed charity reviews to reconsider them and update them with current information. In that process, our conclusion about PSI changed, due specifically, to (a) our view of the way in which PSI uses and evaluates the monitoring data that they collect and (b) new data we found that provided estimates for changes in health behaviors across all of Sub-Saharan Africa.

In our July 2009 review of PSI, we had concluded that

Data from [PSI’s monitoring and evaluation] surveys indicates that PSI’s products are likely being used (more below). It is unclear, however, whether the availability of PSI’s products causes behavioral change (such as condom use by individuals who did not previously use condoms).

While we weren’t sure that PSI’s programs were effectively creating behavior change, we felt that PSI was carefully evaluating its programs in a way that would identify failing programs and lead to improvements. We were ambivalent about PSI’s impact but felt good enough about its monitoring and evaluation process to recommend it. We believe that a charity (a) focused on programs with strong track records (as PSI is by promoting condom and bednet use) that (b) has an unusual degree of transparency while (c) carefully evaluating its programs to determine whether they are working is worth recommending to donors.

Earlier this year, PSI questioned our conclusion that its programs don’t cause behavior change. PSI made its case for impact by stating that “out of 163 available multi-round studies of behavior change, 82 have shown statistically significant impact on at least one behavior change indicator.” We didn’t find this case compelling as we see many ways PSI could measure these results without having significant impact. Specifically:

  • PSI’s test for “impact” – a single indicator showing a statistically significant change in behavior – seems unreasonably generous. PSI didn’t offer a compelling justification for the test it used. This led us to question the quality of process PSI uses to monitor and evaluate whether its programs are largely succeeding or failing.
  • We also found data providing estimates for bednet use across Sub-Saharan Africa and compared this data to measured changes in PSI’s focus areas to broader changes in bednet and condom use across Sub-Saharan Africa. Our analysis of this data shows that changes of health product use in PSI programs seems in line with changes in non-PSI areas across Sub-Saharan Africa (details in our review of PSI’s impact): PSI’s average change was 3-4% for condoms (compared to 2-3% across Sub-Saharan Africa) and 12-14% for bednets (compared to 23-28% across Sub-Saharan Africa). While we do not believe this demonstrates that PSI is not having impact, this data does shift our assessment of the likelihood PSI’s programs are effective and increases the burden of proof on PSI to demonstrate its programs’ effectiveness.

In part, the downgrade is a reflection of the fact that we are continually digging deeper and raising the bar higher for our recommendations. When we first encountered PSI, it was the only nonprofit we could find that seemed to have any meaningful data on its impact; the limited slice of data we looked at seemed to indicate impact and to be consistent with its own cost-effectiveness estimates; and that was enough for us. By now, we are more careful about demanding data that is broadly representative of the full organization’s activities, and in our fuller analysis of PSI’s data we were less compelled. We also place high value on admitting and responding to shortcomings. If PSI agreed with our view that its data does not convincingly demonstrate impact, we would consider a higher rating based on discussions of its ongoing learning and adjustment.

We still believe that PSI stands out from the vast majority of international aid charities because of its focus on programs with strong track record and its unusual and commendable degree of transparency. (If I had to guess I’d say that PSI is probably in the 99th percentile of all international aid charities in terms of its impact per dollar.)

Nevertheless, our focus at GiveWell is identifying the very best charities and for the reasons above, we no longer feel comfortable placing PSI in that group. We plan to continue to watch PSI carefully and reevaluate its status as new information becomes available.

Japan update 4/13/11

(updated on: ) | by Holden

It’s been roughly a month since the devastating 9.0 earthquake in Japan. We’ve been continuing to follow the situation and re-examine our recommendations for donors. At this point we feel that

  • The situation in Japan has recently worsened, with aftershocks and negative developments regarding the nuclear crisis. It is too early to tell whether this will change our take on the government’s ability to finance an appropriate relief/recovery effort (which in turn would leave room for donations to do more good).
  • Putting the recent worsening aside and looking at new information over the past couple of weeks, we feel there is a slightly stronger case than before against our bottom line, but not strong enough to overturn our recommendations. We still recommend Doctors Without Borders, which helps with relief and recovery efforts worldwide (including far less well-resourced efforts). For those looking to help Japan specifically, the Japanese Red Cross is the best option we see.

Details follow.

Update on OCHA and ReliefWeb

It remains the case that – as we have observed in past updates – the Financial Tracking System lists appeals for many countries but lists no appeal for Japan (archived).

In our previous update, we cited a staffer as stating that the lack of an official appeal reflected “the Japanese national capacity to respond.” Now OCHA has made more explicit official statements than before about the need for assistance.

  • A March 28 situation report stated:

    the need for any further international humanitarian presence or internationally procured relief supplies is limited and any such assistance should only be provided upon the request of the Japanese Government and in accordance with their stated criteria. In addition it is important not to overburden affected prefectures and local communities who are working at full capacity and do not have the resources to coordinate unsolicited offers of assistance

  • The latest OCHA report is from April 1 (archived) and states:
      It is still unknown how many people are living outside evacuation centres but the number is certainly large. Many people who were originally living in evacuation centres have returned to their homes even if they were damaged and without water and power. These people do not receive basic supplies from the municipal authorities but are becoming the focus of attention by local NGOs and volunteers …

      The team notes that whilst coordination challenges remain the Government of Japan can and will cover the needs of the affected population and that, with some very specific exceptions and then only at the direct request of the Government, there is no requirement for further international assistance at this time …

      OCHA has officially ended its support role in-country to the Government of Japan for the emergency. NGO consortia Japan Platform and Japan NGO Center for International Cooperation will take the lead in information exchange and coordination of international NGOs with their local partners.

Update on major funders

In our previous update, we did a very rough comparison of the responses to disasters in Haiti and Japan by major funders, i.e., governments and foundations. Since then, we have been pointed to a better source of data on aid from the U.S. specifically, which allows us to directly compare aid in the month after the Haiti earthquake to the month after the Japan earthquake.

We compiled data from Haiti fact sheets and Japan fact sheets into this spreadsheet (XLS). This chart shows total aid to each country from all USAID sources, with the number of days after the disaster (1/13/2010 for Haiti; 3/11/2011 for Japan) on the x-axis:

Haiti received funds from six different USAID agencies, while Japan received funds from only two: the Department of Defense (DoD) and the Office of Foreign Disaster Assistance (OFDA). DoD provided $252 million worth of aid to Haiti in the first month, vs. just over $70 million to Japan (likely mostly in the form of direct assistance rather than cash). OFDA provided just over $200 million to Haiti, vs. ~$7 million to Japan (nearly all of which was committed within the first week).

As we observed in our previous update, we find it unlikely that the U.S. would provide less aid to Japan (an ally) than it had to Haiti, unless it perceived limited needs for its assistance.

We haven’t done similar analysis of aid from other governments. The ratio between U.S. aid and total aid from other governments is about the same now as it was in our previous update (the U.S. has given about 2x as much as all other governments combined).

Another change since our last update is that the Gates Foundation made a $1 million grant to Mercy Corps, which we questioned it about (and posted its answers).

Update on the Japanese government

  • The OCHA report linked above states that “The Government of Japan has received 134 offers of assistance from countries as well as 39 offers from international organizations. It has accepted relief items from 29 countries and international organizations.”
  • An AlertNet story about the possible future impact of a future earthquake in Tokyo states:

    “Japan has enough savings surplus to fund reconstruction of (the quake-hit northeast) … but if you had to multiply the costs for Tokyo, Japan would become dependent on foreign funds,” said Jesper Koll, director of equities research at JP Morgan in Tokyo.

    Along with the Moody’s statement in our last update, this is an additional reason to believe that the government can finance an appropriate relief/recovery effort. As a side note, this article makes the disturbing argument that a 7.3 earthquake (much less severe than the one that hit Sendai) could be even more devastating than this earthquake if it hit Tokyo – and this disaster has a high chance of occurring over the next 30 years.

  • I haven’t seen much criticism of the government’s relief effort (as opposed to its response to the ongoing nuclear crisis), but the Economist gives harsh criticism and implies that the government has not done what it could:
      For all that Mr Kan has attempted to be seen at the front, in Tokyo the sense of a looming humanitarian crisis in the north has been slow to sink in. That is partly because nuclear worries have absorbed much of the government’s attention. Few politicians in a centralised system have bothered to travel north themselves. The media, taking their cue from the Tokyo establishment, have not thought properly to report the unfolding struggle for food and fuel.

      Yet businessmen and victims say supplies are being held up as bureaucracies fall back on tired old rules and straitjacket procedures. Lorries full of supplies have been unable to get petrol on the empty expressway north, reserved for “emergency” vehicles. While this severe shortage of fuel spread through northern Japan, oil companies were sitting on huge supplies which by law they had to keep in reserve. If ever an occasion for their use was justified, it was this catastrophe. Yet the government took ten days to beg for (not order) their release. From the start, Mr Kan should have declared a state of emergency. Even now, clear lines of authority for handling the many-headed crisis have not been properly established.
  • The worsening of the situation with recent events may make it harder for the government to fully fund the recovery effort: “The government and main opposition party have agreed to a spending package to get some reconstruction work started, but setting a large additional budget will be difficult due to Japan’s heavy debt burden.”

Japanese Red Cross

  • The latest Japanese Red Cross information bulletin is from March 28 (archived). (As a side note, the list of updates I was using before seems to have disappeared (see this page as it appeared previously).) Like the previous information bulletin (discussed in our last update), it has somewhat difficult-to-interpret language regarding donations: “JRCS is receiving cash contributions from some Red Cross Red Crescent National Societies in the spirit of solidarity.”
  • We have learned that Red Cross societies do not generally use the Reliefweb system for appeals (a search for appeals on Reliefweb from Red Cross societies yields very few results). Therefore, the fact that the Japanese Red Cross has not appealed through this system is not evidence against room for more funding; its lack of an appeal through the Red Cross system was discussed in our last update.
  • In past updates, we stated that the Japanese Red Cross appeared to be planning direct cash distributions to survivors, but that this was not entirely clear. Media reports since then have clarified the situation: an Alertnet story explicitly states that “The 108 billion yen ($1.26 billion) of relief money is meant to be handed to disaster victims in cash.” (The L.A. Times states this as well.) As stated in earlier updates, we support the idea of cash distributions to survivors.
  • There has been some criticism (see the links immediately above) that distribution has been too slow; the Red Cross’s response is essentially that it is difficult to distribute so much money both quickly and fairly.

International charities

The Chronicle of Philanthropy (archived) reports $246.9 million in donations, most of it to the American Red Cross ($169.5 million), Americares ($5.4 million), Catholic Relief Services ($4.9 million), Mercy Corps ($6.6 million), the Salvation Army (~$5.6 million), Save the Children ($9.6 million), United Way Worldwide (~$10 million) and World Vision ($7.9 million).

As in previous updates, we scanned the websites of particularly prominent charities (including those listed directly above) to see what they are disclosing about their activities. In general, they seem to be focused on distribution of various supplies and on maintaining various forms of “child-friendly spaces” and psychosocial support.

  • Americares reports a “relief shipment, sent from AmeriCares warehouse in Connecticut at the invitation of the Japanese government … Containing over 17 tons of medicines, medical supplies and hygiene items, the shipment, valued at more than $525,000 will be received by AmeriCares partner, the Tohoku University Hospital.”
  • Catholic Relief Services (archived) states that it is not involved in immediate relief but plans on being involved in long-term recovery (specifics are not given).
  • Mercy Corps (archived) has been posting frequent updates; it appears to be working with a local partner on both immediate relief and longer-term recovery (we have discussed the latter in a previous post).
  • Salvation Army (archived) states that 100% of donations will be sent to the Salvation Army Japan (archived), whose last update (March 14) states that it “has three emergency service relief teams operating in areas devastated by the earthquake and tsunami” and that “Arrangements are being made for emergency service personnel from The Salvation Army International Headquarters (IHQ) to go to Japan to assist with the Army’s relief effort.”
  • Save the Children (archived) has posted a briefing document emphasizing distribution of supplies (mostly not specified, aside from “back-to-school supplies”), construction of “Child-Friendly Spaces” and “psychosocial support.” Its longer-term plan includes a fund to “appeal for, assess and award grants to national and community groups involved in re-establishing child care and education” and pursuing advocacy toward the goal that “Child Friendly Spaces and childcare services are included in all prefecture planning, preparation and response operations.”
  • United Way Worldwide (archived) states that “The Central Community Chest of Japan has helped provide supplies to volunteer centers including: bicycles, mattresses, cell phones, and laptops. Additional supplies requested by the volunteer centers include motorbikes, prefab houses and tents, copy and fax machines, PCs and printers.”
  • World Vision states, “We now expect to have received enough donations to fully fund this intended 24-month response. We are encouraging those who wish to continue to help to donate to World Vision’s general Disaster Response Fund.” Regarding the specifics of its activities, it states that “we will focus on further distribution of relief and recovery items, continuing to set up protection programs for children and the elderly, while establishing community kitchens in affected areas” but is not specific about longer-term plans (“World Vision will be looking at how to best support families by providing necessary supplies to … shelters”).
  • Doctors Without Borders (archived) has put out one update since our last update, stating that “A team of six MSF psychologists have started working with the survivors of the massive earthquake and tsunami that hit northeast Japan on March 11.”

We have included the most substantive information we could find in most cases; the websites are generally fairly light on details. We have not seen any nonprofit that appears to be planning to distribute cash directly to survivors, as the Japanese Red Cross is.

A BBC article also notes that the UK’s Disasters Emergency Committee, which “coordinates the efforts of 13 UK charities during international crises” has refrained from an appeal for Japan.

We continue to be concerned about the idea that these organizations may be more interested in the appearance of helping (and the money that comes with it) than in being as helpful as possible (which, according to OCHA, could often mean staying out of the way). As discussed previously, the apparent lack of emphasis on cash gifts to survivors (for the longer-term phase) contributes to this concern.

Japanese charities

The OCHA report discussed above mentions three sources of coordination/information on nonprofits’ efforts (note that the second two of these links are in Japanese, and I am working off translations from the Google Chrome browser):

  • The Japan Platform (archived) has posted three updates since our last update. A couple of highlights:
    • The April 1 set of recommendations for international NGOs states that “it is generally not lack of relief supplies which is a problem, but more for the issue of better coordination to reach the needy. JPF and JANIC are trying to fill the gap of the needs from the field, and we are grateful for your understanding and long term support.” It also states that “Some Japanese NGOs, which are well organized and self-sustainable (food, accommodation, and communication etc.), have started their activities.” It advises international NGOs to consider partnering with Japanese nonprofits and to coordinate closely with local Disaster Volunteer Committees.
    • The latest general update has a table (page 6) and map (page 7) giving very broad overviews of nonprofits’ activities and funding. Many, but not all, local nonprofits seem to be in the “assessment” phase. The table gives financial figures in yen for broad activities, which I would guess refer to expected expenses. The figures are difficult to read, but I estimate that they add up to about 166 million yen; private donations are reported at a little over 3.8 billion yen.
  • JANIC (archived) has posted 8 updates since our last update, but nothing that appears to shed light on the activities and needs of nonprofits.
  • Japan Civil Network for Disaster Relief in East Japan (archived) was not discussed in our last update. It provides a large list of participating organizations but no other information that I can find on activities and funding needs.

Bottom line

In my view, the case against our previous bottom line has strengthened slightly for two reasons:

  • There have been accusations of slow/insufficient responses by the Japanese government and Red Cross, and if true these would leave more room for nonprofits to add value even if not invited/funded to do so by the government and Red Cross. We don’t know how valid these accusations are.
  • Nonprofits have published more substantive information than before on their activities, and there is not as much language implying that they are still essentially on standby. That said, the overall substantiveness of the reports is still generally low, and the focus on providing services (rather than cash) over the long term concerns us. In addition, just because nonprofits are active doesn’t mean that (a) their assistance is adding value relative to the government and Red Cross; or (b) they have room for more funding.

A donation to one of the local coordinating bodies (Japan Platform, JANIC, Japan Civil Network for Disaster Relief in East Japan) may result in aid being delivered somewhere the government and Red Cross aren’t reaching. However,

  • Criticism of the official relief effort still appears relatively rare to us and we don’t know how credible such criticism it is (we’d expect some criticism even of the best relief effort, given the difficulty of the situation).
  • Looking at the words and actions of OCHA and government funders still paints a picture of very limited overall needs for non-government assistance.
  • The numbers we have imply that nonprofits have gotten more than enough funding for what they are planning.
  • The reasons we gave previously for preferring the Red Cross to other nonprofits, if you are intent on having your funds spent in Japan, still stand.

  • Therefore, we stand by our previous bottom line.

The Gates Foundation’s grant for Japan relief: Our questions and its response

(updated on: ) | by Holden

Previously, we went through all the evidence we could find regarding whether Japan disaster relief has what we call room for more funding. One of our observations was that

The Gates Foundation, in particular, gave two grants within a week of the Haiti earthquake but doesn’t appear to have given anything for Japan so far.

Since then, The Gates Foundation has announced (archived) a $1 million grant to Mercy Corps.

The situation in Japan is confusing to donors, and the Gates Foundation is better positioned than individuals (and better positioned than we are) to sort through the confusions. If it posted a substantive explanation of its grant – and answers to the natural questions this grant raises – it could be a great help to individual donors, who have given over $161 million (U.S. donors only) to the relief effort.

We are therefore disappointed with its communications to date. On its website, it has put out only a short statement, with no explanation of the decision other than “While the emergency in Japan falls outside the typical scope of the foundation’s Emergency Response grantmaking, the magnitude of the crisis called for a response.” It provided an additional statement in response to our questions, which is posted below.

The Gates Foundation grant raises the following questions:

  • Out of all the organizations soliciting funding for Japan disaster relief/recovery, how and why was Mercy Corps chosen?
  • In particular, why was Mercy Corps chosen over:
  • A Mercy Corps representative has made it clear that it does not intend to spend all allocated funds on emergency relief: (archived):

    We are going to do post-trauma work with kids, and we are looking at how to make that really culturally appropriate … We are also going to get involved in the local economy. We are probably going to distribute vouchers that people can use to get into the stores as they reopen and buy goods, and pump up the local economy. We are going to look at small businesses that don’t have insurance and don’t have access to government funding, and may need support.

How does this square with the Gates Foundation’s emphasis on emergency relief in its release? (It discusses Peace Winds’s work in emergency services but does not mention these less emergency-relief-oriented activities.)

  • Why did the Gates Foundation make a grant to Mercy Corps specifically for Japan, instead of giving it a grant to be allocated where most needed (as Mercy Corps requested (archived))?
  • How does this grant square with the statement by OCHA that “the need for any further international humanitarian presence or internationally procured relief supplies is limited and any such assistance should only be provided upon the request of the Japanese Government and in accordance with their stated criteria”? Has the aid of Mercy Corps been requested by the Japanese Government?
  • Why was the grant for $1 million and not more or less?
  • Does the Gates Foundation have advice for individual donors interested in doing as much good as possible? Should they give to Mercy Corps rather than to a Red Cross organization? Should they give to help Japan rather than another of the many urgent situations worldwide (archived)?We sent a draft of this post to the Gates Foundation last Friday, and after some back-and-forth we received the following statement as its response (posted with permission):
      The emergency in Japan falls outside the scope of the foundation’s usual emergency response grantmaking, since our work is largely focused on developing nations, but the foundation felt the severity of the earthquake and tsunami called for a response.
      Mercy Corps is experienced in delivering humanitarian aid to wide geographic areas and in complex logistic situations. In addition, they also have a long-standing partnership with Peace Wind Japan, which is actively involved in the ongoing relief efforts.
      We have also worked with Mercy Corps during a number of emergency relief operations, including last year’s earthquake in Haiti.
      We are fully supportive of the work Mercy Corps and Peace Wind are doing in Japan. We will continue to monitor the situation and would urge people interested in making a donation to the relief efforts there to contact one of many international organizations working in Japan. A list of our partners who do emergency relief work, many of whom are already involved with Japanese relief efforts, can be found on the emergency response page on our website.

    Our takeaway is that the Gates Foundation grant doesn’t provide much reason to change our existing view of the situation. We believe that Mercy Corps’s apparent plans (quoted above) fall firmly within the category of “restitution” (and we have questioned the appropriateness of these plans, relative to giving out cash to survivors).

Assorted links 4/5/11

| by Holden

A few links from the past month that we aren’t doing full posts on, but might be of interest for people interested in effective giving:

  • Via Chris Blattman, a randomized controlled trial “find[s] no evidence that teacher incentives increase student performance, attendance, or graduation, nor … any evidence that the incentives change student or teacher behavior” (note this study took place in the U.S.) This can be added to a long list of high-quality studies with disappointing outcomes in the area of improving education in the U.S. (For a relatively recent review of these studies, see page 2 of a book chapter linked from the same post.) We think it’s important to look for rigorous evidence of effectiveness when giving to improve education, rather than assuming that logical-seeming programs will be effective.
  • A recent study (PDF) argues that KIPP, one of our top U.S. charities, relies on high attrition (i.e., many of its students leave over time) and high funding to achieve its results. KIPP responds with objections to the study’s methodology. I haven’t thoroughly reviewed the study or the response.

    My first impression is that the study is indeed seriously flawed, as KIPP suggests, but it’s also important to note that our endorsement of KIPP does not depend on the idea that it can serve all students or can operate on the same terms as public schools. For us, it’s rare enough – and valuable enough – to find a charity that can reliably improve education outcomes and narrow the achievement gap.

  • Felix Salmon refers to Michael Bloomberg’s promotion of giving to Japan as “philanthropy theater”: “The point is really to be seen to be doing good, to feel as though you’re making a difference.” I agree with this characterization of such giving, and I think it’s a good example of giving that’s all about the giver, not the recipient.
  • Sean Stannard-Stockton discusses his “four approaches to philanthropy” and makes a point that I think is important: “effective giving” is often conflated with what he calls “Strategic Philanthropy,” in which the funder diagnoses social problems and designs solutions themselves, seeing nonprofits as mere contractors. We find this a particularly challenging and un-promising form of giving, and we prefer what Sean calls “charitable giving”: giving to nonprofits that have found their own solutions, already know what works, and are able to help more people with more dollars.
  • Aid Watch publishes unsatisfying responses from World Vision to its questions about giving away shirts. The exchange is reminiscent of many of our past exchanges with charities.