The GiveWell Blog

Against Promise Neighborhoods

| by Holden

We are in favor of scaling up proven programs, but against the Promise Neighborhoods initiative.

As far as we know, the only evidence that the Harlem Children’s Zone (or any similar approach) has been effective is the relatively recent study showing impressive effects on test scores at its charter schools. We discussed this study in a four-part series and concluded that:

  • The effect demonstrated was extremely impressive and unusual.
  • There are serious questions about how “real” the effect is (to what extent did it come from narrow “teaching to the test?”), how likely it is to be sustained as opposed to temporary, and how significant it is in terms of likely effects on actual life outcomes.
  • These questions aside, there are also major questions about just what aspect(s) of Harlem Children’s Zone are crucial and whether they can be replicated at all, let alone at a reasonable cost.

Given this situation, we don’t feel it’s time to attempt a replication in 20 communities at once, at a cost that seems likely to stretch into the billions if and when these replications are fully carried out.

We’re not just concerned about mis-spending money. We’re concerned about overreacting to evidence, overpromising results, and thus damaging the credibility of future proposals along these lines. We’re concerned that the funds will be allocated, the Promise Neighborhoods will be rolled out, and 10 years from now we’ll check back and see no narrowing of the achievement gap.

We hope that someday, there will be a truly replicable program with an extremely strong case that it can put a significant dent in the achievement gap. If and when that day comes, a failed Promise Neighborhoods scale-up – and any other oversold programs – will come back to haunt us.

We feel it is appropriate to pursue some replication of, and experimentation with, the Harlem Children’s Zone model. We feel a rollout of this magnitude would be a mistake.

Unitus

(updated on: ) | by Holden

Unitus today released a rather sudden announcement that it will be releasing its staff and “shift[ing] its resources and activities to areas of maximum socio-economic impact for underserved people throughout the world that have yet to attain either scale or commercial viability.”

I agree with Sean Stannard-Stockton that while the announcement presents the closing as a simple strategic change in direction (due to success, rather than failure), its suddenness is odd and raises many questions.

We have always found Unitus to be an opaque organization. It is one of several large U.S. microfinance organizations whose value-added is unclear.

We urge Unitus to provide more information than is in its press release.

If the “change of direction” is in response to failures and/or disappointments, we urge Unitus to be clear on this point and to share as much information as possible about what has happened so that others can learn from its experience.

If the “change of direction” is, as the press release implies, a big-picture strategic change brought about by less need for its activities, we have the following questions:

  • What, specifically, will Unitus be focusing on from here?
  • Why does its new focus require a complete change in personnel?

  • Why has the announcement been made so suddenly? Recent newsletters and press releases seem to imply that there are substantial unexploited opportunities for the work Unitus is doing; did these give the wrong impression? What has changed?

Coming on the heels of the LAPO controversy and our recent post about USAID’s “failure” to target the poorest, we see this event as yet another reason to be wary of the appealing stories associated with microfinance.

Aid is not the only thing that reduces poverty

| by Holden

Earlier this month, a Gates Foundation representative (Mark Suzman, Acting President of the Global Development Program) made a post epitomizing what I feel is a key fallacy in the world of giving: that any and all progress in struggling countries can be attributed to aid.

The thesis of his post is that “A greater focus on results and accountability means that overall aid spending has been getting smarter, more focused and more effective. Increasingly, taxpayer dollars are spent on proven interventions that are saving and improving lives.” And the only support given for this thesis is observations about aggregate improvements in health, wealth and education (drops in childhood deaths, drops in the number of people living under a given poverty line, etc.)

Mr. Suzman concedes that “Economic growth in China and India has been the primary engine of this improvement,” but in his discussion of other regions including Africa, there is a strong implication (difficult to convey in an excerpt, but clear if you read the piece) that the mere fact of improvement points to the effectiveness of aid.

There is no mention of possible non-aid-related factors behind the improvement in these regions, such as:

  • Improvements in government programs and government accountability, which could happen because of aid to governments or for other reasons
  • Improvements in technology
  • Local people making progress on their own problems, even if such progress isn’t visible in national-level GDP statistics

This fallacy is one that we see often. People often ask how we can recommend that donors not support certain areas, such as water or education, saying things like “If we don’t support these areas, who will?”

The answer may be that no one has to. It’s worth reminding ourselves that the first countries to emerge from poverty didn’t receive any aid from wealthier countries, and there is no easily discernible influence of aid in many of those that have emerged since.

People can and do solve their own problems. Rather than giving ourselves responsibility for everything they’re struggling with, we should focus on the areas in which we’re most likely to be able to help.

Singularity Summit

| by Holden

Among those who follow GiveWell, there is some interest in the Singularity Institute for Artificial Intelligence and its mission of lowering the risks associated with the creation of artificial intelligence that “[leaves] human abilities far behind.” We have been asked several times to share our views on its work and the value of a donation given to it.

My only knowledge of this issue, as of now, comes from reading Less Wrong and Overcoming Bias and speaking with the Institute’s President, Michael Vassar. I’m interested in this community partly because it has a lot of people (including Mr. Vassar) who think critically and analytically about how to accomplish as much good as possible, considering all options and putting positive impact above other considerations; in that sense their values overlap strongly with ours.

At this point

  • I believe that there are enormous risks and upsides associated with artificial intelligence. Managing these deserves serious discussion, and it’s a shame that many laugh off such discussion.
  • I do not feel that the Institute is a strong opportunity for a donor to accomplish good. I sketched my reasons in this comment and will eventually lay out my thoughts more thoroughly.
  • I do intend to learn more about this area and am open to changing my mind in either direction.

Consistent with the last point, I will be attending the Singularity Summit this year and encourage others interested in this topic to consider doing the same.

GiveWell will be moving to Mumbai (India) for 3 months

(updated on: ) | by Holden

GiveWell’s 3 full-time staff (myself, Elie Hassenfeld and Natalie Stone) will be living and working in Mumbai from mid-August through late November.

Developing-world aid has become a major focus for us, and we hope to have more opportunities to see aid work up close (along the lines of my trip to Africa earlier this year).

Please let us know if you have suggestions for charities we should visit, contacts at such charities, or any other advice/suggestions/contacts.

Microfinance’s “failure” to reach the poorest

(updated on: ) | by Holden

USAID’s most recent report on microfinance and microenterprise development tells an interesting story and, in my view, shows just how widely microfinance has been (and continues to be) misunderstood. While many advocate that microfinance institutions focus on people under the global “extreme poverty line”, USAID’s report implies that actually doing so is rare and even unrealistic.

Background: the myth of targeting the poorest

The international “extreme poverty line” is around the equivalent of US$1.25 per day, and around 1.4 billion people worldwide (and over half of those in sub-Saharan Africa) are estimated to live below this line (see the discussion in our international aid report).

Many seem to believe that people in this category are appropriate – even ideal – as clients. For example, see Opportunity International and Grameen Foundation stressing the need to reach the “poorest” and “most vulnerable.” Both Accion and CGAP cite the entire 3-billion-strong set of people under the US$2/day line as potential microfinance clients (upwards of 50% of this set falls below the “extreme poverty line”).

U.S. official aid seems to have taken this idea particularly far. For the past several years, USAID has been required by law to target the “very poor,” defined partly with reference to this “extreme poverty line”:

Both the Microenterprise for Self Reliance Act of 2000 (henceforth, the 2000 Act) and the MRAA mandate that at least half of all USAID funding for microenterprise development directly benefit the very poor. The 2000 Act initially defined the “very poor” as the bottom [poorest] half of those living below each country’s national poverty line … Subsequent amendments to the 2000 Act mandated a second, much more ambitious approach … the amended law created a second definition of the “very poor” — those living on less than the equivalent of $1 per day, calculated using purchasing power parity (PPP) exchange rates. The law made clear that, for any given country, the applicable definition of the very poor would be the more inclusive one.

(Note that “$1/day” may be a reference to the $1.25/day “extreme poverty line” discussed here – see note 6.)

Investigating actual poverty levels of clients

To its credit, USAID put significant effort into tracking whether it was actually meeting this goal, developing poverty assessment tools for assessing clients’ poverty levels and requiring certain grantees to use them. The results:

Among the eight microfinance institutions that applied and reported on the Poverty Assessment Tools, the average share of Funds Benefiting the Very Poor (FVP) is estimated at 28.5 percent, up from 16.3 percent in FY 2007. … For the 14 enterprise development programs that applied and reported on the Poverty Assessment Tools, average FVP is estimated at 26.0 percent, up from 20.5 percent in FY 2007 …

USAID did not come close to its target of 50% “extremely poor” clients. Furthermore, it concluded that continuing to push for this target would be unwise:

As matters stand, USAID sees no promising options for meeting the FVP target. It cannot do so by reallocating funds among its existing partners, because with the exception of one small program, none had more than 50 percent “very poor” clients. It cannot do so by shifting funds to established microenterprise organizations that are not already receiving USAID funding, because few if any such organizations are voluntarily applying the USAID-certified poverty assessment tools, and no such organization has offered solid evidence that it has more than 50 percent “very poor” clients …

Misguided target?

USAID does not conclude that microfinance/microenterprise projects should be de-emphasized (it observes that “the great majority of clients … are very poor, at least in commonly used terms”). Instead, it concludes that the idea of serving the poorest was unrealistic/inappropriate in the first place.

the overall pattern of results lend further weight to the point that USAID raised in last year’s Annual Report – that current law imposes too low a threshold for being “very poor.” This very narrow definition makes it impossible for USAID to allocate its microfinance and microenterprise funding so as to reach the legislative target of directing 50 percent of the benefits of microenterprise funding to the “very poor,” without undermining other goals emphasized in the same legislation, such as sustainability and support for broad-based economic growth.

Unfortunately, this definition of being “very poor” was adopted without any evidence that a 50 percent FVP target based on this definition could be reached. Two years of results using the poverty assessment tools strongly suggest that the target cannot be reached without inflicting undesirable side effects on sustainability and economic development. In short, USAID sees no realistic prospect of reaching the target contained in the law, and urges prompt and serious consideration of changes in the law. (Bold mine; italics in original)

I’m inclined to agree with USAID’s conclusion. I agree that people with incomes well above the “extreme poverty line” can still be very poor, certainly poor enough that I’m interested in donating to help them. So my point is not that microfinance is being carried out inappropriately, or is failing to reach the very poor.

Rather, I’m noting yet another way in which microfinance seems to have been badly misunderstood by its biggest funders and proponents. USAID, and by implication its grantees, seem to have thought that they were serving the world’s poorest – to the point of legislating it – without any data, and wrongly. It’s another debunked myth, and another sign that the funding and the stories have gotten ahead of the facts.