Normally, we focus on identifying outstanding charities, and minimize the time spent on opaque or otherwise lackluster ones. But lately, we’ve gone into a bit more detail about our take on several of the best-known and most appealing charities out there.

What all of the charities below have in common is that (a) we have major questions and concerns about their activities; (b) the information necessary to see how serious these concerns are does not seem to be available. (In most cases our assessment is based on significant back-and-forth with the charities themselves, though in some cases we are going off their website.)

• Kiva: commonly perceived as a way to (a) make interest-free loans to (b) entrepreneurs you personally select. In fact, it is neither. (See GiveWell Board member Tim Ogden’s summary of the recent discussion/controversy.)

  We ultimately see a Kiva “loan” as a gift to a microfinance institution, and we have major concerns about supporting any but the most outstanding microfinance institutions as well as concerns about Kiva’s due diligence process specifically.

  Full set of blog posts on Kiva

• Grameen Foundation and other large U.S. microfinance charities: in addition to our general concerns about microfinance charities (and the fact that these charities have not provided the information that would address them), we find that these organizations are rarely even clear about what their value-added is. Also see the disappointing (and eye-opening) evaluation of the Village Phone program.

  Full set of blog posts on Grameen Foundation

• Heifer International: commonly perceived as a way to “give a cow to a poor family as a gift,” but this is in fact a donor illusion -donations support Heifer International’s general “agricultural assistance” activities. We have concerns about this general area and concerns about giving livestock specifically. Neither Heifer’s website nor its grant application have provided the kind of information needed to address these concerns.

  Full set of blog posts on Heifer International

• Smile Train: Prof. Steven Levitt of Freakonomics sees it as using an innovative model to deliver cleft surgeries for $250 apiece. Our analysis suggests that it has “more money than doctors” and that it is therefore out of room for more funding. Smile Train’s recent removal of the charts we questioned and its own explanation of why it is giving grants to other major charities appear to support our analysis. In addition, we have major concerns about surgery charities that Smile Train has not provided enough information to address.

  Full set of blog posts on Smile Train

• UNICEF offers donors the chance to “buy” vaccines and other items. We suspect that this offer is a donor illusion, but the larger issue is that UNICEF provides almost no information at all about its huge variety of activities and how they are going. Most giant international aid charities (Save the Children, World Vision, etc.) are similarly complex and opaque, with CARE coming the closest to meaningful transparency.

  Full set of blog posts on UNICEF

• The Acumen Fund stresses its metrics-driven approach to “patient capital,” investing in businesses that aim for long-term sustainability (i.e., profitability) and social impact. As of now, we see no substantive available information about its track record.

  Full discussion

• The Robin Hood Foundation also stresses its emphasis on metrics, quantification, and analytical thinking. But we have found that it is unwilling to share any information about the results of its grantmaking. (Note that it also pushes the “0% overhead” donor illusion.)

  Full set of blog posts on The Robin Hood Foundation

• The Millennium Villages project states that it is “demonstrating that it’s possible to escape … extreme poverty” (front page of its website). As far as we can tell, very few results have been demonstrated to date, with a few boosts in crop yields as its main evidence of impact - evidence we feel is highly questionable.

  Full discussion

• The Worldwide Fistula Fund has been praised by Nicholas Kristof as “giving new life to the pariahs.” Our assessment is that the Fund is pushing a debatable agenda whose effectiveness has yet to be shown.

  Full discussion

• The Carter Center is, in our view, by far the “best bet” (for an individual donor, our focus) of the charities listed in this post. It has achieved substantial, demonstrable impact in the past through its work on guinea worm eradication, and unlike all of the charities listed above, it publicly discloses substantive technical information on the impact of its past activities.

  However, there is a crucial piece of information missing: how it allocates its budget between programs (in particular, how much it spends on its stronger and better-documented vs. more questionable programs). It has declined to share this (basic, in our view) information in repeated exchanges.

  Full set of blog posts on The Carter Center

We think the above charities are fairly representative of “average” charities in international aid. Some tell better stories than others and some have more disclosure than others. But in almost all cases, international aid charities are (a) carrying out complex projects that can fail to do good (or even do harm) in a variety of ways, and (b) not systematically sharing the information that would make it possible to assess how their work is going.

GiveWell is devoted to finding charities in which we can have more confidence. We’ll be discussing our two top-rated charities working internationally in forthcoming posts.

We’ve thought and written a lot about microfinance lately. As of now, here’s where we stand.

What microfinance is and isn’t

First, it’s important to recognize that most of what you’ve heard about microfinance is false. It isn’t primarily about funding business expansion.. It isn’t a “proven solution” to poverty. And it doesn’t leverage your donation far more than other options.

Rather, we think of microfinance as a way to help people with low, volatile incomes manage their financial lives, an idea that is well argued in the recent Portfolios of the Poor study. This study implies that microfinance is really about providing one more option for borrowing rather than the only way to borrow, and that the borrowing is continual rather than “one crucial loan to escape poverty” - more like a credit card than a business investment. (This would explain why “graduation” from microlending programs appears rare).

What to look for

Does microfinance do good? It depends on a lot of things.

• If loans are constantly and heavily subsidized, they can be thought of as similar to giving out cash, in which case our primary concern is that benefits reach the right people.
• On the other hand, if loans are not subsidized, a microfinance institution’s profits could be taken as a sign that it has paying customers. This in turn could be a sign that it is providing empowerment.

With the latter goal (which seems to be the more common one), there is a big question about what role donations can and should play. We have expressed serious concerns about mixing donations with for-profit enterprises, with the possible result that donations end up padding profits (concept; example). In addition, we worry that there are too many donations blindly chasing the microfinance “story,” with the result that donations end up disappearing into nebulous activities.

There is also a question about the extent to which loans are truly providing empowerment. There is evidence that borrowing is bad for at least some borrowers.

We have developed a set of critical questions both about microlending and microsavings, to get at the question of whether an institution is helping people. We’ve looked hard for organizations that can answer our questions.

What we’ve found

In trying to answer the above questions, we’ve become fairly pessimistic about the area of charitable microfinance in general.

• The large U.S. charities are rarely clear even about what their value-added is. To the extent that I’ve been able to see their due diligence on partners (most of which is unfortunately confidential), it seems focused on financial performance, with much weaker examination of social impact-related questions.
• Something as simple as the “repayment rate” turns out to be reported, by standard practice, in what we feel is a highly misleading way. We’ve been shocked at how hard it is even to get what we consider the “real repayment rate” out of a microfinance charity.
• We’ve also been surprised by how often people drop out of microlending programs, generally for negative reasons.
• All in all, the picture we have is of a sector that tells stories about helping people but largely measures and rewards financial performance. As a result, microfinance institutions seem to have strong incentives to maximize their scale and their profits, even if doing so isn’t good for the borrowers.

Bottom line

All in all, we would guess that microfinance as a whole has done a great deal of good, but has also probably done some harm. We are more pessimistic specifically about microfinance donations in the current environment. For the reasons outlined above, we believe that giving to an “average” or “typical” microfinance charity – or giving with an illusory “peer to peer” relationship as the extent of your due diligence – is a fairly bad bet. At the very least, it will deliver far less good, and far more potential harm, than the typical microfinance narrative suggests.

Yet we still find the basic idea of providing financial services to people with low and volatile incomes very appealing as a way to help people … if it is done in a way that stresses social impact and uses donations responsibly.

We believe that microsavings is a particularly promising area, although we haven’t found a microsavings charity we can be confident in.

We believe that the Small Enterprise Foundation is a microlending institution that is truly and appropriately focused on achieving positive social impact. We’ll be writing more about it.

We’re looking for a good option for U.S. donors interested in supporting microfinance. We’ve been examining the largest, most prominent U.S.-registered charities in this area: Grameen Foundation, Unitus, Accion, Women’s World Banking, Opportunity International and FINCA. All of these are large organizations that list a variety of “partner” microfinance institutions.

One thing that might surprise a donor in this area is that sometimes the “partnerships” overlap. For example, Women’s World Banking lists Enda Inter-Arabe in its network, and so does Grameen-Jameel (which itself is a joint venture of the Grameen Foundation). Neither organization’s profile mentions the other.

Grameen Koota lists the Grameen Foundation and Unitus (among others) as donors and lists Unitus and Accion as consultants. Unitus’s page on Grameen Koota doesn’t mention Accion; Accion’s doesn’t mention Unitus; the Grameen Foundation sounds like its relationship is currently more consultant-like than donor-like.

As you can see from the profiles linked above - and as we’ve found from talking to the large microfinance charities - most (not all) of their activities consist of consulting or “technical assistance,” rather than founding, directly supporting or investing in microfinance institutions. The relationship between them and their “partners” often appears relatively limited, and it’s difficult to pin down who’s responsible for what.

Focusing on technical assistance and consulting isn’t necessarily a bad thing - perhaps U.S. expertise is just what these institutions need to serve more clients and serve them better. But we aren’t sure this is the case - it’s also possible that the U.S. charities have sprung up because of strong donor demand for the “story” of microfinance, and are consulting because they have more than they can productively spend on investing and creating.

What we know is that it’s inherently hard to gauge a charity’s impact when its main activities consist of free (or heavily subsidized) technical assistance. Even if we had all the information we want about the partner MFIs themselves, we’d still need to understand the extent to which help with business planning or product design made a difference to them.

So far, we haven’t seen many signs that the U.S. charities are critically asking this question, or making it possible to hold them accountable for the answer.

Ever since I heard about the Grameen Foundation’s Village Phone program, I’ve been optimistic. The program involves helping people in remote villages run pay-for-use cellphone services: they get their cellphone, and a loan to buy it, via Grameen, then charge other villagers to use it. It’s an approach to fighting poverty that’s (a) relatively new; (b) using a product that hasn’t been available for a long time but seems clearly useful to anyone doing business in remote areas; (c) utilizing a “franchise model” where people in the villages take a stake in the product.

It was near the top of my “Probably helping people, even though we don’t yet have systematic evaluation yet” list. Now Chris Blattman points to a discouraging evaluation that found “absolutely no impact of the phones on trading activity or availability of goods in local markets” and very small (non-significant) impacts on profits and measures of well-being (school enrollment, consumption of meat, etc.)

This bottom-line result does not, by itself, mean the program “doesn’t work.” It could work very differently in different contexts (discussed below), and there are some possible issues with the paper (which is very recent, and is not a randomized controlled trial). But one thing I like about the study is that it doesn’t just discuss impact - it examines many aspects of the program, and exposes assumptions that may otherwise have gone unquestioned.

Assumption 1: the phones are in high demand and operators easily cover their costs. In fact, usage of the phones was around 4 hours a month, or 8 minutes a day (pg 19). As a result, profits from the phones were not enough to keep up with loan payments (pg 19). Grameen reportedly has responded by changing loan and franchise terms (pg 30). Tuvugane (pg 5), a less sophisticated phone product that was already common in the villages, may have been good enough for most purposes.

Assumption 2: farmers who use the phones benefit from better pricing power. Even though farmers with access to the phones became much more likely to arrange their own transport to market, there was no apparent effect on the prices they received for their goods, possibly due to established relationships with buyers (pg 16).

Assumption 3: if someone chooses to become a cellphone operator, they’re going to benefit from it. In fact, there was a very strange pattern in the businesses of people who became phone operators. Their hours worked rose significantly both for their new phone business and for their already-existing businesses, but their profits and wages paid did not rise (pgs 17-18). A possible explanation is that operators wanted to be available for cellphone users and so stayed at their workplaces longer, but that the extra hours didn’t translate into extra profits. In any case, it’s a pattern that doesn’t seem encouraging, and seems to deserve further investigation.

Bottom line: a product that was supposed to be helpful and in high demand arguably ended up as a bad investment for the franchise operators. This doesn’t mean it shouldn’t have been tried, or that it shouldn’t be tried in the future. But it points to the importance of testing assumptions empirically, rather than scaling up a program as widely as possible based on an appealing story.