Back in December, we expressed concerns about LAPO, one of Kiva’s largest microfinance partners. Last month, the New York Times ran an article implying criticism of LAPO. Now, Kiva has suspended its partnership with LAPO. A couple of questions this raises:

Which of the several objections to LAPO have led to the suspension?

Several concerns have been raised about LAPO:

• The New York Times article focuses largely on the high interest rates and forced savings (which cause the interest rates to be higher). We have argued that focusing on interest rates in isolation is a mistake.
• The Times article also raised concerns about LAPO’s transparency/accountability, including the question of whether it is collecting savings without the right legal license and the question of whether it is misleadingly presenting its interest rates to the outside world.
• To us, the most compelling concern about LAPO is its 49% dropout rate.

How many more of Kiva’s partners might have similar problems, and what is Kiva doing to address this concern?

As the case of LAPO shows, the interest rates reported to Kiva may not fully capture what clients are paying. More broadly, it appears to me that Kiva’s due diligence on partners is intended to assess the risk of money being lost and/or going to illegitimate organizations, but not the more difficult-to-assess risk encountered with LAPO: that a partner’s activities may be leading to overindebted, misinformed, and/or dissatisfied clients, and thus result in financial success but not positive social impact. The stories posted on Kiva’s website do not, in my view, help to address these risks either.

When the New York Times discovers a problem, makes it very public and causes change, this is a good thing. But I don’t think we can expect this mechanism to reliably address the broader issue. Kiva has had enormous success getting people excited about microfinance and getting capital to MFIs; I would like to see more work go into making sure those MFIs are benefiting the people they serve.

Normally, we focus on identifying outstanding charities, and minimize the time spent on opaque or otherwise lackluster ones. But lately, we’ve gone into a bit more detail about our take on several of the best-known and most appealing charities out there.

What all of the charities below have in common is that (a) we have major questions and concerns about their activities; (b) the information necessary to see how serious these concerns are does not seem to be available. (In most cases our assessment is based on significant back-and-forth with the charities themselves, though in some cases we are going off their website.)

• Kiva: commonly perceived as a way to (a) make interest-free loans to (b) entrepreneurs you personally select. In fact, it is neither. (See GiveWell Board member Tim Ogden’s summary of the recent discussion/controversy.)

  We ultimately see a Kiva “loan” as a gift to a microfinance institution, and we have major concerns about supporting any but the most outstanding microfinance institutions as well as concerns about Kiva’s due diligence process specifically.

  Full set of blog posts on Kiva

• Grameen Foundation and other large U.S. microfinance charities: in addition to our general concerns about microfinance charities (and the fact that these charities have not provided the information that would address them), we find that these organizations are rarely even clear about what their value-added is. Also see the disappointing (and eye-opening) evaluation of the Village Phone program.

  Full set of blog posts on Grameen Foundation

• Heifer International: commonly perceived as a way to “give a cow to a poor family as a gift,” but this is in fact a donor illusion -donations support Heifer International’s general “agricultural assistance” activities. We have concerns about this general area and concerns about giving livestock specifically. Neither Heifer’s website nor its grant application have provided the kind of information needed to address these concerns.

  Full set of blog posts on Heifer International

• Smile Train: Prof. Steven Levitt of Freakonomics sees it as using an innovative model to deliver cleft surgeries for $250 apiece. Our analysis suggests that it has “more money than doctors” and that it is therefore out of room for more funding. Smile Train’s recent removal of the charts we questioned and its own explanation of why it is giving grants to other major charities appear to support our analysis. In addition, we have major concerns about surgery charities that Smile Train has not provided enough information to address.

  Full set of blog posts on Smile Train

• UNICEF offers donors the chance to “buy” vaccines and other items. We suspect that this offer is a donor illusion, but the larger issue is that UNICEF provides almost no information at all about its huge variety of activities and how they are going. Most giant international aid charities (Save the Children, World Vision, etc.) are similarly complex and opaque, with CARE coming the closest to meaningful transparency.

  Full set of blog posts on UNICEF

• The Acumen Fund stresses its metrics-driven approach to “patient capital,” investing in businesses that aim for long-term sustainability (i.e., profitability) and social impact. As of now, we see no substantive available information about its track record.

  Full discussion

• The Robin Hood Foundation also stresses its emphasis on metrics, quantification, and analytical thinking. But we have found that it is unwilling to share any information about the results of its grantmaking. (Note that it also pushes the “0% overhead” donor illusion.)

  Full set of blog posts on The Robin Hood Foundation

• The Millennium Villages project states that it is “demonstrating that it’s possible to escape … extreme poverty” (front page of its website). As far as we can tell, very few results have been demonstrated to date, with a few boosts in crop yields as its main evidence of impact - evidence we feel is highly questionable.

  Full discussion

• The Worldwide Fistula Fund has been praised by Nicholas Kristof as “giving new life to the pariahs.” Our assessment is that the Fund is pushing a debatable agenda whose effectiveness has yet to be shown.

  Full discussion

• The Carter Center is, in our view, by far the “best bet” (for an individual donor, our focus) of the charities listed in this post. It has achieved substantial, demonstrable impact in the past through its work on guinea worm eradication, and unlike all of the charities listed above, it publicly discloses substantive technical information on the impact of its past activities.

  However, there is a crucial piece of information missing: how it allocates its budget between programs (in particular, how much it spends on its stronger and better-documented vs. more questionable programs). It has declined to share this (basic, in our view) information in repeated exchanges.

  Full set of blog posts on The Carter Center

We think the above charities are fairly representative of “average” charities in international aid. Some tell better stories than others and some have more disclosure than others. But in almost all cases, international aid charities are (a) carrying out complex projects that can fail to do good (or even do harm) in a variety of ways, and (b) not systematically sharing the information that would make it possible to assess how their work is going.

GiveWell is devoted to finding charities in which we can have more confidence. We’ll be discussing our two top-rated charities working internationally in forthcoming posts.

We’ve thought and written a lot about microfinance lately. As of now, here’s where we stand.

What microfinance is and isn’t

First, it’s important to recognize that most of what you’ve heard about microfinance is false. It isn’t primarily about funding business expansion.. It isn’t a “proven solution” to poverty. And it doesn’t leverage your donation far more than other options.

Rather, we think of microfinance as a way to help people with low, volatile incomes manage their financial lives, an idea that is well argued in the recent Portfolios of the Poor study. This study implies that microfinance is really about providing one more option for borrowing rather than the only way to borrow, and that the borrowing is continual rather than “one crucial loan to escape poverty” - more like a credit card than a business investment. (This would explain why “graduation” from microlending programs appears rare).

What to look for

Does microfinance do good? It depends on a lot of things.

• If loans are constantly and heavily subsidized, they can be thought of as similar to giving out cash, in which case our primary concern is that benefits reach the right people.
• On the other hand, if loans are not subsidized, a microfinance institution’s profits could be taken as a sign that it has paying customers. This in turn could be a sign that it is providing empowerment.

With the latter goal (which seems to be the more common one), there is a big question about what role donations can and should play. We have expressed serious concerns about mixing donations with for-profit enterprises, with the possible result that donations end up padding profits (concept; example). In addition, we worry that there are too many donations blindly chasing the microfinance “story,” with the result that donations end up disappearing into nebulous activities.

There is also a question about the extent to which loans are truly providing empowerment. There is evidence that borrowing is bad for at least some borrowers.

We have developed a set of critical questions both about microlending and microsavings, to get at the question of whether an institution is helping people. We’ve looked hard for organizations that can answer our questions.

What we’ve found

In trying to answer the above questions, we’ve become fairly pessimistic about the area of charitable microfinance in general.

• The large U.S. charities are rarely clear even about what their value-added is. To the extent that I’ve been able to see their due diligence on partners (most of which is unfortunately confidential), it seems focused on financial performance, with much weaker examination of social impact-related questions.
• Something as simple as the “repayment rate” turns out to be reported, by standard practice, in what we feel is a highly misleading way. We’ve been shocked at how hard it is even to get what we consider the “real repayment rate” out of a microfinance charity.
• We’ve also been surprised by how often people drop out of microlending programs, generally for negative reasons.
• All in all, the picture we have is of a sector that tells stories about helping people but largely measures and rewards financial performance. As a result, microfinance institutions seem to have strong incentives to maximize their scale and their profits, even if doing so isn’t good for the borrowers.

Bottom line

All in all, we would guess that microfinance as a whole has done a great deal of good, but has also probably done some harm. We are more pessimistic specifically about microfinance donations in the current environment. For the reasons outlined above, we believe that giving to an “average” or “typical” microfinance charity – or giving with an illusory “peer to peer” relationship as the extent of your due diligence – is a fairly bad bet. At the very least, it will deliver far less good, and far more potential harm, than the typical microfinance narrative suggests.

Yet we still find the basic idea of providing financial services to people with low and volatile incomes very appealing as a way to help people … if it is done in a way that stresses social impact and uses donations responsibly.

We believe that microsavings is a particularly promising area, although we haven’t found a microsavings charity we can be confident in.

We believe that the Small Enterprise Foundation is a microlending institution that is truly and appropriately focused on achieving positive social impact. We’ll be writing more about it.

We recently looked at Kiva’s largest partner MFI, LAPO (Lift Above Poverty Organization), as part of our evaluation process for an economic empowerment grant in sub-Saharan Africa.

In brief, we found two surprising pieces of information:

• LAPO is very profitable.
• There’s good reason to be concerned about LAPO’s social impact.

As Natalie recently described on our research list, we’ve contacted a handful of individual microfinance institutions in Sub-Saharan Africa to assess whether one might be able to answer the key questions we ask to evaluate a microfinance organization.

One of the steps we took was to look at Kiva’s largest MFI partners. Because Kiva partners are both (a) relatively well-known (due to its presence on Kiva) and (b) underwent Kiva’s due diligence process, we guessed that they might be a reasonable place to begin our search.

When we looked closely at LAPO, we found the following, all of which concerned us (Note: we haven’t yet contacted LAPO as our aim, at this point, was to identify the most promising organizations, not confidently dismiss any particular organization. Because our brief review of LAPO opened several relatively large questions, we chose to move on, as we often do).

• In the last 3 years (2006-2008), LAPO had significant profit margins (23-28%).
• In its Mix Market Social Performance Report (xls), LAPO reported a 49% dropout rate. As Holden wrote in our post on evaluating a microfinance charity, dropping out of a program may indicate participants “voting with their feet” and choosing to leave a program that they don’t find beneficial. It is also possible that “drop outs” instead consist of those who “graduate” from the program, i.e., improve their incomes/credit to the point where they can access credit from elsewhere (or no longer want/need credit). However, my instinct is that it’s unlikely that close to 50% of participants are quickly moving up to access more formal sources of credit.
• LAPO’s Client Exit Study report (doc) reports that individuals need manager approvals to withdraw savings, and that managers investigate the reason for withdrawal before approving (Pg 3). This seems to undermine many of the benefits of saving, which presumably aims to help people deal with risk and unexpected situations.

Does LAPO sound like an institution that needs (or should receive) Kiva’s interest-free funding?

Its appears highly profitable, but its social impact is much less clear given the high drop-out rate, significant hurdles for depositors to withdraw savings. These facts paint a slightly worrying picture of LAPO as an organization that may be earning significant profits through relatively restrictive regulations for clients while getting interest-free funding through Kiva. Perhaps there is a special arrangement here as with Xac Bank, but it certainly raises a concern about charity-minded capital funding profits.

David Roodman raises the concern that Kiva capital could be effectively “padding profits” at a profitable microfinance institution. He concludes,

If social investors provide capital at prices below commercial rates to enterprises with “double bottom lines” (profit and social benefit), how do the investors assure that their cheap capital isn’t being used to boost just one of those bottom lines?

We feel that this is a major concern, and one that also applies to larger-scale social enterprise investment (see last week’s discussion of Acumen Fund, particularly the part about VisionSpring).

With a for-profit, everyone is looking to get their investment back. With a nonprofit, everyone is (ideally) looking to achieve social impact. But an organization that has both charitable and for-profit investors can get caught in the middle: taking donations, but measuring its success in terms of profits instead of lives changed.

The situation is particularly dangerous when profits are viewed as a “proxy” for social impact, and thus become the only measure looked at. Imagine an enterprise that could sell food below market prices, thanks to support from donations. It could be distorting the local economy (by outcompeting local farmers), failing to reach those truly in need, and ultimately failing to accomplish anything besides selling food for less than it costs to people who turn it around for a profit. And yet, all of this could be consistent with a good bottom line, which would make both the investors and the donors happy.

One way to avoid this problem is to refuse to use profits as a proxy for social impact - to insist on rigorous assessment of whether an enterprise is changing lives, rather than settling for the logic of “if it’s selling it must be helping.” But such rigorous assessment costs money, and exacerbates the already great challenge of turning a profit. Jim Fruchterman’s recent comment illustrates the low likelihood that you’ll see much of this approach working in practice.

Another possible approach is to get very specific about how donations are and aren’t being used. Any sort of “hybrid” organization ought to be able to show a history of using donations to absorb risk, but ultimately creating ventures whose profitability and sustainability does not depend in any way on continued subsidies. Our basic feeling is that demonstrating such a thing would be harder than it sounds, and that we have not yet seen an organization that seems capable of doing so.

Bottom line: it’s difficult to hold an organization accountable unless all its investors are on the same page about what it’s accountable for. Blended value makes perfect sense in theory, but in practice, it seems like a huge challenge that nobody is clearly up to. In the meantime, it may make more sense for businesses to be businesses and charities to be charities.

Nathaniel Whittemore writes that it’s “time to move on” regarding the recent Kiva controversy. I disagree.

It’s true that Kiva handled the criticism admirably, and made significant changes to its website to improve clarity for donors. It’s also true that Kiva has a stronger case than many for being generally transparent and impactful. Finally, it’s true that those of us who have been blogging about Kiva are a bit tired of the subject.

But none of this changes the fact that many (I would guess the vast majority) of Kiva’s enthusiastic users don’t know how it works, and would be upset if they found out. If you doubt this, just look at the reaction to the recent New York Times story that brought this issue to the attention of people outside our corner of the blogosphere. We may be “over” this issue, but most people have still never heard of it.

The common perception of Kiva - repeated to us by supporter after supporter - is that it enables people to (a) make interest-free loans to (b) the entrepreneurs they personally select. In fact, Kiva users are effectively making gifts (since the loans are interest-free for them, but not for the borrowers) to microfinance institutions. If they knew this, they might prefer giving directly to microfinance charities instead, especially since they’d then be able to get a tax deduction and give significantly more. Or they might prefer another gift entirely - perhaps a health program that offers great impact but no “personal connection,” or perhaps a DonorsChoose project that offers “real” personal connection but arguably little impact.

The time to “move on” should not be based on Kiva’s “handling” the situation or our growing tired of it - it should be based on Kiva’s supporters, by and large, understanding how Kiva works. We think we’re nowhere near that point. We urge those who know the truth about Kiva to continue spreading the word.