Normally, we focus on identifying outstanding charities, and minimize the time spent on opaque or otherwise lackluster ones. But lately, we’ve gone into a bit more detail about our take on several of the best-known and most appealing charities out there.

What all of the charities below have in common is that (a) we have major questions and concerns about their activities; (b) the information necessary to see how serious these concerns are does not seem to be available. (In most cases our assessment is based on significant back-and-forth with the charities themselves, though in some cases we are going off their website.)

• Kiva: commonly perceived as a way to (a) make interest-free loans to (b) entrepreneurs you personally select. In fact, it is neither. (See GiveWell Board member Tim Ogden’s summary of the recent discussion/controversy.)

  We ultimately see a Kiva “loan” as a gift to a microfinance institution, and we have major concerns about supporting any but the most outstanding microfinance institutions as well as concerns about Kiva’s due diligence process specifically.

  Full set of blog posts on Kiva

• Grameen Foundation and other large U.S. microfinance charities: in addition to our general concerns about microfinance charities (and the fact that these charities have not provided the information that would address them), we find that these organizations are rarely even clear about what their value-added is. Also see the disappointing (and eye-opening) evaluation of the Village Phone program.

  Full set of blog posts on Grameen Foundation

• Heifer International: commonly perceived as a way to “give a cow to a poor family as a gift,” but this is in fact a donor illusion -donations support Heifer International’s general “agricultural assistance” activities. We have concerns about this general area and concerns about giving livestock specifically. Neither Heifer’s website nor its grant application have provided the kind of information needed to address these concerns.

  Full set of blog posts on Heifer International

• Smile Train: Prof. Steven Levitt of Freakonomics sees it as using an innovative model to deliver cleft surgeries for $250 apiece. Our analysis suggests that it has “more money than doctors” and that it is therefore out of room for more funding. Smile Train’s recent removal of the charts we questioned and its own explanation of why it is giving grants to other major charities appear to support our analysis. In addition, we have major concerns about surgery charities that Smile Train has not provided enough information to address.

  Full set of blog posts on Smile Train

• UNICEF offers donors the chance to “buy” vaccines and other items. We suspect that this offer is a donor illusion, but the larger issue is that UNICEF provides almost no information at all about its huge variety of activities and how they are going. Most giant international aid charities (Save the Children, World Vision, etc.) are similarly complex and opaque, with CARE coming the closest to meaningful transparency.

  Full set of blog posts on UNICEF

• The Acumen Fund stresses its metrics-driven approach to “patient capital,” investing in businesses that aim for long-term sustainability (i.e., profitability) and social impact. As of now, we see no substantive available information about its track record.

  Full discussion

• The Robin Hood Foundation also stresses its emphasis on metrics, quantification, and analytical thinking. But we have found that it is unwilling to share any information about the results of its grantmaking. (Note that it also pushes the “0% overhead” donor illusion.)

  Full set of blog posts on The Robin Hood Foundation

• The Millennium Villages project states that it is “demonstrating that it’s possible to escape … extreme poverty” (front page of its website). As far as we can tell, very few results have been demonstrated to date, with a few boosts in crop yields as its main evidence of impact - evidence we feel is highly questionable.

  Full discussion

• The Worldwide Fistula Fund has been praised by Nicholas Kristof as “giving new life to the pariahs.” Our assessment is that the Fund is pushing a debatable agenda whose effectiveness has yet to be shown.

  Full discussion

• The Carter Center is, in our view, by far the “best bet” (for an individual donor, our focus) of the charities listed in this post. It has achieved substantial, demonstrable impact in the past through its work on guinea worm eradication, and unlike all of the charities listed above, it publicly discloses substantive technical information on the impact of its past activities.

  However, there is a crucial piece of information missing: how it allocates its budget between programs (in particular, how much it spends on its stronger and better-documented vs. more questionable programs). It has declined to share this (basic, in our view) information in repeated exchanges.

  Full set of blog posts on The Carter Center

We think the above charities are fairly representative of “average” charities in international aid. Some tell better stories than others and some have more disclosure than others. But in almost all cases, international aid charities are (a) carrying out complex projects that can fail to do good (or even do harm) in a variety of ways, and (b) not systematically sharing the information that would make it possible to assess how their work is going.

GiveWell is devoted to finding charities in which we can have more confidence. We’ll be discussing our two top-rated charities working internationally in forthcoming posts.

Seth Godin makes an appealing case for “social enterprise investment” along the lines of The Acumen Fund:

When two people trade, both win. No one buys a bar a soap unless the money they’re spending for the soap is worth less to them than the soap itself.

When someone in poverty buys a device that improves productivity, the device pays for itself (if it didn’t, they wouldn’t buy it.) So a drip irrigation system, for example, may pay off by creating two or three harvests a year instead of one.

…

How does Acumen Fund create these markets? The answer is patient capital. The companies that are selling solar lamps to replace kerosene or water purification systems in tiny villages, or housing projects for peasants in Pakistan or even ambulance services in Mumbai fully intend to make a profit, but the venture capitalists on Sand Hill Road aren’t in a hurry to invest in them. The investments are a little too risky, take a little too long or a little too unproven to take a chance on.

So Acumen finds these entrepreneurs on site in the developing world, funds them, teaches them and pushes them to build really big organizations. A to Z has literally thousands of people in their modern factory creating malaria bed nets in Tanzania. And so it grows.

There are reasonable arguments that this sort of larger-scale financing is far more promising than microfinance, which typically focuses on tiny microenterprises that are unlikely to achieve large scale and create jobs. If we could find a strong charity in this area, we’d be excited to recommend it. But as with other areas, we try to go beyond the story and ask critical questions, and so far we have found no organization - including the Acumen Fund - that appears to answer them.

Our main question in this area pertains to an investor’s track record. We’d expect large-scale investing to have a success rate well under 50%, but we’d want to see at least an example or two of real success in the past, consisting of any of the following:

• An investment that has been shown to improve the lives of customers (for example, tracking bednet use in areas served by A to Z).
• An investment that has been shown to create jobs, i.e., employ people with previously low incomes.
• A financial return. (Note that it isn’t enough to point to a “portfolio” member and observe that it’s a successful, self-sustaining business. Anyone can make a loan or grant to an already-successful business; this doesn’t mean they’re having an impact.)

Given Acumen Fund’s reputation for (a) general excellence and (b) focus on evaluation (see the Acumen Fund discussing these issues here and here), we would have expected them to produce such examples. But after evaluating them for our economic empowerment grant, we have concluded that:

• They do not directly track social impact of the organizations they invest in.
• They do not have information on the effects of their investments on jobs.
• They have declined to share information on the performance of their loans, and have not yet exited any equity investments.

A couple of other concerns:

• At least one of their investments, VisionSpring, has a major charitable solicitation component (for example, they have applied for funding from us). Acumen’s loan could end up getting repaid by donations, which wouldn’t go well with the idea of “patient capital to create self-sustaining institutions.”
• Their documents are largely at too general and vague a level to give a concrete sense of how their investments are going.

Given this situation, it’s hard to see the organization as more than an experiment at this point. No matter what its team’s credentials, there is the enormous question of how developed-world business experience translates to the goal of making good and socially impactful investments in the developing world.

As with other unproven charities, it seems to us that Acumen should be primarily funded by those who are very close to the relevant people and issues, until it is able to demonstrate not just a good story but some real results. Other similar organizations seem to be similar cases.