Normally, we focus on identifying outstanding charities, and minimize the time spent on opaque or otherwise lackluster ones. But lately, we’ve gone into a bit more detail about our take on several of the best-known and most appealing charities out there.

What all of the charities below have in common is that (a) we have major questions and concerns about their activities; (b) the information necessary to see how serious these concerns are does not seem to be available. (In most cases our assessment is based on significant back-and-forth with the charities themselves, though in some cases we are going off their website.)

• Kiva: commonly perceived as a way to (a) make interest-free loans to (b) entrepreneurs you personally select. In fact, it is neither. (See GiveWell Board member Tim Ogden’s summary of the recent discussion/controversy.)

  We ultimately see a Kiva “loan” as a gift to a microfinance institution, and we have major concerns about supporting any but the most outstanding microfinance institutions as well as concerns about Kiva’s due diligence process specifically.

  Full set of blog posts on Kiva

• Grameen Foundation and other large U.S. microfinance charities: in addition to our general concerns about microfinance charities (and the fact that these charities have not provided the information that would address them), we find that these organizations are rarely even clear about what their value-added is. Also see the disappointing (and eye-opening) evaluation of the Village Phone program.

  Full set of blog posts on Grameen Foundation

• Heifer International: commonly perceived as a way to “give a cow to a poor family as a gift,” but this is in fact a donor illusion -donations support Heifer International’s general “agricultural assistance” activities. We have concerns about this general area and concerns about giving livestock specifically. Neither Heifer’s website nor its grant application have provided the kind of information needed to address these concerns.

  Full set of blog posts on Heifer International

• Smile Train: Prof. Steven Levitt of Freakonomics sees it as using an innovative model to deliver cleft surgeries for $250 apiece. Our analysis suggests that it has “more money than doctors” and that it is therefore out of room for more funding. Smile Train’s recent removal of the charts we questioned and its own explanation of why it is giving grants to other major charities appear to support our analysis. In addition, we have major concerns about surgery charities that Smile Train has not provided enough information to address.

  Full set of blog posts on Smile Train

• UNICEF offers donors the chance to “buy” vaccines and other items. We suspect that this offer is a donor illusion, but the larger issue is that UNICEF provides almost no information at all about its huge variety of activities and how they are going. Most giant international aid charities (Save the Children, World Vision, etc.) are similarly complex and opaque, with CARE coming the closest to meaningful transparency.

  Full set of blog posts on UNICEF

• The Acumen Fund stresses its metrics-driven approach to “patient capital,” investing in businesses that aim for long-term sustainability (i.e., profitability) and social impact. As of now, we see no substantive available information about its track record.

  Full discussion

• The Robin Hood Foundation also stresses its emphasis on metrics, quantification, and analytical thinking. But we have found that it is unwilling to share any information about the results of its grantmaking. (Note that it also pushes the “0% overhead” donor illusion.)

  Full set of blog posts on The Robin Hood Foundation

• The Millennium Villages project states that it is “demonstrating that it’s possible to escape … extreme poverty” (front page of its website). As far as we can tell, very few results have been demonstrated to date, with a few boosts in crop yields as its main evidence of impact - evidence we feel is highly questionable.

  Full discussion

• The Worldwide Fistula Fund has been praised by Nicholas Kristof as “giving new life to the pariahs.” Our assessment is that the Fund is pushing a debatable agenda whose effectiveness has yet to be shown.

  Full discussion

• The Carter Center is, in our view, by far the “best bet” (for an individual donor, our focus) of the charities listed in this post. It has achieved substantial, demonstrable impact in the past through its work on guinea worm eradication, and unlike all of the charities listed above, it publicly discloses substantive technical information on the impact of its past activities.

  However, there is a crucial piece of information missing: how it allocates its budget between programs (in particular, how much it spends on its stronger and better-documented vs. more questionable programs). It has declined to share this (basic, in our view) information in repeated exchanges.

  Full set of blog posts on The Carter Center

We think the above charities are fairly representative of “average” charities in international aid. Some tell better stories than others and some have more disclosure than others. But in almost all cases, international aid charities are (a) carrying out complex projects that can fail to do good (or even do harm) in a variety of ways, and (b) not systematically sharing the information that would make it possible to assess how their work is going.

GiveWell is devoted to finding charities in which we can have more confidence. We’ll be discussing our two top-rated charities working internationally in forthcoming posts.

We recently complained that “UNICEF provides no information about where the money goes and what projects are in progress.” Some might feel that this complaint comes from unrealistically high standards of transparency, especially for organizations such as UNICEF. How is an organization spending $2.7 billion a year supposed to report its activities?

Our answer would be: “like the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) does.” (Page 55 of its 2008 annual report shows that its budget size is very close to UNICEF’s at $2.7 billion.)

GFATM provides an online program search of all its activities. For any grant it has given (example), you can see (if completed) the grant proposal, grant agreement, and reports on progress. In other words, you can see how much has spent and how (and whether) progress has been tracked.

GFATM recently released the kind of document we have never seen from any other charity approaching its size: an overall evaluation of its activities and impact. Not a general discussion of the organization; not a “meta-evaluation” discussing the quality of past evaluations; a discussion of the overall impact of all of GFATM’s activities across the world. Furthermore, this report was in no way a fundraising document; it was frank about the fact that inadequate evidence exists for GFATM’s impact to date (see the discussion at our review).

GFATM comes under a lot of criticism, even from its own evaluators. We ourselves have many reservations about its work, as our review establishes. But we have seen very few charities - and no other charities approaching its size - that can make as strong a claim to being a transparent organization and a learning organization.

GFATM proves that neither size nor celebrity support need stop a charity from being clear about what it’s working on and how it’s going.

UNICEF offers you the chance to buy measles vaccines for 100 children for $27.10. And lest you complain that you’ve heard this one before, it assures you specifically that “While other organizations allow supporters to purchase ’symbolic’ gifts, Inspired Gifts are actual items.”

Is this finally the “real personal connection” donors have been waiting for?

We can’t say for sure. Unlike Kiva, UNICEF provides no information about where the money goes and what projects are in progress. But we can ask a few critical questions:

• There are many costs for vaccination programs besides the vaccines themselves. Who is paying these costs?
• If some other party puts up the “fixed costs” for a given campaign (labor, logistics, etc.), but UNICEF’s catalogue only “sells” 1/2 the needed vaccines, will the other 1/2 of the relevant population go unvaccinated?
• Or will other sources of funds cover the remaining need, making the cost of the campaign essentially fixed? If this is the case, in what sense is the donor “buying” the vaccine?

Our guess is that UNICEF has a large pool of funding allocated to these campaigns, aside from the money that comes in through “Inspired Gifts” (which seems to be paying for very small numbers of items). We would guess that UNICEF will “officially” match up donations through the “catalogue” vehicle to, for example, vaccines while shifting its other funds from vaccines to delivery costs.

Why does all of this matter? Because UNICEF is advertising immunizations for 27 cents apiece. In reality, it almost certainly costs more than that - all things considered - to deliver a vaccination. That would make this another case in which a charity misleadingly zooms in on “your” money rather than considering all costs - a subtle, but substantive, donor illusion.

There’s no smoking gun, though, because there is no transparency.

Why has Kiva been singled out for so much criticism lately (see GiveWell Board member Tim Ogden’s summary)?

Part of the answer is that Kiva has arguably been misleading donors - but that can’t be the full answer. David Roodman’s original post could never have come about without the fact, as Mr. Roodman puts it, that “the way Kiva actually works is hidden in plain sight.” And our followup analysis on repayment rates was only possible because Kiva makes all its repayment data publicly and easily available. As Elie said in that post, “Similar analysis would be impossible for nearly every other charity I can think of.”

Contrast Kiva with, for example, UNICEF. Kiva makes it possible to trace the path of your donation, to the extent that such tracing is realistic (and it largely turns out to be more along the lines of “you funded a certain MFI” rather than “you funded a certain person”). UNICEF doesn’t even seem to have a breakdown of how much money is going to each continent. We definitely can’t find information on questions like (a) What specific projects are you funding? (b) What is your role in each? (c) What new projects are planned, and where? (d) How is each project going, whom is it affecting, and how?

There are no strange patterns in UNICEF’s numbers because there are no numbers. There are no contradictions because there is no concrete information. And the intent here isn’t to single out UNICEF - it’s merely one of the vast majority of international aid organizations about which we know essentially nothing.

Giving an impression to donors that’s undermined by the facts is a minor scandal. When will complete opacity - simply sharing no information at all - be a major one?