The disaster in Haiti - and the media coverage of it - pull at the emotions in a way that everyday suffering in the developing world does not. However, our rough calculations suggest that in fact, a donor can have a bigger impact for less money by funding top charities’ everyday activities to reduce unnecessary death and debilitation.

We estimate a “generous” cost-effectiveness figure for a donation to Haiti is by considering (a) the total amount given and (b) the total number of people affected by the disaster.

Total amount given: It’s hard to find definitive figures for the amount of money already donated to the relief efforts, but ReliefWeb provides what appears to be a reasonable lower-bound (i.e., conservative) estimate. As of January 27, ReliefWeb reports that $1.2 billion had been given or pledged to Haiti relief efforts. (See document A on ReliefWeb’s Haiti Earthquake: Appeals and Funding page.) Note that its numbers clearly do not include all charities or all revenue sources: for example, it lists only $2.8 million raised by Partners in Health, whereas the Chronicle of Philanthropy’s roundup - up as of two days ago but seemingly down at this writing - lists $40 million (and also includes many charities that don’t appear at all in Reliefweb’s stats.)

Number of people affected: Haitian authorities estimated that 1 million people were left homeless by the quake (according to ABC World News) and perhaps 3 million were “affected” (according to the Guardian). For the sake of our calculuation, we’ll assume a range of 1-3 million people affected.

Expenses per person affected: using the Reliefweb number (assuming that nothing is excluded from it and that no more money is forthcoming, both assumptions that clearly understate the funds) yields a current level of $395-$1,185 pledged or donated per person affected. (For some interesting context, we’ve seen estimates that the 2004 Indian Ocean Tsunami affected 2,000,000 people and donors gave approximately $14 billion — funding per person of $7,000.)

We estimate that our top-rated charities save a life for approximately that amount. While it’s clearly not possible to directly compare the impact of relief efforts to the impact of saving an individual’s life, our feeling is that saving an individual’s life is likely to have a significantly larger effect on that person than the future change likely to occur in the average life of a Haitian due to the relief efforts.

Reader Evan writes:

I’ve been thinking about how best to donate to Haiti, and I reviewed some of the materials on your website and found them pretty helpful and persuasive. So thank you! But then my law firm announced that it would match donations to the Red Cross or Doctors Without Borders. Given that, I think I have to donate to one of those orgs: even if my money would probably be better spent elsewhere, it’s hard to imagine that it would be more than twice as well spent. Do you disagree?

My intuition here is different than Evan’s. My guess would be that giving to one of our top-rated charities could easily accomplish more than twice as much good as supporting the efforts of the Red Cross or Doctors Without Borders in Haiti.

This guess is largely based on two factors:

1. The large divergence in relative cost-effectiveness of different programs (which can approach a factor of 1,000, not just a factor of 2) combined with the reasonable position that disaster relief is not among the most cost-effective avenues for charitable funds.
2. A back-of-the-envelope calculation for cost-effectiveness of efforts in Haiti which puts it well below the cost-effectiveness of our top charities.

In this post, I’ll look at the first factor. I’ll post more on the second issue in a future post.

Cost-effectiveness for different approaches to helping people varies widely

The most cost-effective programs are so much more impactful per dollar than other programs that a much smaller donation to a top program will likely help significantly more people. We’re careful about our use of cost-effectiveness figures, and the Disease Control Priorities Report’s (DCP) in particular (which we think constitute “best-case” scenarios rather than what a donor can expect from his donation), but we do think they give a reasonable basic sense of the differences between different kinds of programs.

Figures 2.2 and 2.3 on pages 41-2 of the DCP report provide cost-effectiveness estimates for many common programs charities run. (These are all presented using the $/DALY metric. For more information on what this is, see our overview for interpreting the DALY metric.) Some of the most cost-effective programs are deworming programs ($3/DALY), expanding immunization coverage ($7/DALY), and bednets to prevent malaria ($11/DALY).

Some of the least cost-effective (but common among charities) programs are improved water and sanitation to prevent diarrhea ($4,185/DALY), some types of maternal and neonatal care packages ($1,060/DALY), and Antiretroviral therapy to treat HIV/AIDS ($922/DALY).

These examples are not meant to demonstrate that the less cost-effective programs are necessarily less worthy, but they do illustrate that the impact per dollar a donor can expect from his gift can easily vary by 2-3 orders of magnitude, even under assumptions that programs are essentially being carried out as intended. Of course, if some programs are poorly executed or simply ineffective, the difference can be much larger still.

With that context, when choosing which charity to support, I wouldn’t trade much confidence-in-an-organization to merely double the size of my donation.

As we’ve discussed before, with limited information we’d tentatively guess that disaster relief funds are closer to the less-cost-effective end of the range rather than the most-cost-effective end. With that in mind, I’d guess that a gift to VillageReach or Stop TB could easily accomplish more than twice as muc good as a gift supporting the Red Cross or Doctors without Borders in Haiti.

The above is very general and though relevant, not at all specific to the situation in Haiti. In a future post, I’ll post more on some specifics regarding Haiti and why I think it offers further support to the notion that donors can accomplish more good by giving to our top charities, even if they give less.

Two other small notes

There are a couple other factors that contribute (though in a relatively small way) to my conclusion here:

• It doesn’t seem appropriate to consider causing one’s company to give a donation to be equivalent to doubling one’s cost-effectiveness. The firm may have taken matching funds from a pool already allocated to charitable giving, or the partners may have given the funds to charities themselves. Even if the funds wouldn’t otherwise go to charity, the firm likely has another motive for giving, which should lead you to consider how this program differs from other embedded giving programs, which we think are of dubious additional value.
• Giving to a charity because it has demonstrated effectiveness has the additional benefit of signaling to other charities that effectiveness matters to donors. A core belief of ours at GiveWell is that rewarding charities for effectiveness in changing lives will incentivize other charities to improve their programs to compete for those donor funds. Proactive giving (i.e., trying to choose the best charity available) furthers this dynamic; passive giving (choosing from a predefined list) hampers it. It’s also quite possible that in a very direct sense telling your company that you’ve chosen to give in this way could influence them to adjust their own giving towards more considered and effective charities.

We are concerned about the way repayment rates are often reported. We’ve written about this issue before, arguing that different delinquency indicators can easily be misleading and pointing to one example we found where a microfinance institution’s reported repayment rate substantially obscures the portion of its borrowers that have repaid loans.

Following the links from David Roodman’s recent post about Richard Rosenberg, we found another paper Mr. Rosenberg authored making all the same points, much better than we did. The paper is Richard Rosenberg’s. “Measuring microcredit delinquency: ratios can be harmful to your health.” CGAP Occasional Paper #3. 1999. Available online here (pdf).

Relevant quotes from Mr. Rosenberg’s paper

The importance of using the “right” delinquency measure:

MFIs use dozens of ratios to measure delinquency. Depending on which of them is being used, a “98 percent recovery rate” could describe a safe portfolio or one on the brink of meltdown. (Pg 1)

The measure we’ve been asking for seems to be equivalent to what he calls the “collection rate.”

Most of the discussion will be devoted to three broad types of delinquency indicators: (a) Collection rates measure amounts actually paid against amounts that have fallen due. (b) Arrears rates measure overdue amounts against total loan amounts. (c) Portfolio at risk rates measure the outstanding balance of loans that are not being paid on time against the outstanding balance of total loans. (Pg 2)

It’s essential to not only know which measure is being used, but precisely how an MFI calculates its version of the measure:

But the reader must be warned that there is no internationally consistent terminology for portfolio quality measures—for instance, what this paper calls a “collection rate” may be called a “recovery rate,” a “repayment rate,” or “loan recuperation” in other settings. No matter what name is used, the important point is that we can’t interpret what a measure is telling us unless we understand precisely the numerator and the denominator of the fraction. (Pg 2)

Mr. Rosenberg describes different tests to which MFIs should subject various delinquency measures to determine which is most appropriate. For GiveWell’s purposes, one of the key tests is the “smoke and mirrors” test:

Can the delinquency measure be made to look better through inappropriate rescheduling or refinancing of loans, or manipulation of accounting policies? This is our smoke and mirrors test. (Pg 3)

The practice of rescheduling and renegotiating loans:

When a borrower runs into repayment problems, an MFI will often renegotiate the loan, either rescheduling it (that is, stretching out its original payment terms) or refinancing it (that is, replacing it—even though the client hasn’t really repaid it—with a new loan to the same client). These practices complicate the process of using a collection rate to estimate an annual loan loss rate. Before exploring those complications and suggesting alternative solutions for dealing with them, the author needs to issue a warning: any reader looking for a perfect solution will be disappointed. The suggested approaches all have drawbacks. It is important to recognize that heavy use of rescheduling or refinancing can cloud the MFI’s ability to judge its loan loss rate. This is one of many reasons why renegotiation of problem loans should be kept to a minimum—some MFIs simply prohibit the practice. (Pg 10)

The strengths of PAR (”portfolio at risk”) as a measure:

The international standard for measuring bank loan delinquency is portfolio at risk (PAR). This measure compares apples with apples. Both the numerator and the denominator of the ratio are outstanding balances. The numerator is the unpaid balance of loans with late payments, while the denominator is the unpaid balance on all loans The PAR uses the same kind of denominator as an arrears rate, but its numerator captures all the amounts that are placed at increased risk by the delinquency. (Pg 13)

And its weaknesses:

Like many other delinquency measures, the PAR can be distorted by improper handling of renegotiated loans. MFIs sometimes reschedule—that is, amend the terms of—a problem loan, capitalizing unpaid interest and set- ting a new, longer repayment schedule. Or they may refinance a problem loan, issuing the client a new loan whose proceeds are used to pay off the old one. In both cases the delinquency is eliminated as a legal matter, but the resulting loan is clearly at higher risk than a normal loan. Thus a PAR report must age renegotiated loans separately, and provision such loans more aggressively. If this is not done, the PAR is subject to smoke and mirrors distortion: management can be tempted to give its portfolio an artificial facelift by inappropriate renegotiation. (Pg 16)

PAR can also be misleading in a situation where an MFI is growing rapidly (a key argument of our past posts):

Another potential distortion in PAR measures is worth mentioning. Arguably the PAR denominator should include only loans on which at least one payment has fallen due, so that late loans in the numerator are compared only to loans that have had a chance to be late. Nevertheless, it is customary to use the total outstanding loan balance for the denominator. The distortion involved is usually not large for MFIs, because the period before the first payment is a small fraction of the life of their loans. For instance, for a stable portfolio of loans paid in 16 weekly installments with no grace period, a PAR of 5.0 percent measured with the customary denominator (total outstanding portfolio) would rise only to 5.3 percent using the more precise denominator (excluding loans on which no payment has yet come due.) However, if a portfolio is growing very fast, or if there is a grace period or other long interval before the first payment is due, then the customary PAR denominator can seriously understate risk. Pg 17

Table 6 on Pg 19 summarizes the strengths of weaknesses of different measures:

Why is this important?

Given how complicated this all is, we think that MFIs need to be clear and transparent about (a) which measures they use and (b) precisely how they calculate them.

However, this isn’t the case. For example, we aren’t confident that most MFIs normally report rescheduled and renegotiated loans as at-risk in PAR measures.

On the one hand, Commenter Ben writes, “Best practice is to treat all loans that have been rescheduled as PAR.” (This is consistent with MixMarket’s glossary, which indicates that, “[A PAR measure] also includes loans that have been restructured or rescheduled.”

Nevertheless, “best practice” may not correlate with “in practice.”

• This Kiva document (its “Partnership Application”) is explicit in the definition of PAR 30: “The value of loans outstanding that have one or more repayments past due more than 30 days. This includes the entire unpaid balance of the loan, including both past due and future installments, but not accrued interest or renegotiated loans.” (emphasis mine) Note that, to Kiva’s credit, it explicitly asks for renegotiated loans separately in the application.
• As Holden recently commented, “At least one MFI has indicated to us that it does not report [renegotiated loans in its PAR measures].”

The definition you read today isn’t necessarily the one that MFIs are using.

What measure do we use and why?

We’ve written before that our preferred measure is what the paper discussed above calls the collection rate. While the collection rate measure fails to provide a warning to MFIs that their portfolio is in danger, it is the strongest on Mr. Rosenberg’s “Bottom-line” test because it simply and clearly measures failed repayments. It’s therefore less susceptible to obfuscation and manipulation.

For GiveWell’s purposes, we need a delinquency measure that most clearly reports borrowers’ situations. While PAR measures provide information, it’s clear that PAR measures are more valuable to evaluating the risk of an MFI’s portfolio, which while relevant is not our key concern.

Update: see our official page on Haiti earthquake relief, which consolidates advice from us and a few other sources we have high opinions of.

Reader Brigid writes:

“I would love to hear any thoughts you have on contributions in light of the crisis in Haiti. My sense is that now of all times is when people give significantly without due diligence into a charity’s impact and that donor have more illusions than generally (i.e., My gift is going directly to a hurt Haitian). It seems inevitable the news piece several months after the event when donors are surprised/angry to learn their gifts were not used as they believed they were.

Is there any way for an average donor to help the crisis in Haiti right now? Is there a way to “capture” the generosity that these events inspire while still focusing on impact? Specifically to GiveWell: would your team consider focusing quick efforts on analyzing charities that are addressing the crisis in Haiti (i.e. would you shift your mission at this moment)? Or, would you say: despite the current crisis in Haiti, any contribution an individual donor wants to make will still impact more people if, for example, given to fighting tuberculosis through Stop TB Partnership.”

A few notes:

• Unfortunately, “disaster relief” is not an area we’ve researched. When we’ve (very briefly) looked into the cause of “disaster relief” in the past, we haven’t found substantive, well-supported, specific recommendations for where to give. More at two posts from May 2008, following the cyclone in Burma:
  • Cyclone Relief: Recommendation and Questions
  • Emergency Assistance for Donors
• According to the Disease Control Priorities Project, emergency aid is one of the worst uses of donations, despite being one of the most emotionally compelling. For more, see our post on the cost-effectiveness of disaster-focused aid.
• Our intuition then, as it is now, is that the most effective way to help those in need is to give to effective organizations. A few of our recommended organizations have a presence in Haiti:
  • The Stop TB Partnership provides tuberculosis drugs in Haiti.
  • Population Services International has worked in Haiti since 1989.
  • Partners in Health’s flagship clinic is located in Haiti.

Before we start giving our answers to the questions of this post, I wanted to share some raw data that we look at to gauge how things are going.

The charts/tables below cover the following:

• “Money moved,” i.e., donations made to GiveWell-recommended charities due to GiveWell’s research
• Website traffic.

This is just a subset of the information we have. We’ll be releasing a more complete set of charts/tables/data shortly.

The table below shows the support each of our recommended charities received in 2009. Update, 1/8/2010: VillageReach sent us an updated file that includes donors through the end of 2009. The updated table is below. Note: VillageReach’s total fell as we discovered that we had erroneously double-counted some funds.

You can view the original table we posted here.

• “Pledgers” refers to people who made GiveWell Pledges (advance commitments to give based on our research) in 2008, before our recent report was completed, and followed through on these commitments in 2009.

• “Large gifts” refers to donors who made large gifts, and directly told us (and the charities they were giving to) that GiveWell’s research had been the key factor in where they gave.
• “Economic empowerment grant” refers to a grant made directly by GiveWell, with funds from a single donor.
• “Through website” refers to gifts made through the “Donate Now” buttons on GiveWell.net (some through Google and some through Network for Good (NFG)).
• “Grants” refers to grants made directly by GiveWell, mostly with funds that were restricted by donors for regranting (we also granted just under $15,000 in unrestricted funds).
• “Direct to charity” refers to donations that VillageReach received, not through GiveWell’s website, and believes it can confidently attribute to GiveWell (this is due to the fact that VillageReach is a relatively small organization that does not get many donations from unfamiliar individuals). We are still awaiting data from 12/22/09-12/31/09, so we expect the final version of this number be higher.

The following two charts show the amount donated and number of donors through the GiveWell site, comparing 2007-2009 (and the beginning of 2010).

Finally, we show monthly web traffic to the GiveWell site and blog. Two notes: (1) we unfortunately lost tracking for much of 2008 — that explains the lack of data during that period. (2) We had an immense spike on 12/20/2007 due to media coverage; we’ve purposefully set the left-axis as it is to make it easier to view the rest of the chart.

Our current top priority is assessing the state of GiveWell: what we’ve accomplished, where we stand, and where we should focus our limited resources next. Over the coming weeks, we’ll be trying to examine ourselves as dispassionately and critically as possible, and sharing our self-review in something close to real time via this blog.

GiveWell’s mission is to find outstanding charities and publish the full details of our analysis to help donors decide where to give. The ultimate goal is to have significant impact on the flow of donations, moving toward a world in which donors reward charities for success in improving lives. The major questions about GiveWell, as I see them, are as follows.

Questions for “customers,” i.e., people considering using GiveWell’s research to decide which charities to support

• Does GiveWell provide quality research that highlights truly outstanding charities in the areas it has covered?
• Is it practical for donors to evaluate and use GiveWell’s research in the areas it has covered?
• Has GiveWell covered enough areas to be useful?

Additional questions for stakeholders, i.e., people considering giving their time, money and other support directly to GiveWell (these include the GiveWell Board and staff)

• Is GiveWell’s research process “robust,” i.e., can it be continued & maintained without relying on the co-Founders?
• Does GiveWell present its research in a way that is likely to be persuasive and impactful (i.e., is GiveWell succeeding at “packaging” its research)?
• Does GiveWell reach a lot of potential customers (i.e., is GiveWell succeeding at “marketing” its research)?
• Is GiveWell a healthy organization with an active Board, staff in appropriate roles, appropriate policies and procedures, etc.?
• What is GiveWell’s overall impact, particularly in terms of donations influenced? Does it justify the expense of running GiveWell?

For all of these questions, we intend to discuss

• The progress we’ve made since November 2008 (when we last laid out a business plan)
• Where we stand today, relative to where we need/hope to be to consider GiveWell a success
• What we can do to improve

Our self-review won’t be entirely comparable to the reviews we perform of other organizations. The latter tend to be focused on the “end product,” as we stay agnostic on progress other organizations have made and how they can improve. When evaluating ourselves, it is essential that we examine “intermediate indicators” as well as our ultimate impact, and think critically about the different paths we can take to improve.

Philanthropedia has responded to our take on its microfinance report:

• Philanthropedia’s response to GiveWell’s analysis - Part I
  
• Philanthropedia’s response to GiveWell’s analysis - Part II

We find the response to be encouragingly straightforward. For the most part, it agrees with the concerns we have raised and commits to addressing them. Good Intentions are Not Enough argues that the response raises more questions than it answers, and we agree in substance, though to us the most important part of the response is an honest recognition of shortcomings and an expressed intent to improve.

A couple of responses on points where there is some disagreement:

Re: incentives. Our original post argued that Philanthropedia, if it became highly influential under its current model, would create bad incentives for experts (allowing them to continue to keep their thoughts under wraps) and charities (encouraging them to win over experts in ways unrelated to improving their social impact).

• Philanthropedia states, “Even if only 31 experts agreed to publish their bios and stand behind the results, that is exactly 31 more than before.” We disagree that this is a good thing. The experts have not been linked with recommendations (as far as we can tell) - all we know is that they shared their votes, and we know the aggregate result. The problem here is that no one is individually accountable for their own recommendations and reasoning. If it became influential, this model would allow experts to have the benefits of transparency (i.e., influence over individual donations) without the accountability, which we feel would be a net negative impact on the incentives for experts to be transparent.
• Philanthropedia states that it intends to rigorously guard against “gaming of the system.” But the dynamics we are concerned about are less related to “rule-breaking” or outright conflict of interest (for example, a charity pays an expert for a positive rating) than to distortion of ratings in a softer sense. Becoming “popular in a certain crowd” can be accomplished in a lot of ways that have nothing to do with improving impact. That’s why it’s so important that people who recommend charities put as much as possible of their reasoning out in public, where others can ask, “Are these reasons strong enough to explain this person’s support?”

Should donors use Philanthropedia’s current report?

In its current form, we feel that Philanthropedia amounts to a set of recommendations unlinked to either people or reasoning. That’s essentially asking donors to trust anonymous people, which we feel is a bad idea and not helpful for impact-focused donors. (And if donors need a “Who’s Who in microfinance,” that information is already available.)

For context: we are generally sympathetic to the “It’s not perfect, but it’s better than nothing” argument. We are acutely aware that donors have very limited resources today, and we ourselves have generally erred on the side of publishing/sharing our research as quickly as possible. Our research process and our own knowledge have enormous room for improvement (and this was more true when we first published research in November 2007). But we have not gone as far as publishing recommendations while in the middle of our process - we wanted to make sure we could clearly present our criteria and the charities considered, so that people could check our process and hold us accountable if they wished to do so.

There are good arguments for Philanthropedia’s sharing/promoting its research even in its current state, and it’s ultimately Philanthropedia that makes that call.

Now that giving season has ended, we will be shifting our priorities and slowing down the pace of blog posts. Here’s a quick overview of the highlights from our last few months:

• You Can Save A Life: GiveWell in a nutshell. Why give to a charity with a great story and nebulous impact, when by giving to a top charity you can actually save a life?
• Frank discussions of our top two charities, VillageReach and Stop Tuberculosis Partnership.
• Celebrated charities we don’t recommend: a roundup of recent critiques of non-recommended charities.
• Our take on microfinance:
  • 6 myths about microfinance, debunked.
  • “High repayment rates” don’t mean what you think.
  • Summary of our full 17-post series.
• Our take on other approaches to “economic empowerment”:
  • Agricultural training/technology/knowledge sharing
    
  • Gifts of livestock (e.g., Heifer International)
    
  • Social enterprise investment (e.g., Acumen Fund)
• 5-post series on “room for more funding”, the essential question about charities that no one seems to ask.
• Poor in the U.S. = rich: why U.S. poverty and overseas poverty are no comparison. (Also see our post on different definitions of “hunger”.)
• Some thoughts on GiveWell’s general approach:
  • We focus on gifts that help people, not gifts that feel good. We think the latter currently gets too much attention.
  • We don’t focus on “how much a charity spends on overhead.” A broad consensus is emerging that this is the worst way to pick a charity.
    
  • We believe that the burden of proof is on the charities to show that they’re having an impact. Our “default assumption” about a charity is that it isn’t living up to its story, and we think great charities are the exception rather than the rule.

  • As such, we are willing to give a charity a negative rating based on a failure to share meaningful information, even if we don’t have hard proof that it’s failing. This is probably the single most controversial aspect of our process, and we defend it at length in this post.

I believe that Philanthropedia has a promising model. We were glad to join with them on a recent press release urging donors to look beyond administrative expense ratios. We are potentially interested in collaborating with them and/or incorporating their work in the future. I examined their new report on microfinance with great interest.

At this point, I feel that Philanthropedia’s execution of the model has too many serious problems to make it of use to a donor, or to make it a positive influence on the sector. Philanthropedia is a very young organization and we expect/hope to see major improvement, which is why we are sharing these detailed thoughts (we have also discussed them with Philanthropedia).

Overview:

• The definition of a Philanthropedia “expert” is unclear, and we are concerned that the distinction between experience and expertise is being neglected.
• Only 31 of 131 “experts” even appear to have their names accessible on the site. The little we know about the “experts” gives us substantial concerns about their representativeness and credibility.
• Because of concerns about who the experts are, how they are chosen, and what criteria they are applying, we find that we can’t make sense of the final output (i.e., recommended microfinance organizations). We disagree strongly with the recommendations, but cannot see enough of what went into them to examine the sources of the disagreement.
• We would not want donors to give based on this report, not only because of the problems listed above, but because we believe that doing so would reinforce bad incentives.

Details follow. This is a long post, but if you are a nonprofit professional interested in the Philanthropedia model - or a donor considering following its recommendations - I urge you to read it all. There are more substantial concerns than I can present briefly.

The definition of a Philanthropedia “expert” is unclear

How does Philanthropedia define an “expert”? The most specific answer I can find to this question is via the Philanthropedia blog: “To find these experts, we do a variety of things such as research thought leaders in the space, look for program officers at foundations who specialize in this area, research universities who have faculty focused on these issues, identify journalists who write extensively about the topic, look for executive directors or heads of nonprofits working in the space, etc.” The Philanthropedia FAQ adds that “We target experts in a social cause through a combination of cold emails and warm referrals (on the basis of professional and personal connections).”

But the numerical breakdown of experts cites 8% academics, 8% funders, 53% nonprofit executives, and 24% unspecified “others.” (We’re not sure why the percentages add up to <100% but consider this a minor issue.)

• Why such a large proportion of nonprofit executives and “others”? Is this proportion deliberate, based on any particular view of the right proportions, or has it simply “fallen out” of which people Philanthropedia was able to reach and which chose to participate? Is it a function of the “personal connections” mentioned by the Philanthropedia FAQ?
• It seems clear that these numbers don’t come close to the total numbers of academics, nonprofit executives, etc. who would have relevant perspectives. So why these “experts” and not others? Philanthropedia’s description of its process is too vague to give an answer to this question.
• Who are the “others?” Are they all “journalists who write extensively on the topic” or do many fall into the “etc.” category?

Philanthropedia is all about trust. It provides very little information that donors can assess for themselves - the premise is that experts are better positioned to make recommendations. So it is essential to be crystal clear about just what constitutes the “expertise” that makes someone’s opinion so much more valuable and credible than an individual donor’s. We feel that Philanthropedia needs to add a substantial amount of information before it can be considered clear enough.

And as we have written before, we believe there is a huge difference between “experience” and “expertise” - particularly in the nonprofit sector with its broken feedback loops.

Who are the experts?

When we look at what information is available about the “experts” behind the microfinance report, we note:

• The report appears to have surveyed 131 total “experts.” As mentioned above, the vast majority of these are nonprofit executives and “other” and only 8% (~10) are academics.
• This leaves very little room for representation of microfinance skeptics. Nearly everyone, if not everyone, surveyed is likely coming from a worldview that embraces the core assumptions behind the pro-microfinance mentality, and thus embraces microfinance the way it is currently carried out. It seems fairly clear how this issue can lead to bias: for example, people who are already committed enough to microfinance’s potential to be funders/nonprofit executives seem likely to focus more on scale (reaching as many people as possible) rather than social impact (asking critical questions about microfinance’s effect on clients).
• Names and affiliations are provided for only 31 of the 131 “experts.” For 21 of the 31, the only information given is the name, position and organization name (and not all of these organizations can easily be found on the web - for example, is this the WAVE Foundation that employs Anwar Hossain, who is listed as one of the “experts”?)
• What we can see about these “experts” sheds very little light on how they were chosen and how representative they are.
  • Some are top-level executives and some are not.
  • 3 of the 31 represent Unitus; one represents Tearfund, a very large charity working in diverse areas; but other major U.S. microfinance-focused charities such as Grameen Foundation and FINCA have no representatives in this set. (Perhaps Unitus’s heavy representation has something to do with its low rank, since Philanthropedia requires that nonprofit professionals not vote for their own organization.)
  • In what regions have these “experts” done their work? With what kinds of programs? In what ways might their experience and allegiances be biased? Do they have valuable experience and impressive accomplishments or merely many years of being employed in this area? We don’t have the information that could address these questions.

The recommended charities

As stated above, Philanthropedia provides very little information on the recommended charities aside from their names, links to their websites, and figures whose exact meaning is unclear but which seem to indicate the level of “expert support.”

Our main observation is that the list reads like a “who’s who in large U.S. microfinance charities.” This pertains to our above observation that microfinance skeptics don’t seem well represented.

Most of the ratings are clustered fairly close together (7%-11% range). Accion is at the top with 19%, and CARE, FINCA and Unitus are at the bottom. It’s unclear exactly what these numbers mean - should we take from this that Accion is particularly outstanding or that Unitus has problems? Probably not, since Philanthropedia states that it doesn’t rank nonprofits.

We would not, at this point, recommend any of the charities listed:

• Accion, Grameen, Opportunity International, Women’s World Banking and Unitus all fall into the category of “consultant” charities that we’ve expressed reservations about. For all of these organizations, we have serious concerns about their social-impact-focused due diligence.
• Freedom from Hunger, BRAC and CARE all run a wide variety of programs including but not limited to microfinance. It’s not clear that your donation to these organizations will even fund expansion of microfinance (regardless of whether you restrict it).
• We know less about Pro Mujer and FINCA. We have not looked deeply into either as part of our ongoing grant application (Pro Mujer does not work in Africa and FINCA declined to apply).
• MIX Market is an information source (and an excellent one, in my view), not a provider of direct services. I would be open to recommending it but would need to know what it would do with more funds.
• As for Kiva, we have written a lot about it. At this point our basic take is that it’s effectively getting your money to partner microfinance institutions, and we don’t have enough information, or confidence in, its due diligence process on partners from a social impact point of view (our posts on Kiva list a couple particular, substantial points of concern).

The “expert quotes” provided by Philanthropedia do not address these concerns, and in fact raise questions about whether the “experts” are even trying to help donors accomplish good. Take the Kiva page for an example. (Click “Expert assessment” from that link to view quotes.) Nearly all strengths listed pertain to Kiva’s marketing ability: “They have a cute model,” “They are appealing to donors,” and more. In other words, you are being advised to give to a charity because it’s good at making you want to give to it.

The few comments that pertain to other aspects of Kiva raise other concerns:

• “They are truly innovative as a capital source for microfinance institutions.” The exact nature of the innovation is unspecified, but probably amounts to more praise for the marketing.
• “Kiva charges 0% interest rate on capital.” Kiva charges 0% to its partners, but borrowers have to pay interest. Is that a good thing, and if so, why?
• “They provide a direct link from funder to borrower” - this is simply false.
• “They are not neglecting the US domestic poverty issue” - this comment comes up in more than one organization’s profile. We have argued that it is better not to focus on US domestic poverty; in any case, this is clearly a difficult, value-laden judgment call.

The “expert quotes” on other organizations raise similar concerns. Overall, it is not apparent that the “experts” are giving advice based on how an individual donor can have the most impact. It is not clear what their recommendations mean.

As for our own take on which microfinance charities are best, we have deep concerns about this sector that have led us to go beyond U.S. charities and seek out outstanding microfinance institutions. We’ll be writing about more about the best we’ve found, the Small Enterprise Foundation.

We are coming from an attitude of skepticism toward microfinance in general. Our skepticism is certainly debatable, but we have been explicit about it and discussed our thoughts on microfinance at length. A donor who wants to know where our advice is coming from and what it’s based on can find out. We feel it is important that donors have similar context on the people cited by Philanthropedia.

Reinforcing bad incentives

We pay a lot of attention to incentives. When discussing how we should rate charities, we constantly ask, “If charities all knew we were handing out ratings on this basis and expected donors to give based on our ratings, how would they work the system? What would we expect to change?” (For example, we give negative reviews to charities that publish no information, not just charities whose materials raise concerns; we don’t want to provide incentives to hide information.)

We feel that if Philanthropedia stuck to its current approach and donors used it:

• “Experts” would not have any incentives to increase their own transparency above current, unacceptable levels. They could influence others’ giving by giving recommendations without substantive reasons and without even disclosing their names.
• Charities would have strong incentives to do whatever got them recommended by Philanthropedia’s set of experts. This could include networking, flattery, wining/dining, outright conflicts of interest, and learning to speak of their programs in ways that would appeal to Philanthropedia experts, all of which might end up being more cost-effective from the charities’ perspective than optimizing and demonstrating their social impact.

Bottom line
I urge Philanthropedia to:

• Publish exhaustive details of how “experts” are defined, selected and invited.
• Publish names and affiliations of all experts who are invited to participate, whether or not they accept.
• It is particularly important to publish names and affiliations for those who do participate because donors should not be asked to trust anonymous people.
• Address the concerns we have raised about the representativeness and credibility of experts, if these concerns still appear valid once the identities and process are disclosed.
• Publish the templates for all surveys sent to experts. Be explicit about the criteria by which experts are asked to recommend charities.
• Ask experts to make their best case publicly and to be clear about whether they’re recommending charities to impact-focused donors, praising charities for marketing abilities, or something else.

We believe the Philanthropedia model has a lot of potential. We would find great use, for donors and for ourselves, in a site that could credibly claim to have assembled the charities with the most support from people with relevant experience.

Philanthropedia has accomplished a lot in a short time, but we don’t believe its content is yet appropriate for distribution to individual donors looking for help with their giving decisions.