I believe that Philanthropedia has a promising model. We were glad to join with them on a recent press release urging donors to look beyond administrative expense ratios. We are potentially interested in collaborating with them and/or incorporating their work in the future. I examined their new report on microfinance with great interest.

At this point, I feel that Philanthropedia’s execution of the model has too many serious problems to make it of use to a donor, or to make it a positive influence on the sector. Philanthropedia is a very young organization and we expect/hope to see major improvement, which is why we are sharing these detailed thoughts (we have also discussed them with Philanthropedia).

Overview:

• The definition of a Philanthropedia “expert” is unclear, and we are concerned that the distinction between experience and expertise is being neglected.
• Only 31 of 131 “experts” even appear to have their names accessible on the site. The little we know about the “experts” gives us substantial concerns about their representativeness and credibility.
• Because of concerns about who the experts are, how they are chosen, and what criteria they are applying, we find that we can’t make sense of the final output (i.e., recommended microfinance organizations). We disagree strongly with the recommendations, but cannot see enough of what went into them to examine the sources of the disagreement.
• We would not want donors to give based on this report, not only because of the problems listed above, but because we believe that doing so would reinforce bad incentives.

Details follow. This is a long post, but if you are a nonprofit professional interested in the Philanthropedia model – or a donor considering following its recommendations – I urge you to read it all. There are more substantial concerns than I can present briefly.

---

The definition of a Philanthropedia “expert” is unclear

How does Philanthropedia define an “expert”? The most specific answer I can find to this question is via the Philanthropedia blog: “To find these experts, we do a variety of things such as research thought leaders in the space, look for program officers at foundations who specialize in this area, research universities who have faculty focused on these issues, identify journalists who write extensively about the topic, look for executive directors or heads of nonprofits working in the space, etc.” The Philanthropedia FAQ adds that “We target experts in a social cause through a combination of cold emails and warm referrals (on the basis of professional and personal connections).”

But the numerical breakdown of experts cites 8% academics, 8% funders, 53% nonprofit executives, and 24% unspecified “others.” (We’re not sure why the percentages add up to <100% but consider this a minor issue.)

• Why such a large proportion of nonprofit executives and “others”? Is this proportion deliberate, based on any particular view of the right proportions, or has it simply “fallen out” of which people Philanthropedia was able to reach and which chose to participate? Is it a function of the “personal connections” mentioned by the Philanthropedia FAQ?
• It seems clear that these numbers don’t come close to the total numbers of academics, nonprofit executives, etc. who would have relevant perspectives. So why these “experts” and not others? Philanthropedia’s description of its process is too vague to give an answer to this question.
• Who are the “others?” Are they all “journalists who write extensively on the topic” or do many fall into the “etc.” category?

Philanthropedia is all about trust. It provides very little information that donors can assess for themselves – the premise is that experts are better positioned to make recommendations. So it is essential to be crystal clear about just what constitutes the “expertise” that makes someone’s opinion so much more valuable and credible than an individual donor’s. We feel that Philanthropedia needs to add a substantial amount of information before it can be considered clear enough.

And as we have written before, we believe there is a huge difference between “experience” and “expertise” – particularly in the nonprofit sector with its broken feedback loops.

Who are the experts?

When we look at what information is available about the “experts” behind the microfinance report, we note:

• The report appears to have surveyed 131 total “experts.” As mentioned above, the vast majority of these are nonprofit executives and “other” and only 8% (~10) are academics.
• This leaves very little room for representation of microfinance skeptics. Nearly everyone, if not everyone, surveyed is likely coming from a worldview that embraces the core assumptions behind the pro-microfinance mentality, and thus embraces microfinance the way it is currently carried out. It seems fairly clear how this issue can lead to bias: for example, people who are already committed enough to microfinance’s potential to be funders/nonprofit executives seem likely to focus more on scale (reaching as many people as possible) rather than social impact (asking critical questions about microfinance’s effect on clients).
• Names and affiliations are provided for only 31 of the 131 “experts.” For 21 of the 31, the only information given is the name, position and organization name (and not all of these organizations can easily be found on the web – for example, is this the WAVE Foundation that employs Anwar Hossain, who is listed as one of the “experts”?)
• What we can see about these “experts” sheds very little light on how they were chosen and how representative they are.
  • Some are top-level executives and some are not.
  • 3 of the 31 represent Unitus; one represents Tearfund, a very large charity working in diverse areas; but other major U.S. microfinance-focused charities such as Grameen Foundation and FINCA have no representatives in this set. (Perhaps Unitus’s heavy representation has something to do with its low rank, since Philanthropedia requires that nonprofit professionals not vote for their own organization.)
  • In what regions have these “experts” done their work? With what kinds of programs? In what ways might their experience and allegiances be biased? Do they have valuable experience and impressive accomplishments or merely many years of being employed in this area? We don’t have the information that could address these questions.

The recommended charities

As stated above, Philanthropedia provides very little information on the recommended charities aside from their names, links to their websites, and figures whose exact meaning is unclear but which seem to indicate the level of “expert support.”

Our main observation is that the list reads like a “who’s who in large U.S. microfinance charities.” This pertains to our above observation that microfinance skeptics don’t seem well represented.

Most of the ratings are clustered fairly close together (7%-11% range). Accion is at the top with 19%, and CARE, FINCA and Unitus are at the bottom. It’s unclear exactly what these numbers mean – should we take from this that Accion is particularly outstanding or that Unitus has problems? Probably not, since Philanthropedia states that it doesn’t rank nonprofits.

We would not, at this point, recommend any of the charities listed:

• Accion, Grameen, Opportunity International, Women’s World Banking and Unitus all fall into the category of “consultant” charities that we’ve expressed reservations about. For all of these organizations, we have serious concerns about their social-impact-focused due diligence.
• Freedom from Hunger, BRAC and CARE all run a wide variety of programs including but not limited to microfinance. It’s not clear that your donation to these organizations will even fund expansion of microfinance (regardless of whether you restrict it).
• We know less about Pro Mujer and FINCA. We have not looked deeply into either as part of our ongoing grant application (Pro Mujer does not work in Africa and FINCA declined to apply).
• MIX Market is an information source (and an excellent one, in my view), not a provider of direct services. I would be open to recommending it but would need to know what it would do with more funds.
• As for Kiva, we have written a lot about it. At this point our basic take is that it’s effectively getting your money to partner microfinance institutions, and we don’t have enough information, or confidence in, its due diligence process on partners from a social impact point of view (our posts on Kiva list a couple particular, substantial points of concern).

The “expert quotes” provided by Philanthropedia do not address these concerns, and in fact raise questions about whether the “experts” are even trying to help donors accomplish good. Take the Kiva page for an example. (Click “Expert assessment” from that link to view quotes.) Nearly all strengths listed pertain to Kiva’s marketing ability: “They have a cute model,” “They are appealing to donors,” and more. In other words, you are being advised to give to a charity because it’s good at making you want to give to it.

The few comments that pertain to other aspects of Kiva raise other concerns:

• “They are truly innovative as a capital source for microfinance institutions.” The exact nature of the innovation is unspecified, but probably amounts to more praise for the marketing.
• “Kiva charges 0% interest rate on capital.” Kiva charges 0% to its partners, but borrowers have to pay interest. Is that a good thing, and if so, why?
• “They provide a direct link from funder to borrower” – this is simply false.
• “They are not neglecting the US domestic poverty issue” – this comment comes up in more than one organization’s profile. We have argued that it is better not to focus on US domestic poverty; in any case, this is clearly a difficult, value-laden judgment call.

The “expert quotes” on other organizations raise similar concerns. Overall, it is not apparent that the “experts” are giving advice based on how an individual donor can have the most impact. It is not clear what their recommendations mean.

As for our own take on which microfinance charities are best, we have deep concerns about this sector that have led us to go beyond U.S. charities and seek out outstanding microfinance institutions. We’ll be writing about more about the best we’ve found, the Small Enterprise Foundation.

We are coming from an attitude of skepticism toward microfinance in general. Our skepticism is certainly debatable, but we have been explicit about it and discussed our thoughts on microfinance at length. A donor who wants to know where our advice is coming from and what it’s based on can find out. We feel it is important that donors have similar context on the people cited by Philanthropedia.

Reinforcing bad incentives

We pay a lot of attention to incentives. When discussing how we should rate charities, we constantly ask, “If charities all knew we were handing out ratings on this basis and expected donors to give based on our ratings, how would they work the system? What would we expect to change?” (For example, we give negative reviews to charities that publish no information, not just charities whose materials raise concerns; we don’t want to provide incentives to hide information.)

We feel that if Philanthropedia stuck to its current approach and donors used it:

• “Experts” would not have any incentives to increase their own transparency above current, unacceptable levels. They could influence others’ giving by giving recommendations without substantive reasons and without even disclosing their names.
• Charities would have strong incentives to do whatever got them recommended by Philanthropedia’s set of experts. This could include networking, flattery, wining/dining, outright conflicts of interest, and learning to speak of their programs in ways that would appeal to Philanthropedia experts, all of which might end up being more cost-effective from the charities’ perspective than optimizing and demonstrating their social impact.

Bottom line
I urge Philanthropedia to:

• Publish exhaustive details of how “experts” are defined, selected and invited.
• Publish names and affiliations of all experts who are invited to participate, whether or not they accept.
• It is particularly important to publish names and affiliations for those who do participate because donors should not be asked to trust anonymous people.
• Address the concerns we have raised about the representativeness and credibility of experts, if these concerns still appear valid once the identities and process are disclosed.
• Publish the templates for all surveys sent to experts. Be explicit about the criteria by which experts are asked to recommend charities.
• Ask experts to make their best case publicly and to be clear about whether they’re recommending charities to impact-focused donors, praising charities for marketing abilities, or something else.

We believe the Philanthropedia model has a lot of potential. We would find great use, for donors and for ourselves, in a site that could credibly claim to have assembled the charities with the most support from people with relevant experience.

Philanthropedia has accomplished a lot in a short time, but we don’t believe its content is yet appropriate for distribution to individual donors looking for help with their giving decisions.

Before giving season ends, we wanted to make sure to put in a good word for the UPenn Center for High-Impact Philanthropy (which we have mentioned before).

We have carefully reviewed the Center’s reports on malaria and U.S. education, both of which overlap heavily with the research we have done. We consistently find their statements to be research- and evidence-based, with citations that make it clear where their reasoning comes from. Their reports are focused on the practical issues of philanthropy and give examples of specific successful nonprofit programs.

We have learned from these reports and we believe they have significant value for donors.

We haven’t yet carefully reviewed their most recent report, on U.S. housing, health and hunger, but if and when we research these causes it will be one of the first sources we consult.

As Peter Singer and William Easterly recently discussed, there is a severe shortage of groups that provide substantive public information about the facts of giving and the research relevant to it. The Singer/Easterly discussion noted GiveWell and Good Intentions are Not Enough (another resource we find helpful) as examples of such groups. We think that the Center belongs on the list as well.