The GiveWell Blog

Should charity evaluation be “fair”?

One of the goals we do not have is the goal of being “fair” to charities, in the sense expressed in this comment:

You are right on in your focus on provable results, but some areas are much easier to evaluate than others … A bad charity to me is one that is misleading, not transparent, or ineffective in comparison to its peers.

It seems common that people wish to be “fair” to charities, focusing on merit (how well they do what they do) rather than results. (Objections that our process favors large charities often come from a similar place.) How do people trade off merit vs. results in other domains?

  • Investing: if company A looks more likely to be profitable than company B, most people will invest in company A. They won’t give a second’s thought to considerations like “But company B is in a more difficult sector.”
  • Consuming. If you’re buying an iPad, I doubt you’ve worried much about how unfortunate Notion Ink is for not having the brand name, marketing budget and sheer size to compete for your dollars.
  • Sports. Here we take significant steps to “level the playing field.” We are very careful about which advantages (strength, speed) we allow and which (equipment, performance-enhancing drugs) we do not.

We feel strongly that charity evaluation should not be seen as a contest, bringing glory to meritorious charities and dishonor to scams. Instead, it should be seen as a pragmatic way to get money to where it can do as much good as possible. “Merit,” “A good attempt given the difficulty of the problem,” etc. should be left out of the picture.

That’s why we’ve always taken more care to eliminate “false positives” than “false negatives” in our recommendations. If there’s an ineffective charity we recommend, that’s a real problem. If there’s a good one that our process has failed to identify, that may be bad from a “fairness” perspective, but it’s not nearly so problematic for the goal of helping donors accomplish good. Evaluators that are more inclined to give charities the benefit of the doubt probably “wrong,” and anger, fewer charities – but they’re less effective in directing funding to where it will accomplish a great deal of good.

And that’s why we’re unapologetic about our bias toward charities working on tangible, measurable goals. Health is an “easier” sector than economic empowerment for clearly defining goals, tracking progress toward them, learning from mistakes, and ultimately making a positive difference. That’s a reason to prefer health charities, not a reason to “handicap” them. (Note that we do think there can be good reasons to give to “less measurable” causes, including philosophical preferences; more on this in a future post.)

When we spend money on ourselves (investing or consuming), we think exclusively about meeting our own needs and wants. It’s only fair that when we spend money on others (charity), we should think exclusively about meeting theirs.

Microfinance interest rates

One of the more difficult things to understand about the microfinance institutions we’ve investigated is the “true” interest rate they’re charging their borrowers. In July 2009, David Roodman of the Center for Global Development wrote:

It appears that many MFIs impose subtle fees that effectively raise interest rates. Some charge one-time loan origination fees. Some require borrowers to deposit a percentage of each loan amount with the MFI in a savings account that pays interest at a rate lower than that on the loan. Some overcharge for credit-life insurance bundled with the loan. Another criticized practice is to charge interest on the full loan amount even as the outstanding balance declines over the repayment cycle. Such “flat-rate” interest effectively doubles the interest rate compared to “declining-balance” interest since the average balance over the cycle is half the starting amount. Also, MFIs may also prefer to quote their rates on a monthly basis, hoping to exploit borrowers’ ignorance of how a seemingly modest 6 percent per month compounds into 100 percent per year.

Our experience with the microfinance organizations we have investigated to date suggests that these are real issues that donors should be aware of when interpreting interest rate data.

To be clear, we don’t think charging a high interest rate signifies wrongdoing on the part of a microfinance bank. High interest rates may be the best way to minimize losses and serve more people, and client participation at high interest rates may be an indicator that they are getting a service they value. We just want to note how striking the difference is between the initial “cited” interest rate donors often hear about and the “effective” interest rate taking all factors into account.

We recently evaluated a microfinance institution (MFI) in Malawi, the Microloan Foundation, as part of our process for distributing a grant to an economic empowerment charity in Sub-Saharan Africa. Its stated interest rate for its most popular loan type is 20%, but:

  • 20% is the rate over the course of the 4-month loan. The “nominal Annual Percentage Rate (APR)” (a common way of stating interest rates in standardized terms and the rate which U.S. lenders are required to provide to borrowers) of this loan, with no other costs, would therefore be 60%.
  • Interest is calculated as a flat rate. 20% of the whole loan amount is charged each payment, instead of 20% of the remaining loan balance. This method raises the nominal APR from 60% to 93%. On a $100 loan of this type, a borrower would pay $20 in interest compared with only $12.80 on a loan with declining balance interest.
  • Payments are due every two weeks, instead of every month, so that the first payment is due only two weeks after the loan is made. Requiring 8 payments instead of 4 raises the nominal APR from 93% to 102%.
  • The Microloan Foundation requires borrowers to hold 20% of the loan amount in a savings account which cannot be accessed until the loan is repaid. On a loan of $100, this requirement reduces the effective size of the loan to $80, while decreasing the effective size of the last payment due by $20 (because savings are then accessible). The savings requirement raises the APR from 102% to 149%.
  • Arguably, the APR used above (i.e. the “nominal APR”) understates the interest rate people are paying because it does not take into account the compound value of interest. (Wikipedia’s entry on APR has a discussion of the relevant issues.) At relatively low interest rates, such as the interest rates we’re used to in the U.S., the “nominal APR” (what is usually reported) and the “effective APR” (the “mathematically true” interest rate) are usually very close to each other – but at the much higher interest rates charged by microfinance institutions, the “effective APR” can be considerably higher, raising the question of which one should be quoted to give Americans the best picture of what people are being charged. The Microloan Foundation’s nominal APR of 149% is equivalent to an effective APR of 326%.

Not only are these final “effective” interest rates many times bigger than the initial “20%” figure, they’re also significantly higher than would be implied by looking at MLF’s nominal gross portfolio yield according to MixMarket (93% for 2008; Mix Market defines this as interest and fees divided by the gross average loan portfolio ).

MLF doesn’t charge fees on its loans, but other microfinance institutions do, and these can cause further significant increases in the effective interest rate. (For example, adding a 5% fee to the beginning of a loan that without fees would have a nominal APR of 40% raises its nominal APR to 66%.)

From what we’ve seen, fees, compulsory savings, and the flat interest rate method seem to be fairly common among microfinance institutions. Of the 65 MFIs who had submitted a Social Performance Standards Report to Mix Market as of late 2009, 42% use the flat interest rate exclusively, 29% held compulsory savings accounts, and 72% collected fees on at least some loan products (according to their Mix Market profiles).

Understanding the true cost of credit for a borrower is important for reasons discussed previously:

  • High interest rates (combined with high rates of repayment and low drop out rates) show that borrowers are willing to pay a high price for the loan, arguably implying that they value this service.
  • If interest rates are low, microfinance institutions may be effectively giving cash transfers, at which point the mere fact of participation becomes less meaningful, and it becomes more important to ask whom the handouts are going to.
  • On the other hand, the higher the interest rate, the more we worry about whether borrowing is good for the borrowers (and about anecdotes like this one).

A fairly new initiative (started in 2008), MFTransparency, is working to create a more open discussion about microloan pricing, and has compiled and published interest rate data for two countries, Bosnia and Cambodia. We look forward to following the progress of this initiative and to drawing on its data to inform our investigations of microfinance organizations.

Think Before You Give: A new podcast with Elie Hassenfeld and Saundra Schimmelpfennig

GiveWell has teamed up with Saundra Schimmelpfenig of Good Intentions Are Not Enough to produce Think Before You Give, a podcast aimed at helping to empower and educate donors. We enjoy Saundra’s work and are happy to be working with her on this.

The Think Before You Give website now has the first podcast available. The topic is the use of “administrative costs” in rating charities (an idea that has come under fire lately). The next episode will have an interview with the author of Tales from the Hood.

Also see Saundra’s announcement. We’re interested in feedback & suggestions for the podcast.

TakeAction @ GuideStar

We are thrilled to announce a partnership with GuideStar via its new feature, TakeAction @ GuideStar.

TakeAction allows donors to select a cause that interests them and see general information as well as recommended charities, with content provided by GiveWell, Philanthropedia, and Great Nonprofits.

To date, GuideStar has provided only basic information, and only information at the level of “charities,” not “causes.” A donor searching for a particular charity could find financial information and (in some cases) open-participation reviews from Great Nonprofits, but could not find (a) information about the charities’ likely effectiveness & impact; (b) information to help get basic grounding in a cause, and choose the best charities within it.

We are particularly excited about the latter change. We believe strongly in “active giving” (finding the best charity possible in your cause of interest) as opposed to “passive giving” (starting with a particular charity in mind and giving as long as it raises no red flags). We believe that GuideStar’s new content is a major and important step toward helping its users give to the best possible organizations, and create healthier incentives in the sector.

TakeAction is new, and we expect that it has a lot of progress yet to make. (We believe that all three of the partner organizations have a lot of progress of our own to make, hopefully leading to coverage of more causes among other things). Today’s launch is only a preliminary step. GuideStar recognizes this and is eager for feedback, so please send it using the “Feedback” links on their various pages.

But for a sector that has been dominated by flawed, superficial metrics and giving focused on donors’ needs, rather than recipients’, this is an enormously encouraging development. One of the oldest, best-established donor resources is partnering with new ventures so that it can make real progress in what it enables donors to do. We hope this is just the beginning.

A good response re: diversion of skilled labor

I previously expressed concerns about diversion of skilled labor: the possibility that nonprofits are outbidding the local private sector for top local talent. John de Wit of the Small Enterprise Foundation emailed us with a response:

On the question of whether we divert skilled labour from other potentially productive pursuits let me try to comment on this by referring to our actual practice:

  • At the moment I can only think of two staff members, both current or past, who were working with any other agency which focussed on poverty alleviation or microcredit or some other development field … Both worked for different NGOs which did water provision. Neither came to us because of better pay. The one came because of being somewhat frustrated at the former employer and we wanted them to do work which was more in line with their personal professional interests. The other came because they were keen to work more on a part-time basis.
  • The vast majority of our employees were unemployed when they came to work for SEF and it is only in the “professional” posts that we have employed people who already had employment. These worked for general commercial companies e.g. a producer of canned goods, a commercial farm, a legal firm etc.
  • In general we are only able to pay at the 25-percentile level versus the general market. i.e. we grade all posts in the organisation using a standard grading system which is employed by commercial firms, non-profits and some government agencies. We then obtain national survey data for what employers are paying at each grade level. We try to pay at least at the 25-percentile level. This means that 75% of employers pay more than us and 25% pay less than us. (Paying at this level is not a good position to be in.) To illustrate this please see the attached file [confidential]
  • Generally when a job candidate is already employed then we offer about 10% more than their current package.
  • Besides from pay and our location another major challenge we have in attracting and retaining good people is that most people would rather work for a large, high profile company than some small “unknown” NGO in a small “unknown” town. The big companies have a very good public image and appear to offer extensive career opportunities.

I asked for clarification on what he meant by “unemployed” (chronic or transitional?) and “professional positions.” He sent me a complete breakdown of what he knew about the previous employment of major staff. I am not sharing the breakdown (due to the presence of individual names and salary information), but here is a summary:

Employment situation prior to joining SEF  
Started as DF/admin (not “professional”) 64%
Previously employed 11%
Unclear 8%
Employed directly out of school 8%
Volunteer 5%
Chronically unemployed 4%

The question of “diversion of skilled labor” is one that we haven’t even asked charities, because we’ve felt it is so unlikely that they will have substantive information to address it. SEF’s email suggests otherwise for at least one case.

A proposal for donors interested in causes/charities we haven’t covered

I was talking with a friend of mine recently about how he decides which charities to support, and he said:

I really like the GiveWell approach, but there are two reasons it’s not practical for me to base most of my charitable decisions on it. First, you just haven’t covered a lot of the areas I care about. I want to give to support food banks, but you haven’t covered food banks. Second, a lot of the time, I get requests from friends or solicitations from charities (referred by friends), and I need information on a specific charity — that’s not something GiveWell provides.

These two issues — GiveWell’s lack of breadth in coverage of different causes and specific charities — are probably the most common points a lot of donors make when they think about using our research.

Here, I want to make a proposal that I think solves the problem for donors like my friend. If you agree with GiveWell’s philosophy about giving, do the following:

  • First, when a charity (or friend) solicits you to support their cause, list a set of important questions you’d need them to answer to give your confidence that their approach is working. This is the approach GiveWell generally takes. (For example, see our questions for surgery charities, water charities and microlending charities.)

    If you need help creating a list of questions, email us and we’ll send you our thoughts. If you have your own, send them to us, so we can publish the questions that donors are using, and others can rely on the questions that have already been created.

  • If they can answer your questions compellingly, and using specifics and facts rather than generalities and stories, great! Write them a check. (Unfortunately, this result has been unusual in my experience.)
  • If they can’t answer your questions, write a check to a donor-advised fund and tell them that when they can answer your questions, you’ll recommend a grant to them from your account.

Here’s an example of how this would work.

A charity approaches you and asks for a donation. Let’s say it’s a food bank. The charity says, “People are hungry. Giving to us will help provide poor individuals with the food they need to survive. And, our approach is to pick up food that’s going to be thrown out by local stores and restaurants, so your donation is leveraged and will help a lot of people.”

Instead of just writing a check, ask the charity the following (these are just a few questions that come to mind when thinking about this issue):

  1. Is using donations to pick up food the only program you run, or do you run other programs as well? What portion of your overall budget does each program account for?
  2. Who are the people that your food bank serves? What type of food-needs do they have? (You may be surprised.)
  3. Is money, specifically, a bottleneck to providing more people more food? (This is part of the room-for-more-funding question that we think is essential to investigate.) That is, it seems plausible that the bottleneck to providing more food is the supply of “leftover food,” not funds.
  4. How much more food can you commit to provide if you receive another $100,000? $1 million?
  5. Is the food you’re providing safe? Healthy? What type of food do you provide? Have you ever needed to discard food because it had spoiled? What rules do you follow to make decisions to discard food? How does your organization’s senior management know that the food delivered is high quality?

The beauty of this approach is that (a) you force yourself to give charitably when asked — you’re not just ignoring charities or friends; (b) you help to create good incentives for charities by only rewarding those that can make a convincing case for strong results; (c) you’ll help us create a repository of questions to ask charities working on different causes; and (d) you’ll still get a tax deduction.