The GiveWell Blog

Medicine and philanthropy

(updated on: ) | by Holden

David Leonhardt’s excellent piece on health care reminded me of the debates within philanthropy.

For most of human history … [doctors’] treatments consisted of inducing vomiting or diarrhea and, most common of all, bleeding their patients … Yet patients continued to go to doctors, and many continued to put great in faith in medicine … There was a strong intuitive logic behind those old treatments; they seemed to be ridding the body of its ills. They made a lot more sense on their face than the abstract theories about germs and viruses that began to appear in the late 19th century … So the victory of those theories would require a struggle. The doctors and scientists who tried to overturn centuries of intuitive wisdom were often met with scorn. Hippocrates himself wrote that a physician’s judgment mattered more than any external measurement, and the practice of medicine was long organized accordingly.

The single most common retort to the GiveWell approach is that the staffers, volunteers, and even donors “know” the programs they fund are working, based on their intuitions.

Then again, many highly intuitive programs have been shown not to work, and there’s good reason to distrust intuition in this area:

Behavioral researchers have come to believe that there is a clear pattern to when intuition works and when it doesn’t. “Intuitive diagnosis is reliable when people have a lot of relevant feedback,” says Daniel Kahneman, a Nobel laureate in economics who recently collaborated on a project about intuition with Klein. People need a great deal of experience, and the feedback from these experiences — whether a treatment is working, say — needs to come quickly and to be clear. “But,” Kahneman adds, “people are very often willing to make intuitive diagnoses even when they’re very likely to be wrong.” When doctors have been asked to estimate the likelihood of a treatment succeeding based on experience, for example, they give wildly divergent answers. Medicine is full of such examples.”

Feedback is near-nonexistent in the field of philanthropy.

As Toyota built better cars than its competition for less money, it won new customers. Some rivals matched its successes (as Honda did); some lost market share (as Detroit did). No such dynamic exists in health care. William Lewis, a former director of the McKinsey Global Institute who studies productivity, says that the economic benefits from the various quality movements have been quite large but that they are also largely in the past. Most industries have incorporated Deming’s big ideas and are now making only incremental progress. “However, there is one big exception,” Lewis adds. “You guessed it: health care.”

I can think of another exception.

Evaluating microsavings

(updated on: ) | by Holden

We’re excited about the idea of microsavings as opposed to microlending. But it isn’t enough to see that an organization offers microsavings. We need to know:

  1. Are savings services being provided relatively efficiently? How many clients are served per dollar of operating expenses?
  2. Are clients able to access their funds when they need them? We have heard anecdotal concerns about client dissatisfaction with the difficulty or bureaucracy involved in accessing savings. In addition to the proxies for satisfaction discussed in our earlier post, we’d like to see the “turnover” of client accounts: does money go in and out, or sit stagnant?
  3. What are the interest rates/fees on the accounts? Excessive fees would concern us, but so would extremely generous interest rates, which would make the program less like a savings account than like giving out cash.

Some of the questions at our earlier post (regarding profitability, client income levels, and client satisfaction) also apply.

If microsavings is more needed, why does microcredit get more attention?

(updated on: ) | by Holden

Portfolios of the Poor tells a story you won’t hear from a typical microfinance charity:

Sankar was a landless, illiterate rickshaw driver, whose wife had Grameen membership. They had borrowed from Grameen Bank a few times – in fact one loan had helped him buy his rickshaw. Suddenly his wife told him they would have to open a GPS [savings account] in order to get the next loan. He was suspicious, he told us. “And now?” we asked. He chuckled. “Now, we try to avoid loans and just use the GPS.” Pressed to explain, he said that his income was small but sufficient for their daily needs and they had nothing to invest an expensive loan in. Their priorities now were for their children, and the GPS seemed, compared to borrowing, a cheaper, more relaxed, longer-term way of providing for their future (marriage for the girl, a business for the boy). Like Jharimon, Sankar borrowed sometimes and saved always. “Grameen should have done this years ago,” he said, echoing what many others had told us. (Page 170; emphasis mine)

What do the poor need more, savings or loans? There isn’t much information on this question – something I find pretty shocking given that microfinance is decades old. But the limited evidence seems to point to savings, at least for the poorest clients:

  • The general impression given by Portfolios of the Poor (quoted above), the only systematic attempt I’m aware of to see how the poor manage their financial lives.
  • A 2001 survey of SEWA Bank clients, which found that the bank was “far more important as a depository for savings in early 1998 than it was as a source of credit” (page 79) and that “39 per cent of borrower households were below the $1 a day line, compared to 53 per cent of saver households and 67 per cent of [non-client] households.” (page 81) This is the only study we’ve located directly assessing savings vs. loans clients at an MFI – please let us know if you know of more – but it is consistent with charities’ and scholars’ off-the-record comments to us that the poorest clients tend to be more interested in savings than in loans.
  • The strongest studies of microfinance, which show weak/no effects for two traditional microcredit program and positive effects for a savings program.
  • The purely logical argument that savings provides the same “risk management” benefits as loans, without the same risk of harm that comes with putting poor people into debt at high interest rates. (David Roodman among others makes this argument.)

But loans are far more prevalent

Looking at the 537 MFIs listed on MixMarket under US-based networks (source data), we note that

  • 517 of 537 appear to have introduced loans before they introduced savings; 0 of 537 appear to have introduced savings first. (This may be an artifact of changing data reporting standards, but if Mix collected loans data before savings data, that fact is also indicative).
  • In their most recent year, 503 of 537 MFIs had more outstanding loans than savings; 34 of 537 had more outstanding savings than loans.

Why?

Why are loans so overwhelmingly common compared to savings, when the little available evidence points to a greater need for savings?

Is it for purely logistical reasons? Savings and loans each present different challenges, but the necessary “innovations” seem to have come first and been expanded more for loans.

Or is this another case where the best program loses to the best donor story?

Too much attention on the giver’s experience; not enough on the recipient’s

(updated on: ) | by Holden

From the recent discussion on Tactical Philanthropy:

As philanthropists/donors/funders, we spend so much time thinking about how to maximize social benefit through our activities, that often we lose sight of the personal benefits that we experience from these endeavors.

We disagree. Consider the current state of the nonprofit sector.

Bottom line: giving today is all about the giver. The sector revolves around telling donors great stories, while charities’ actual impact is unexamined and essentially irrelevant.

We do believe that the benefits of giving for the giver are important, particularly from a fundraising perspective, but we think the current level of attention to them is out of control.

Our process: Narrowing the field

(updated on: ) | by Holden

One of the aspects of our research process that has generated some objections is our use of “heuristics,” i.e., shortcuts to winnow the field of recommended charities from 300+ to a manageable number for closer investigation. The heuristics we use are described here. A good statement of the objections comes this comment at Hatrack forums:

I don’t care if a charity’s evaluation and monitoring reports are on their website, as long as they are publically available in some way. And while I agree with many of thier priorities, 75% of funding or more matching a list of specific programs is not vetting cost-effectiveness, it’s vetting whether or not the organization has the same priorities as Givewell does.

This post briefly explains and defends our approach. It does not discuss our criteria (proven, cost-effective, scalable, transparent), but rather the shortcuts we use to identify the charities most likely to meet those criteria.

The most important thing to know is that we are always ready to look at charities that don’t pass these heuristics, if they meet our broader criteria. If you know of such charities, please alert us using our submission form. If it appears that the information we require is available – whether or not it’s available on the charity’s website – we will change a charity’s status to “Pending” until we have reviewed it more thoroughly.

Why do we look at what information is available on a charity’s website, instead of searching more comprehensively and contacting them directly?

We have found that going back-and-forth with charities to see what they have internally is extremely difficult and time-consuming for both us and them. We are generally first connected to fundraising staff, and it takes a lot of communication and waiting just to end up talking to someone who knows what information is available. Repeating this process for all 300+ charities we have examined would not be practical, so we use a heuristic to identify the most promising candidates for further investigation.

We are explicit that our research is constrained by practical considerations. Our goal is not to be “perfect” in our assessments but rather to provide better information than donors can find anywhere else.

We do contact all rated charities to let them know about their status and how they can change it if they feel we are in error.

Is there independent evidence that “what information is on the website?” is a reasonable proxy for “what information is available at all?”

Yes. We have also used alternate research methods that involve much more back-and-forth with charities, and feel that the results give support to the “website scanning” heuristic as an imperfect but pretty good predictor of which charities actually have the information we require (particularly evidence of impact).

  • Our first-year research process involved applications for grants of $25,000-$40,000. All non-confidential application materials have been publicly posted. For all five charities that earned a 2 star or better rating through this process, the primary evidence of impact we used is available on or via their website.
  • We’re currently conducting a grant application process for $250,000 and will be publishing the full details of what it turns up in early 2010.

Also note that our “website scanning” heuristic is similar to the method used by William Easterly and Tobias Pfutze to rate aid agencies (PDF). Our aim is similar in that we seek to reward organizations that have both good practices and the transparency to share their practices publicly.

Do we require that charities be running “priority programs” in order to receive further investigation and/or high ratings?

No. The two heuristics we use are “or”, not “and.” We don’t require charities to share our program priorities. Rather, we investigate charities that do share these priorities even if they don’t pass the other heuristic. We do this because we have enough capacity to deeply investigate some “extra” charities, but not all 300+.

Why do we issue ratings to charities that don’t pass our heuristics, rather than simply marking them as “Not examined?”

We feel it would be misleading to simply say “not examined” for the charities that didn’t pass the heuristics. Given the constraints of what information is available and what’s practical, we feel strongly that there is a better case for the highly-rated charities than for the examined-but-not-rated charities. By contrast, a charity that doesn’t appear at all is one we simply haven’t looked at.

We feel it is accurate and important to call our top-rated charities the standouts (by our criteria) from a field of 300+.

Donor illusions

(updated on: ) | by Holden

There’s an persistent conflict in international charity:

  • It feels great to be able to say, “My donation helped THIS person.”
  • But it’s rarely – if ever – practical for that sort of connection to be real.

As a result, international charities tend to create “donor illusions” by implying that donations can be attributed more tangibly, reliably and specifically than they really are. Some charities are more purposefully misleading than others, and some have more prominent and clear disclosures than others, but we feel that all of the cases below end up misleading many donors.

Kiva.org: making a loan
Illusion: You pick a developing-world entrepreneur who needs a loan. You lend money to that entrepreneur, interest-free. With you, s/he wouldn’t have gotten the loan.
Reality: The entrepreneur you’re viewing in your browser probably already got his/her loan and is probably paying significant interest on it. What you’re really doing is sending money to a microfinance institution that uses it as it sees fit (including for loans to less creditworthy people).

Details at GiveWell Board member Tim Ogden’s summary of the recent Kiva debate.

Child sponsorship: supporting a child

Illusion: through an organization such as Save the Children, your money supports a specific child.
Reality: as Save the Children now discloses, “Your sponsorship contributions are not given directly to a child. Instead, your contributions are pooled with those of other sponsors to provide community-based programming for all eligible children in the area.” See this David Roodman post (starting with “The Kiva Story”) for the interesting, scandal-ridden history of this practice.

Heifer International: giving livestock

Illusion: your donation pays for a cow for a specific developing-world family, helping it earn a better living.
Reality: as the fine print says, “Gifts made through this catalog represent a gift to the entire mission.” The entire mission generally includes a lot beyond livestock, including difficult projects like rural extension services.

Other donor illusions are more subtle. More on this in a future post.