These charts from Smile Train imply an appealing story:

(a) Smile Train performs surgeries for $250 apiece.
(b) Smile Train’s main use of donations is to fund $250 surgeries.
(c) A donation to Smile Train funds more surgeries than would the same donation to another cleft palate organization.
(d) If Smile Train had much more money, it would fund many more $250 surgeries.

But after some basic investigation of their website and financials (and conversation with the organization), we believe that:

(a) Smile Train’s total expense per surgery is well over $250.
(b) Directly funding surgeries (the program focused on in the charts above) accounts for under half of Smile Train’s expenses.
(c) Smile Train makes direct grants to other cleft palate organizations, some of which use the same model as the “mission groups” that its charts imply are overly costly.
(d) If Smile Train had more money, it could mean more grants to “mission groups,” general education about cleft palate conditions, and a host of other activities, and would probably not mean more directly funded surgeries.

We feel that, as a result, many of Smile Train’s donors - including Prof. Steven Levitt of Freakonomics and Superfreakonomics - have a highly inaccurate picture of how their donations are used. Details follow.

(a) Smile Train’s total expense per surgery is well over $250.

The same Smile Train page linked above claims 90,000-95,000 surgeries provided in 2008. If this number is entirely correct (not exaggerated or estimated) and we divide it into the roughly $100 million of “money raised” shown above, we obtain a cost-per surgery of over $1000.

In fact, the situation is a bit more complex, because money raised doesn’t equal money spent. To get the latter, we can look at Smile Train’s most recent IRS Form 990, which puts total expenses around $85 million for an implied cost-per-surgery closer to $900.

It’s possible that some surgeries are performed for $250 apiece or less. But are these the surgeries you’re paying for when you donate? That’s where the next few points become relevant.

(b) Directly funding surgeries (the program focused on in the charts above) accounts for under half of Smile Train’s expenses.

Here’s the big picture on Smile Train’s expenses, as far as we can understand them from the most recent IRS Form 990:

“Treatment partnerships” appears to be the core program of funding developing-world doctors to perform surgeries.

The “No information available on website” item refers to $23.6 million spent on “provid[ing] materials on cleft lip and palate for free to anyone interested in this birth defect” (according to the Form 990). We have no other information about this activity.

“Education grants” appear to refer to training developing-world surgeons. “Treatment grants” appear to correspond to grants to other organizations.

(c) Smile Train makes direct grants to other cleft palate organizations, some using the same model as the “mission groups” that its charts imply are overly costly.

Prof. Steven Levitt of Freakonomics and Superfreakonomics writes,

The typical model for cleft repair in developing countries has been to convince U.S. doctors and nurses to volunteer a week’s time, fly to a country, and do 80 surgeries … Smile Train instead partners with and trains local doctors to do the surgeries, which turns out to be far more efficient.

In fact, Smile Train makes grants to a variety of other organizations, including some that focus on the “mission” model (flying U.S. doctors and nurses overseas). The most recent list of grants starts on page 18 of the Form 990, and includes:

• $1.4 million to GiveWell standout Interplast, which funds both missions and work with local doctors.
• $70,000 to Surgical Volunteers International and $25,000 to the Smiles International Foundation, both of which appear to be missions only.
• A wide variety of other grants, including for organizations focused on research, organizations that work primarily in the U.S., and more.

(d) If Smile Train had more money, it could mean more grants to “mission groups,” general education about cleft palate conditions, and a host of other activities, and would probably not mean more directly funded surgeries.

I first encountered some of these oddities back in 2006, and wondered why Smile Train wasn’t simply putting all available funds toward its most cost-effective surgeries (presumably, the ones that cost $250 each). The answer I got at that time was that there was a “doctor shortage” - Smile Train was paying $250 per surgery as much as it could, but had more money than it could use that way, so it engaged in other activities and made grants to other organizations.

Given this situation, I think Smile Train’s approach makes sense. How many charities are raising more cash than they can productively use, and sitting on it instead of giving it away to other organizations?

In terms of spending money it has appropriately, Smile Train may be doing well. But how it brings in that money is another question. When Steve Levitt doesn’t seem aware of how funds are being used, it seems to me that donors are getting the wrong impression.

Prof. Levitt notes that “One thing they don’t do, but maybe they should do, is literally link each $250 donation to a particular child and send before and after pictures.” Smile Train can’t take this suggestion, because it’s not providing a surgery for every $250 raised.

A single-parent family of three in New York, making $8000 per year, makes under half the income level of the Federal poverty line and qualifies for food stamps, TANF (direct cash benefits) and Medicaid. (Details at our guide to U.S. public assistance)

And yet, at $2,667 per person per year, this family is wealthier than 70% of the people in the world. (See the Global Rich List calculator as well as the Giving What We Can version, which may be using more up-to-date data.)

In the poorest parts of the world, fewer than half have access to a latrine or toilet; only 17% own a television; and 19-45% lack access to a reliable source of clean water. In the U.S., practically everyone has all three. (Details.) As we wrote yesterday, the two areas have completely different concepts of “hunger.” And finally, while anyone in the U.S. can ultimately be served in an emergency room, people across the world die or suffer from health conditions for which proven solutions exist.

In the U.S., helping the less fortunate usually means tackling a thorny “equality of opportunity” problem such as improving education or helping struggling adults to find and retain jobs. These problems have a long history of failure and few proven approaches to them. By contrast, helping people overseas can mean delivering something as proven and life-changing as a bednet or tuberculosis treatment.

Hopefully, these observations give some context on why a charitable dollar goes so much further overseas. If you are looking to use your wealth to help those less fortunate, we believe it’s hard to argue the case for U.S. as opposed to international charity, unless you believe that American lives are orders of magnitude more valuable.

There has been a fair amount of buzz lately (examples here, here, here, here) about “food insecurity” in the U.S. According to the Reuters headline, one in seven Americans is short of food. In looking into the data, what has surprised us is how different the meaning of “hunger” is when we’re talking about the U.S. vs. the developing world.

Developing-world hunger: 30% of children underweight

Developing-world hunger is usually discussed in terms of incredibly severe indicators of hunger. For example, a 2008 Lancet study estimates that

• 32% of under-5 children in developing countries are “stunted” (height-for-age severely below normal, such that only 2.3% of children should be “stunted” in a normal distribution)
  
• 20% are “underweight” (weight-for-age severely below normal, such that only 2.3% of children should be “underweight” in a normal distribution)
• 3.5% are “wasted” (weight-for-age even more severely below normal, such that only 0.13% of children should be “wasted” in a normal distribution).

These are indicators of severe, long-term consequences of constant undernutrition for young children.

Food insecurity in the U.S. means anything other than complete and constant food security

The recent USDA release that has formed the basis of the recent discussion of U.S. hunger states:

Eighty-five percent of American households were food secure throughout the entire year in 2008, meaning that they had access at all times to enough food for an active, healthy life for all household members. The remaining households (14.6 percent) were food insecure at least some time during the year, including 5.7 percent with very low food security—meaning that the food intake of one or more household members was reduced and their eating patterns were disrupted at times during the year because the household lacked money and other resources for food. (Emphasis mine)

The report’s summary specifies that food insecurity is usually temporary as opposed to chronic (pg 9) and that children are usually protected from food insecurity even in food-insecure households (pg 6-7).

The “food insecurity” categories are derived from people’s answers to questions like “We worried about whether our food would run out before we got money to buy more” and “We couldn’t afford to eat balanced meals” (full list on pg 3). The details of the answers are found on page 45:

These data imply that anything approaching the sort of hunger measured in the developing world is practically nonexistent in the U.S.

Note in particular the difference regarding children. In the developing world, as shown above, severe child hunger is rampant. In the U.S., even in “food insecure” families, it’s extraordinarily rare for children to go hungry even temporarily. And indeed, World Bank data estimates that 1.3% of U.S. children under 5 are “underweight” - less than the 2.3% that would be expected in a fully normal distribution.

Also note that the USDA report estimates costs for different levels of “food plans” (pg 55), and that its “Thrifty” plan - the cheapest - ranges from $21-$40 per week ($3-$6 per day) depending on age. In the developing world, meanwhile, over 2.5 billion people are estimated to live on less than US$2.50 a day total.

Bottom line

We have no intention of trivializing the situation of those in poverty in the U.S. But for a donor making choices, it can be stunning to see what a different meaning “hunger” takes on when applied at home vs. abroad. Do you value the lives of Americans so much more that you’d rather help people with the second kind of hunger than people with the first?

Seth Godin makes an appealing case for “social enterprise investment” along the lines of The Acumen Fund:

When two people trade, both win. No one buys a bar a soap unless the money they’re spending for the soap is worth less to them than the soap itself.

When someone in poverty buys a device that improves productivity, the device pays for itself (if it didn’t, they wouldn’t buy it.) So a drip irrigation system, for example, may pay off by creating two or three harvests a year instead of one.

…

How does Acumen Fund create these markets? The answer is patient capital. The companies that are selling solar lamps to replace kerosene or water purification systems in tiny villages, or housing projects for peasants in Pakistan or even ambulance services in Mumbai fully intend to make a profit, but the venture capitalists on Sand Hill Road aren’t in a hurry to invest in them. The investments are a little too risky, take a little too long or a little too unproven to take a chance on.

So Acumen finds these entrepreneurs on site in the developing world, funds them, teaches them and pushes them to build really big organizations. A to Z has literally thousands of people in their modern factory creating malaria bed nets in Tanzania. And so it grows.

There are reasonable arguments that this sort of larger-scale financing is far more promising than microfinance, which typically focuses on tiny microenterprises that are unlikely to achieve large scale and create jobs. If we could find a strong charity in this area, we’d be excited to recommend it. But as with other areas, we try to go beyond the story and ask critical questions, and so far we have found no organization - including the Acumen Fund - that appears to answer them.

Our main question in this area pertains to an investor’s track record. We’d expect large-scale investing to have a success rate well under 50%, but we’d want to see at least an example or two of real success in the past, consisting of any of the following:

• An investment that has been shown to improve the lives of customers (for example, tracking bednet use in areas served by A to Z).
• An investment that has been shown to create jobs, i.e., employ people with previously low incomes.
• A financial return. (Note that it isn’t enough to point to a “portfolio” member and observe that it’s a successful, self-sustaining business. Anyone can make a loan or grant to an already-successful business; this doesn’t mean they’re having an impact.)

Given Acumen Fund’s reputation for (a) general excellence and (b) focus on evaluation (see the Acumen Fund discussing these issues here and here), we would have expected them to produce such examples. But after evaluating them for our economic empowerment grant, we have concluded that:

• They do not directly track social impact of the organizations they invest in.
• They do not have information on the effects of their investments on jobs.
• They have declined to share information on the performance of their loans, and have not yet exited any equity investments.

A couple of other concerns:

• At least one of their investments, VisionSpring, has a major charitable solicitation component (for example, they have applied for funding from us). Acumen’s loan could end up getting repaid by donations, which wouldn’t go well with the idea of “patient capital to create self-sustaining institutions.”
• Their documents are largely at too general and vague a level to give a concrete sense of how their investments are going.

Given this situation, it’s hard to see the organization as more than an experiment at this point. No matter what its team’s credentials, there is the enormous question of how developed-world business experience translates to the goal of making good and socially impactful investments in the developing world.

As with other unproven charities, it seems to us that Acumen should be primarily funded by those who are very close to the relevant people and issues, until it is able to demonstrate not just a good story but some real results. Other similar organizations seem to be similar cases.

We recently complained that “UNICEF provides no information about where the money goes and what projects are in progress.” Some might feel that this complaint comes from unrealistically high standards of transparency, especially for organizations such as UNICEF. How is an organization spending $2.7 billion a year supposed to report its activities?

Our answer would be: “like the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) does.” (Page 55 of its 2008 annual report shows that its budget size is very close to UNICEF’s at $2.7 billion.)

GFATM provides an online program search of all its activities. For any grant it has given (example), you can see (if completed) the grant proposal, grant agreement, and reports on progress. In other words, you can see how much has spent and how (and whether) progress has been tracked.

GFATM recently released the kind of document we have never seen from any other charity approaching its size: an overall evaluation of its activities and impact. Not a general discussion of the organization; not a “meta-evaluation” discussing the quality of past evaluations; a discussion of the overall impact of all of GFATM’s activities across the world. Furthermore, this report was in no way a fundraising document; it was frank about the fact that inadequate evidence exists for GFATM’s impact to date (see the discussion at our review).

GFATM comes under a lot of criticism, even from its own evaluators. We ourselves have many reservations about its work, as our review establishes. But we have seen very few charities - and no other charities approaching its size - that can make as strong a claim to being a transparent organization and a learning organization.

GFATM proves that neither size nor celebrity support need stop a charity from being clear about what it’s working on and how it’s going.

We’re looking for a good option for U.S. donors interested in supporting microfinance. We’ve been examining the largest, most prominent U.S.-registered charities in this area: Grameen Foundation, Unitus, Accion, Women’s World Banking, Opportunity International and FINCA. All of these are large organizations that list a variety of “partner” microfinance institutions.

One thing that might surprise a donor in this area is that sometimes the “partnerships” overlap. For example, Women’s World Banking lists Enda Inter-Arabe in its network, and so does Grameen-Jameel (which itself is a joint venture of the Grameen Foundation). Neither organization’s profile mentions the other.

Grameen Koota lists the Grameen Foundation and Unitus (among others) as donors and lists Unitus and Accion as consultants. Unitus’s page on Grameen Koota doesn’t mention Accion; Accion’s doesn’t mention Unitus; the Grameen Foundation sounds like its relationship is currently more consultant-like than donor-like.

As you can see from the profiles linked above - and as we’ve found from talking to the large microfinance charities - most (not all) of their activities consist of consulting or “technical assistance,” rather than founding, directly supporting or investing in microfinance institutions. The relationship between them and their “partners” often appears relatively limited, and it’s difficult to pin down who’s responsible for what.

Focusing on technical assistance and consulting isn’t necessarily a bad thing - perhaps U.S. expertise is just what these institutions need to serve more clients and serve them better. But we aren’t sure this is the case - it’s also possible that the U.S. charities have sprung up because of strong donor demand for the “story” of microfinance, and are consulting because they have more than they can productively spend on investing and creating.

What we know is that it’s inherently hard to gauge a charity’s impact when its main activities consist of free (or heavily subsidized) technical assistance. Even if we had all the information we want about the partner MFIs themselves, we’d still need to understand the extent to which help with business planning or product design made a difference to them.

So far, we haven’t seen many signs that the U.S. charities are critically asking this question, or making it possible to hold them accountable for the answer.

GiveWell spends a lot of time on the question, “Should I give to charity A or charity B?”

One of the things that has surprised us about the world of charity is how many people insist on answering, “Both” or “You can’t/shouldn’t be asking that question.” To them, all that matters is whether a charity does some good, not how much good it does or how it compares to other options.

One statement of the idea comes from none other than the Jeffrey-Sachs led WHO Commission on Macroeconomics and Health (footnote 24):

Many have asked the Commission what to do if the donor money is not made available—in essence, how to triage with less money. We are asked to prioritize millions of readily preventable deaths per year, since we have already narrowed our focus to a small number of conditions that have an enormous social burden and that have low-cost interventions that are at least partially effective. Not only is this kind of triaging ethically and politically beyond our capacity, but it is also exceedingly hard to do in sensible way. Those who hope for a simple answer, for example to focus on the cheap interventions (immunizations) while putting off the expensive interventions (higher-cost prevention programs and antiretroviral therapy needed to fight AIDS) to a later date, misjudge the practical choices we face. The AIDS pandemic will destroy African economic development unless controlled; to fight measles but not AIDS will not begin to meet Africa’s human and economic needs. It would be wrong to go to the other extreme as well, and let the legitimate need to fight AIDS end up starving the cheaper interventions, so we advocate both. Moreover, the infrastructure developed to fight AIDS will support the infrastructure needed to fight measles, especially if strengthening such complementarities is explicitly built into the AIDS control effort. It is vastly more fruitful to design and finance a comprehensive program that addresses many critical health needs than to pick and choose the apparently inexpensive items.

More examples of this mentality can be seen in recent comments to our blog by John, Yael and Steve.

Our response to these comments is simple. Most donors have a certain amount they are willing to give, and that amount is less than enough to fully fund even one of the world’s underfunded humanitarian initiatives.

We don’t believe that any problem that people suffer from should be ignored or trivialized. We do believe that individual donors must choose which to address, and that they should be factoring in questions like “What evidence is there that my money can even do anything to address this problem?” and “How cost-effectively can I create change by giving my money here?”

In their personal and business lives, people constantly make tradeoffs on a limited budget. Imagine if a salesman tried to argue, “How can you choose between this car and that one? They’re both wonderful cars and each would make the other more useful. You should buy them both!”

Yet in the world of charity, the basic and undeniable idea of triage is constantly denied.

UNICEF offers you the chance to buy measles vaccines for 100 children for $27.10. And lest you complain that you’ve heard this one before, it assures you specifically that “While other organizations allow supporters to purchase ’symbolic’ gifts, Inspired Gifts are actual items.”

Is this finally the “real personal connection” donors have been waiting for?

We can’t say for sure. Unlike Kiva, UNICEF provides no information about where the money goes and what projects are in progress. But we can ask a few critical questions:

• There are many costs for vaccination programs besides the vaccines themselves. Who is paying these costs?
• If some other party puts up the “fixed costs” for a given campaign (labor, logistics, etc.), but UNICEF’s catalogue only “sells” 1/2 the needed vaccines, will the other 1/2 of the relevant population go unvaccinated?
• Or will other sources of funds cover the remaining need, making the cost of the campaign essentially fixed? If this is the case, in what sense is the donor “buying” the vaccine?

Our guess is that UNICEF has a large pool of funding allocated to these campaigns, aside from the money that comes in through “Inspired Gifts” (which seems to be paying for very small numbers of items). We would guess that UNICEF will “officially” match up donations through the “catalogue” vehicle to, for example, vaccines while shifting its other funds from vaccines to delivery costs.

Why does all of this matter? Because UNICEF is advertising immunizations for 27 cents apiece. In reality, it almost certainly costs more than that - all things considered - to deliver a vaccination. That would make this another case in which a charity misleadingly zooms in on “your” money rather than considering all costs - a subtle, but substantive, donor illusion.

There’s no smoking gun, though, because there is no transparency.

When we argue that donors should give internationally, one of the most common questions we get is, “Sure, you may be able to save a life in Africa, but what type of life are you saving? If you save a child from malaria will s/he likely die from something else soon after? Will s/he suffer from other problems that significantly reduce his/her quality of life?”

We recently published a report on standard of living in the developing world that tries to answer that question. It looks at what facts are available from relatively broad, plausibly representative studies to answer “what is life like in a poor country?” Here’s a summary of what we learned:

• There’s a strong cross-country correlation between income and reported happiness. The Gallup Poll asked people around the world to rate their life satisfaction on a scale from 1 (the worst) to 10 (the best). People in poor countries are less satisfied with their lives (they ranked their satisfaction as 4.3 out of 10, while rich country residents ranked theirs as 6.7). (More at our review of the Gallup World Poll data.)
• A child who lives past his/her 5th birthday will likely live a full life. Child mortality is much higher in poor countries than in rich countries. But those who live past age 5 have nearly a 70% chance of living until age 60. (More at our overview of life expectancy in the developing world.)
• Most people in the developing world do not have AIDS, river blindness or other severe chronic conditions. Many of the diseases commonly associated with the developing world are fairly rare across all developing counties. Fewer than 1 out of 100 people have HIV/AIDS, 1 out of 40 have lymphatic filariasis, and about 1 out of 2,700 have river blindness. (More on the prevalence of disease in the developing world.)
• Less severe chronic conditions, such as malnutrition and parasite infections, are very common. For example, about a third of children are stunted (significantly shorter than normal due to undernutrition). We are not sure to what extent stunting and other results of a poor or inadequate diet are likely to affect a child’s standard of living over the long term. (More on non-fatal health problems.)
• Incomes are low, but discretionary spending does exist even among the poorest. People in extreme poverty (defined in the past as under US $1 a day of income) do not spend every “additional dollar” on additional food; they frequently own TVs and radios and participate in festivals. (More on what it means to live on less than $1 or $2 per day)

Bottom line

On one hand, people in the developing world have a tangibly lower quality of life. On the other hand, a life saved probably means many more years of functional life. We feel strongly that it’s worth addressing a major problem (such as tuberculosis or immunizations) even if other problems remain unaddressed.

Nathaniel Whittemore writes that it’s “time to move on” regarding the recent Kiva controversy. I disagree.

It’s true that Kiva handled the criticism admirably, and made significant changes to its website to improve clarity for donors. It’s also true that Kiva has a stronger case than many for being generally transparent and impactful. Finally, it’s true that those of us who have been blogging about Kiva are a bit tired of the subject.

But none of this changes the fact that many (I would guess the vast majority) of Kiva’s enthusiastic users don’t know how it works, and would be upset if they found out. If you doubt this, just look at the reaction to the recent New York Times story that brought this issue to the attention of people outside our corner of the blogosphere. We may be “over” this issue, but most people have still never heard of it.

The common perception of Kiva - repeated to us by supporter after supporter - is that it enables people to (a) make interest-free loans to (b) the entrepreneurs they personally select. In fact, Kiva users are effectively making gifts (since the loans are interest-free for them, but not for the borrowers) to microfinance institutions. If they knew this, they might prefer giving directly to microfinance charities instead, especially since they’d then be able to get a tax deduction and give significantly more. Or they might prefer another gift entirely - perhaps a health program that offers great impact but no “personal connection,” or perhaps a DonorsChoose project that offers “real” personal connection but arguably little impact.

The time to “move on” should not be based on Kiva’s “handling” the situation or our growing tired of it - it should be based on Kiva’s supporters, by and large, understanding how Kiva works. We think we’re nowhere near that point. We urge those who know the truth about Kiva to continue spreading the word.

Jenny Aker and Michael Clemens: “Privately and publicly, donors, MFIs and practitioners are expressing concern about the impact of [recent] studies on the future of microfinance.”

David Roodman: “I’ve been surprised by the predominant negativity of the new wave of comments from the NYT article.”

Ultimately, the idea of “true” person-to-person lending is somewhat silly, as are the stories people tell themselves about microfinance. But charities have a tendency to promise donors the sun, the moon and the stars, and donors have a tendency to buy into it.

One problem with this phenomenon is that there’s no mechanism for holding charities accountable. Another problem is that when the truth comes out, there can be whiplash from people realizing things like “My gift might just be helping people get a little more control over their lives, rather than miraculously lifting a person from poverty for every $25 loan.”

Fundraisers: one argument for educating, rather than coddling, donors is that someone is eventually going to educate them anyway. Wouldn’t you prefer that it be your organization, rather than the New York Times (or GiveWell)?

The New York Times recently profiled Chess in the Schools:

The Chess-in-the-Schools program has sought to foster analytical skills on the theory that these will help students succeed academically. The group teaches 20,000 children a year and calculates that it has taught 425,000 children since 1986. Children gather to learn the game at the group’s headquarters in Manhattan.

It seems like 20 years and 425,000 children is quite a lot of investment in the “theory that [chess] will help students succeed academically.” The Times feature provides a calming justification for the investment: “Chess helps promote intellectual growth and has been shown to improve academic performance.” Let’s look at the evidence for this claim.

The study we found

An early-1990s study looks at achievement test scores of chess-playing students over two years at District 9 in the Bronx. It observes that (a) the overall average reading score improved among chessplayers by about 5 percentile points, but didn’t improve among the set of remaining District 9 students; (b) 15 of 22 second-year participants improved their reading scores by some amount, while only 491 of 1118 non-participants in the district - and 245 of 655 non-participants with high reading scores, improved.

This study is riddled with major problems:

• The numbers the researchers choose to compare seem arbitrary and possibly cherry-picked. Why do the researchers look at the “percentage who improved” among second-year chessplayers but not for both years? Why do they compare the second-year students to “high-performing nonparticipants,” but not give the same comparison when looking at all students?
• The problem of selection bias is unusually obvious here. They’re comparing kids who volunteered to play chess against those who didn’t. Think of the chess club members at your school, and ask yourself if they would have been just like all the other kids had chess club not been offered. There’s no reason to think these two groups of kids are otherwise similar or would be expected to respond similarly to school.
• This is a study of somewhere between 22 and 53 students at a single district in the early 1990s. Even if the study were highly rigorous, it would still be a long way from “proof that chess helps promote intellectual growth.”

The studies we couldn’t find

The Chess-in-the-Schools website states:

In 1991 and 1996, Stuart M. Margulies, Ph.D., a noted educational psychologist, conducted two studies examining the effects of chess on children’s reading scores. The studies demonstrated that students who participated in the chess program showed improved scores on standardized tests. The gains were even greater among children with low or average initial scores. Children who were in the non-chess playing control group showed no gains.

Another study in 1999, measured the impact of chess on the emotional intelligence of fifth graders. The results of the study were striking. The overall success rate in handling real life situations with emotional intelligence was 91.4% for the children who participated in the Chess-in-the-Schools program. In contrast, those who were not involved with the chess program had an average overall success rate of only 64.4%.

We’re guessing that the study we’re looking at is an update of the 1991 study since it references no previous studies and discusses results from 1991 and 1992. We can’t find the other studies anywhere. Chess-in-the-Schools provides neither links nor citations.

Even in the best-case scenario, it’s apparently been at least a decade since the last test of the Chess-in-the-Schools model.

“Chess helps promote intellectual growth and has been shown to improve academic performance?”

In researching charities, one of the more discouraging things we’ve learned is how little support it takes for a statement like “Chess helps promote intellectual growth and has been shown to improve academic performance” to be repeated by charities, donors, and even the media.

As far as we can tell, Chess-in-the-Schools is not a demonstrated success story. It’s just been promoted and scaled up like one.

Jeanne Panossian left two very interesting comments on our blog discussing donor irrationality, from the point of view of someone running a small charity.

• On donor illusions: ” … It takes extraordinary ethical fortitude to openly tell people how complicated your organization is, normally a donor has made their basic decision in the first 15 to 30 seconds of a conversation …”
• On the administrative expense ratio: “…While I love to talk all day about [my charity’s impact], I have learned it is not worth my while. Waving a flag and telling people how we pay for our paper clips yields me more funds faster …”

Recommended reading.

How much of microfinance’s popularity in the world of philanthropy comes straight from this story?

I was shocked to discover a woman in the village, borrowing less than a dollar from the money-lender, on the condition that he would have the exclusive right to buy all she produces at the price he decides. This, to me, was a way of recruiting slave labor.

I decided to make a list of the victims of this money-lending “business” in the village next door to our campus.

When my list was done, it had the names of 42 victims who borrowed a total amount of US $27. I offered US $27 from my own pocket to get these victims out of the clutches of those money-lenders. The excitement that was created among the people by this small action got me further involved in it. If I could make so many people so happy with such a tiny amount of money, why not do more of it?

It’s an amazing and moving story. But it’s a story about one giver and 42 beneficiaries in one village.

In 2007, the Grameen Foundation alone saw over $16 million in donations and claimed over 7 million clients served (see its annual report (PDF)). It works in 32 countries on 4 continents. And it’s still putting that $27 front and center.

We know little about microfinance’s actual impact, and much of what we do “know” comes down to myths (myth #6, in particular, seems oddly fitted to the story of the original $27). We’ve seen very little interest in general in pushing skeptically on the appealing stories charities tell.

Dr. Yunus’s original loan was interest-free, while today’s microloans charge interest in the 30%/year range. We know that the for-profit participants in microfinance have been participating for reasons other than one great story.

I don’t feel nearly so confident about the philanthropic participants.

Early in 2009, we were extremely excited about The Carter Center. It seemed so strong that we devoted weeks to understanding it in depth.

As discussed in a blog post we made at the time, several of its programs work on extremely promising “neglected tropical disease control” activities, and there’s a truly unusual amount of disclosure from these programs. It appeared that the Carter Center is near the top of the heap both for what it’s doing and for how it’s sharing information. To boot, it was directly involved in one of the most cited global health success stories, the near-eradication of guinea worm.

The Carter Center also has several programs that don’t seem as promising. At first we nearly dismissed/overlooked these programs. But as we dug deeper, we realized that just because a charity emphasizes its best programs doesn’t mean it’s spending most of its funds on them. Oddly, the one piece of information we couldn’t seem to find anywhere on its website was how much of its budget was allocated to each program. The back-of-the-envelope calculations we did surprised us: the heavily documented river blindness program seemed as though it must be tiny, while the agriculture program hadn’t published anything since 2005 but appeared at that time to be taking up around 10% of the total budget.

We got in touch with The Carter Center and asked for a budget breakdown by program. We spoke to a senior representative and followed up with him 4 times. We even tried getting a connection of ours who has been a major Carter Center donor to ask for the information. It kept getting put off. Today we still don’t have this information.

To be honest, at this point we don’t know whether the “flagship” disease-control programs are at the core of the Carter Center’s work or act as more of a “hook” for donors while it focuses on things like fellowships for mental health journalism. And we have no sense of what a donor accomplishes by giving them a small gift (a gift that, however it’s officially designated, is likely effectively going to fund what the Carter Center wants it to fund due to the issue of fungibility).

To give a sense of the variety of program type and quality, here’s where we stand on a few select programs:

• Guinea worm eradication: as mentioned above, progress to date has been impressive. But the case for further funding is murky - the Carter Center proposes $10-15 million per year to fight the last vestiges of a disease that now affects very few people.
• Lymphatic filariasis control program: cheaply treating a horrible and neglected disease, this program is our favorite of the bunch, but we have no idea how large it is or whether more donations would result in expansion of it.
• Agriculture program: appears to be trying to bring the Green Revolution to Africa, but the last update was in 2005 and did not show encouraging results. The program’s “Staff” page appears only to list Dr. Norman Borlaug, whose accomplishments are truly astounding but who passed away earlier this year.
• Ethiopian Public Health Training Initiative: we’re very much in favor of training more public health professionals if it can be done capably. But this program is very thinly documented, in contrast to the direct disease control programs. Are participants in the program being assessed afterward to see if they’ve improved their skills? Are they contributing to Ethiopia’s health care system?

We wish the Carter Center were as transparent about its budget as it is about (some of) its program activities.