It seems particularly hard to find information about the past impact of “gifts of livestock” programs (such as those promoted by Heifer International). I’ve been thinking about such programs conceptually, though, and I have a lot of trouble understanding the reasoning behind these programs. Two key points:

• It seems like giving out livestock brings with it all of the problems and challenges of giving out cash.
• It seems like giving out livestock also brings additional problems and challenges that don’t apply to giving out cash.

Giving out livestock brings the same problems and challenges as giving out cash

We’ve written before about the idea of cash transfers. One potential problem with giving out cash is that the more powerful people in a community may end up dominating (monopolizing?) the benefits. Giving out truly valuable gifts could interfere with power dynamics, incite jealousy, and fail to reach those that donors actually intend to help.

It seems to me that all of these concerns apply in full force to gifts of livestock. I doubt there are many people in the world who would turn down a free cow (I’m not sure I would). Even if one has no ability to take care of the cow, there is always the option of selling it (if one has access to markets) or simply slaughtering it and eating or selling the meat.

I couldn’t feel confident in a charity giving out livestock unless I saw compelling evidence that they were getting livestock to people in need, and not just to anyone (or just to the most powerful people) interested in free livestock.

Giving out livestock brings other problems and challenges as well

• Are the livestock in good health? Will they meet recipients’ expectations, or will they die or underproduce, potentially causing people to make bad plans and investments?
• Do the recipients of livestock gifts have the ability, in terms of knowledge and resources, to take care of the livestock well? (Similar problems as in the above bullet point could arise if they don’t.)
• Do the recipients of livestock intend to take care of the livestock well? Or is there reason to be concerned that gifts of livestock could lead to cruelty to animals?
• Are there other unforeseen consequences of introducing large numbers of livestock into a community? A few years ago, there were some allegations that “over grazing by goats in arid environments has disastrous effects on the fertility of the land … these ‘gifts’ merely add to the problems of hard-pressed communities because of the drain on limited resources the animal represents.” I haven’t been able to find any facts behind these allegations and I’m not sure what they’re based on.
• Most importantly, might recipients benefit more from other gifts - and why shouldn’t they make that assessment themselves? Perhaps the story Heifer International tells is correct, and livestock would make a tremendous difference for a family. If that’s the case, then a cash gift could be expected to be spent on livestock. If one of the many concerns above applies - or livestock is not what’s most helpful for any number of reasons we simply haven’t thought of - then a cash gift will be used on something else.

  Why is it better for a charity to decide people’s needs for them? This question isn’t entirely rhetorical - there could be a good reason - but it seems that the burden of proof on a statement like “A cow is better for you than anything else you could buy with what the cow costs” should be on the charity.

Bottom line

I have trouble understanding the idea of livestock gifts, from the perspective of maximizing positive impact.

I understand that they make a good ad campaign, possibly because they draw people’s attention to the possibility of using a one-time gift to permanently escape from poverty (even though a cash gift can just as easily lead to a story like this, and could also lead to a lot of other positive stories that we simply haven’t considered).

It’s a scary thought, but it seems possible to me that these programs exist entirely because of how they can be marketed to donors, instead of for any reasons relating to maximizing good accomplished. What am I missing?

Here’s what we know about the Gates Foundation’s agriculture program:

• Gates believes it’s suggestive that “Apart from a few states and small, oil-rich countries, no country has managed a rapid rise from poverty without increasing agricultural productivity. In the poorest countries, agriculture employs a majority of the people.”
• This isn’t a new argument or an undisputed one. See Peter Timmer on Green Revolution “optimists” vs. “pessimists”.
• Gates’s approach is “comprehensive,” targets “no single, simple solution”, and includes farmer training/support, irrigation initiatives, market access initiatives, and funding of agricultural research with a focus on gender empowerment.
• This isn’t a new approach or a historically successful one. The World Bank has focused on essentially the same set of interventions recently, with unclear results, and the previous “holistic” approach of “Integrated Rural Development” is widely considered to have failed. Details at our overview of agriculture aid.

In other words, the Gates Foundation approach – as described – appears to be neither a continuation of things that have worked before nor a fundamentally new approach to the problem. So what might be different this time around?

Lots of things. Better technology could make all the difference; so could a greater degree of commitment. And one way in which the Gates Foundation could really distinguish itself from past efforts would be by doing a superior job learning about what works and what doesn’t – past initiatives have suffered from poor evaluation and very little accessible information about how things have really worked out.

The Gates Foundation’s progress reports so far are extremely preliminary, looking at “inputs” such as “number of farmers organized into groups”. We find these measures wholly appropriate given how early it is in the initiative; and yet, we’ve seen too many programs that still haven’t moved beyond these measures even after claiming “success” and asking individuals to donate and help them scale up.

If the Gates Foundation moves to more rigorous and outcome-focused evaluation over time, we might learn more about what works and what doesn’t, and the “impatient optimism” could turn out to be justified. If not, the Gates Foundation will be making a very expensive gamble with very little information about its odds.

The Gates Foundation states that “Funders have sharply cut their international aid to agricultural development over the past few decades.” It is implied that this is a major reason for the failure to see a “Green Revolution in Africa.”

We have been unable to locate support for this claim.

Using data from OECD - the most reliable source of official aid flows as far as we know - we graphed the proportion of (disbursed) aid to Africa that was classified in the “agriculture” sector since the 1970s. It’s possible that the numbers are artificially depressed for early years due to less standardized reporting, but we see no trend of the type the Gates Foundation describes.

In addition, we previously looked at funding on some of the main vehicles credited with the original “Green Revolution” and found no substantial drop since the 1970s.

It’s frustrating that the Gates Foundation doesn’t provide a source for its claim. In general, the material on its website is at a very broad level and does not make it possible for people curious about its underlying reasoning to drill deeper. That means that any criticism or examination of its work has to be either very superficial or done offline (i.e., in direct communication with the Foundation, an opportunity few people seem to get easily or often).

Added 10/30/09: here’s our source data

Imagine that a charity is able to teach a farmer some basic, useful things about farming (like “crop rotation, dip irrigation and the planting of trees that enrich over worked soil” or “disease-resistant cassava replication, distribution and sale; crop diversification; soil conservation; and expanding market opportunities”). Such simple knowledge could last the farmer forever and be far more useful - especially for the cost - than cash or loans. It’s an often-sold story, and an appealing one.

What charities don’t tell you about “improved farming techniques and technology” is just how long the aid world has been trying to spread them, and how much it has struggled. The basic challenges:

Can agriculture programs reach enough farmers? The right farmers?

A 2006 World Bank paper examines the long history of “agricultural extension” programs and is frank about their problems. For traditional programs, it states that

The cost of reaching large, geographically dispersed and remote smallholder farmers is high, particularly given high levels of illiteracy, limited access to mass media, and high transport costs. Farming systems often entail several crops, livestock, and even within given geographical area, there are variations in soil, elevation, microclimate and farmers’ capabilities and access to resources. With such a large and diversified clientele, only a small fraction of farmers can be served directly (face-to-face) by extension, and agents tend to focus on the larger, better resourced and more innovative farmers. This reduces the potential for farmer-to-farmer diffusion. (Emphasis ours)

The “Training & Visit” model attempted to address these issues through a strong, clear set of hierarchies and responsibilities (see pgs 11-14), but its substantially higher costs - coupled with the fact that, as with previous programs, impact was hard to see - led to its essentially universal abandonment (see pgs 14-15 and pgs 22-23).

When World Vision or Save the Children speaks of spreading improved practices, is it using a “T&V” style intensive-but-costly approach, or a lighter touch that could fail to reach enough (and the right) farmers? It isn’t clear.

Do charities even know what to teach and what to change?

Another general problem cited by the World Bank paper is that “Weak accountability (linked to the inability to attribute impact) is reflected in low-quality and repetitive advice given to farmers, and in diminished effort to interact with farmers, and to learn from their experience.” (Emphasis ours.) In other words, those giving advice may not actually be giving the right advice.

It is hard to find honest and thorough descriptions of how such projects have actually played out in the past, but a couple of striking failure stories should make it clear just how badly outsiders can misjudge what farmers need to learn:

• The DrumNet program in Kenya aimed, successfully, to transition farmers from growing “local crops” (i.e., crops for local/personal consumption) to growing “export crops” (i.e., crops to be sold on the export market). However, a year after the project evaluation was completed, the firm that had been buying the “export crops” stopped due to European regulations, leading to “the collapse of Drumnet as farmers were forced to undersell to middlemen, leaving sometimes a harvest of unsellable crops and thus defaulting on their loans.” (Details at this paper published on the Poverty Action Lab site (PDF).)

• A development program in Lesotho aimed to help local people with crop and livestock management, as well as building roads so they could access markets. However, few of the people in the region were farmers, and conditions were not good for farming. Harsh weather destroyed pilot crop projects, and the roads allowed in competitors who drove the existing local farmers out of business. (From pgs 193-4 of White Man’s Burden)

These aren’t cases of minor missteps - they’re cases where those giving aid did not perceive essential and fundamental aspects of the local economy. That doesn’t mean they were incompetent - it means that understanding a local economy well enough to give truly useful advice may not be easy.

The long and murky history of agricultural assistance

Agricultural programs in Africa have struggled to produce tangible results, both at the micro level (little evidence about how programs have gone) and at the macro level (disappointing progress in Africa-wide crop yields over time).

A variety of approaches have been tried, including the “holistic” approach of simultaneously addressing health, transportation, credit, and agricultural knowledge. This approach was referred to as “Integrated Rural Development” in the 1970s and 1980s and appears to be acknowledged as a failure, although the basic idea behind it may be making a comeback in the “holistic” approach of the Millennium Villages Project and other large charities.

Details at our writeup on agriculture-focused aid.

Bottom line for donors: agricultural technology is not like medicine

Agriculture aid is often presented as a matter of extending the reach of proven technologies and methods. However, the track record of such programs is simply nothing like that of health programs, which often have track records including multiple highly rigorous studies and large-scale, demonstrable successes.

We feel that the burden of proof on agriculture programs is high, but outcomes tracking of any kind is extremely rare. The evaluations that are available tend to raise many concerns about whether results are “cherry-picked” and whether results even point to improved lives.

We recommend that donors be extremely wary of charities working heavily in this area, no matter how good their intentions. We have not identified any that we can have confidence in.

What’s wrong with this “evidence of impact” for high-profile charities?

• The Millennium Villages project’s main evidence of impact so far appears to be increasing crop yields by a factor of 4-15x in 3 villages. (H/T to the recent coverage of Millennium Villages on Aid Watch)
• A KickStart evaluation (available on our website (DOC)) reports that “Family/household incomes of 3,000 farmers increased by 100% within 24 months following acquisition of micro-irrigation pumps.”
• The Carter Center’s agriculture program “provides the tools and means to help farmers in sub-Saharan Africa increase their crop yields through agricultural development, sometimes two- or even threefold.”

Among other possible problems, two major issues jump out:

1. No context on what “normal” variation in incomes looks like for poor farmers. Some years have more favorable weather - and local economic situations - than others. Enough that one year’s income or crop yield could be double another’s? 4x? 20x?

Unfortunately, one of the better pieces of “evidence” that jumps to mind is a 75-year-old novel, The Good Earth, whose farmer protagonist is comfortable one year and has literally zero income the next, for no other reason than the weather. If a given year’s yield were close enough to zero, the next year could be a huge increase (2x, 4x, 20x or more) simply by returning to normal.

I have seen little information on the local year-to-year volatility that poor farmers can experience, but I imagine that it (a) varies greatly from region to region and (b) could easily involve incomes falling and jumping by enormous amounts.

None of the above reports provide any context on this question, beyond qualitative statements about how favorable the rains were in each year examined. None of them employ any sort of “comparison group” of farmers (aside from one vague reference to “farms not using improved seeds and fertilizers” in the Malawi Millennium Village). Ultimately, none accomplish one of the most basic goals of an evaluation: giving a sense of how likely the “gains” they describe are to have arisen by pure chance.

With larger sample sizes, we might be able to use country-level volatility for context. But that brings me to the next problem.

2. We have no assurance that the described gains are representative, as opposed to “cherry-picked.”

All of the above organizations have reputations for consistent and thorough monitoring and evaluation, yet in all cases, we find ourselves looking for “impact” from a tiny subset of their projects.

• The Millennium Villages evaluation “includes results from the Sauri baseline survey, annual reports from Sauri and Koraro and crop-harvest data from Malawi.” How were these three chosen from among the 78 villages mentioned in the abstract?* The report doesn’t say.
• KickStart implies that it is constantly and consistently tracking impact, yet the report linked above discusses a single project over a single year.
• The Carter Center is somewhat explicit that its number is a “high-end” one (”sometimes two- or even threefold”). Examining the most recent annual report for its agriculture program shows a very small upward trend, if any, in crop yields across the countries in which it is working.

Some ways to produce more compelling evidence of impact

1. Be clear about what is being measured and what is being published, and when. It seems to us that in this area, charity evaluation lags far behind clinical trials, which are constantly registered before they are complete so people can track their progress. (The Poverty Action Lab is similarly transparent with its own ongoing projects.)
2. More sample size; more context; use of comparison groups. Discussed above.
3. Look for more sustained improvements in people’s lives. One measure I find superior to straight “income” or “crop yields” is asset accumulation. A jump in income could be temporary; if someone upgrades their roof or sanitation, it’s likely that at least they expect the gain to be a real and lasting one. The Village Enterprise Fund’s evaluation is one of the better charity evaluations I’ve seen in the area of economic empowerment, partly because it focuses on standard of living rather than a simple measure of income.

*It’s possible that the yields mentioned are for “clusters” of villages rather than individual villages; there are only 12 clusters. However, the source documents available for Sauri and Koraro appear to be at the village rather than the cluster level, and the details of how the measurements were made are unclear.

In his Annual Letter, Bill Gates describes the “Green Revolution”:

Almost every country that has become wealthy started with a huge increase in farming productivity. Chart 4 shows the increase in output per acre for various grains, including wheat, corn, and rice, in the United States, India, China, and Africa since 1961. This dramatic increase in output—more than three times—is often called the Green Revolution.

The Green Revolution, and the Rockefeller Foundation’s role in the research that enabled it, is frequently cited as one of philanthropy’s great success stories(1) and a huge contributor to enormous reductions in poverty.(2) Yet as Gates continues, “Africa jumps out as the only case where this [revolution] has not taken place.” Why?

To Gates, the answer appears to come down to insufficient investment:

African countries have widely varying climate conditions, and there hasn’t been the same investment in creating the seeds that fit those conditions. Because agriculture is an essential part of economic growth for most African countries, we are working with others to fund a “Green Revolution for Africa” and other areas that could benefit from this kind of investment.

Gates gives the impression that bringing the Green Revolution in Africa is mostly a matter of repeating what’s worked elsewhere, which would make it an excellent fit for our priorities. But from the rough analysis we’ve done, it appears that there has been at least as much effort to bring about a Green Revolution in Africa as elsewhere, and the obstacles in this area are specific and significant.

CGIAR funding

The Consultative Group on International Agricultural Research appears to have been the main vehicle for philanthropic funding of relevant research.(3) Using data from its website, we put together the chart below showing how much has been spent at its centers in (sub-Saharan) Africa as opposed to its other centers. The proportion was consistently been between 25% and 30% from 1972-2003:(4)

For context, sub-Saharan Africa accounted for 20-30% of the world’s extremely poor in 2003, up from 14-19% in 1990.(5) It certainly seems from this rough cut that funding for relevant research in Africa was in line with funding for relevant research in the rest of the world.

Norman Borlaug

Norman Borlaug is often credited with (and won the 1970 Nobel Peace Prize for) a leading role in the research that made the Green Revolution possible.(6) The transcript of a 2006 Center for Global Development event implies the following about his relative efforts in different areas:

• He started working in Mexico in 1944 (pg 4); “By the late 1950s the cooperative program had made such a contribution to Mexico’s food production that … Borlaug had succeeded in working himself out of a job” (page 3).
• He entered India in 1967, and within 10 years India had gone from threats of famine to self-sufficiency (page 3; see also the account of the Green Revolution given by the Library of Congress’s Country Studies/Area Handbook Series).
• He has been working on bringing similar benefits to Africa since 1985 (pages 6-7) - far longer than he worked in either Mexico or India.

Comparable efforts; disappointing results

There are many factors that may make a Green Revolution in Africa difficult or impossible to bring about, including:(7)

• Agricultural prices have fallen drastically (in real terms) since the original Green Revolution. The benefits to increased crop production may therefore not be as great.
• Africa’s environment, with its high disease burden and difficult climate, presents special - and possibly greater - challenges compared to other environments.
• Much of Africa has low population density and extremely weak infrastructure (including railroads, irrigation and electricity).
• African governments are not providing the sorts of subsidies that Asian governments used to encourage agricultural output.

None of this means that bringing the Green Revolution to Africa is necessarily impossible, or that aiming funding at this goal is necessarily futile. But it’s important to recognize that this goal is a formidable challenge for which no strong precedent exists.

The prospect of an African Green Revolution is extremely appealing, but we don’t feel that this sort of investment can ultimately be counted as “proven and scalable,” and we don’t feel it’s as well-suited for individual donors as many health interventions (which are both proven and likely repeatable, and have simply not been funded enough to reach full coverage - more on this in a future post).