The GiveWell Blog

New content: New research report, new ratings system, DIY Guide, and self-evaluation

(updated on: ) | by Holden

We’ve been working on a lot of changes and additions to the main GiveWell website and at this point have gone live with all of the following:

New research and recommendations on U.S. education and early childhood care. We did some work on these causes in our first year, but we’ve now gone back with our improved and more thorough research methodology.

Do-It-Yourself Charity Evaluation. If you’re looking to do your own due diligence, examining a charity or cause we haven’t covered, check out our DIY guide for suggested critical questions you can ask. We’ve covered every charitable cause we can find or think of.

This is the first donor guide we’re aware of that asks different questions for charities in different causes, rather than taking a “one-size-fits-all” approach. While it means some extra overhead in classifying your charity, it also means we’ve been able to make the questions much more concrete, specific and meaningful.

This is very much “beta” content, with room for improvement in the questions themselves and in the guide’s usability, and we are very interested in your feedback. (Note that Tactical Philanthropy and Chronicle of Philanthropy Online have already discussed this new feature.)

New ratings system. We wrote back in October that we were preparing to scrap out quantitative ratings system for charities. We have now done so. We still give marks of distinction to our top-rated charities, and we still discuss the charities that don’t get these marks of distinction, but we no longer have a “zero-to-three-star rating” for each charity. See the new details of our ratings system.

Charity downgrades. We’ve completed our annual review of our recommended charities, in which we request updated performance information, sometimes request more detailed information than we did in the first place, and rethink our evaluation in general. Three of these charities have received downgrades:

Part of what’s going on here is a gradual raising of our bar, as we have capacity for more analysis and a substantially higher opinion of our top-rated charity VillageReach than we had last year. (In addition to a site visit that involved a lot of time spent discussing and improving our understanding of the organization, we also have updated, concrete, encouraging information on the group’s expansion plans / room for more funding.)

New external evaluations of GiveWell’s work. We have begun accumulating formal reviews of our research from non-staff members; some are scholars in relevant fields while others are simply willing volunteers. We have published the reviews we have so far and intend to add more at a faster pace from here, now that we’ve nailed down the basic ground rules of how to do external reviews of different parts of our work. Note that we have also begun collecting endorsements as a quick sign of our credibility for website visitors.

What’s next: within the next few months we will be publishing

  • Extensive notes from our several months in India, including site visits to over 15 organizations.
  • Research and recommendations on the general cause of disaster relief/reconstruction.
  • Our annual self-evaluation.

After “extraordinary and unorthodox” comes the valley of death

(updated on: ) | by Holden

An anonymous donor got some buzz a few weeks ago with its call on Innocentive for “extraordinary and unorthodox” philanthropic opportunities. It seeks a project that “holds the potential for a transformational impact,” “is unlikely to attract funding elsewhere due to its risky, unorthodox, and/or neglected profile,” and will be able to “attract additional capital from other sources” after the initial “catalytic” funding.

Perhaps ironically, this RFP sounded very familiar to me. In fact, it’s hard for me to see a big difference between it and the $100 million Gates Grand Challenges Explorations, “a unique initiative that supports innovative research of unorthodox ideas” in global health (though the Innocentive proposal above does not explicitly specify a sector, all three of its examples are in global health as well).

Speaking more informally, I’ve heard similar concepts emphasized by most major funders I’ve spoken with. Anyone who has dealt with major foundations should recognize the desire to find a completely new, revolutionary, neglected opportunity that just needs some seed funding to explode.

I do believe that the best opportunities are the under-funded ones. Yet I’m not sure that tiny, neglected innovations are the best places to look for these opportunities – precisely because that’s where all the major funders seem to be looking. I submit that the better place to look for neglected opportunities is the “valley of death” between proof of concept and large-scale rollout.

“Valley of death” is a term from both the investment and research communities. In both cases, broadly speaking, it refers to the difficulty of getting funding between the earliest stages of a project and the latest stages of a project, where investors have neither a sure thing nor a “lottery ticket” for the glory of being “first.”

I believe that the world of philanthropy suffers from its own “valley of death.”

Programs that have enough traction often aim for funding by the government (for example, the Nurse-Family Partnership program has had some success with this). Programs in their earliest stages appeal to foundations’ hunger for bragging rights. But what about VillageReach, which had a single successful “proof of concept” pilot project (initially funded by the Gates and Skoll foundations and the World Bank Development Award) and now needs transitional funding to roll out its model across Mozambique? Large foundations have, to date, declined to fund the rollout, in some cases specifically citing their preference for establishing models rather than rolling them out.

There’s no glory in funding the VillageReach rollout. VillageReach already has shown that what it’s doing has worked; nobody can claim to be brilliant for spotting it. And VillageReach doesn’t need help designing its program (this has been cited to me explicitly as a drawback from the perspective of some major funders).

But if it doesn’t get funded, it isn’t going to happen. If it does, it could change the world. It is, in fact, a fantastic opportunity.

If you asked me which funders are going to have the greatest impact over the next decade or so, I’d bet on the funders extraordinary and unorthodox enough to forget about being “extraordinary and unorthodox” – and instead put in a bid for the Smart Money Award.

Health system strengthening + sustainability + accountability

(updated on: ) | by Holden

We’ve written a lot about our top-rated charity, VillageReach (in particular, see our official review and our 2009 blog post). Consistent with our focus on individual/casual donors, we generally emphasize effectiveness over ambition, and so we have mostly focused on VillageReach’s fit with our criteria: its strong case for effectveness, cost-effectiveness, and room for more funding.

However, VillageReach has some other major strengths as well – strengths that make it a good fit not just for individual donors looking to help people cost-effectively, but also for more ambitious donors hoping to contribute to transformative change.

Health system strengthening – with tangible results

One of the ongoing debates/discussions over health aid involves “vertical” vs. “horizontal” interventions. I recommend William Easterly’s Can The West Save Africa? (start on page 57) for an overview of the issue, but briefly:

  • “Vertical” programs focus on global large-scale campaigns addressing a particular health condition, often focusing on the supply of appropriate medical materials (for example, vaccines and antiretroviral drugs for treating HIV/AIDS) while “horizontal” aid tries to strengthen the health system in general in a particular region.
  • “Vertical” programs have had some major demonstrable successes (for example, most of the cases in Millions Saved). However, many believe that once the major opportunities have been capitalized on, these sorts of initiatives hit diminishing returns, as medical supplies outstrip the health system’s ability to deliver them. Over-funding a particular campaign may even get in the way and undermine other health efforts.
  • “Horizontal” programs have the worthy goal of making the health system stronger as a whole, including reaching those who are hardest to reach. However, perhaps because these programs often involve more complex goals and more leeway for the funded, these programs do not have the same track record of demonstrable success, and arguably have inferior mechanisms for accountability (i.e., making sure that money is spent well and produces actual results). See this Aid Watch post for more on these problems.

VillageReach’s program, focused on health system logistics, is fundamentally more “horizontal” than “vertical” – it’s focused on improving health system capacity. For example, one of the major changes its pilot project brought about was creating a specialized team for delivery of vaccines, which relieved local health system personnel of the responsibility to collect their own vaccines. The aim was not only to get supplies out to the villages in a more reliable way, but to reduce demands and constraints on the many local health system personnel. In the future it may expand this model to other health supplies. It’s not just using donations to purchase supplies – it’s reorganizing the system to improve overall efficiency and even reduce total costs.

Yet unlike many “health system strengthening” initiatives, VillageReach’s program holds itself accountable with specific deliverables, promising – for example – a drastic drop in the frequency with which health centers run out of vaccines. It has run surveys of vaccination rates, comparing the province it worked in to a nearby province. Looking at vaccination rates cannot capture all of the benefits VillageReach aims to provide, but it provides a strong reality check on the model and suggestive evidence for whether it is indeed improving efficiency.

The program thus addresses one of the thornier, and difficult-to-evaluate, areas of global health, while maintaining strong accountability and demonstrable results.

Ambitions of transformation and sustainability – with tangible results in the meantime

Two other constant tensions we perceive in health initiatives are between (a) short- and long-term impact; (b) tangible and transformative impact. In neither case are the two goals clearly contradictory, yet in practice there often seem to be tradeoffs.

For our part, we have focused on short-term, tangible impact. As we lay out in our process overview, we are focused on serving individual/casual donors, and we feel these donors should look first and foremost for accountability and results. A chance at “hitting the jackpot” (creating a permanent and/or far-reaching change) ought to be considered a bonus, not a requirement, especially when the short-term tangible impacts are as good as saving a life for every $1,000 (or less) spent.

Yet the best charity we’ve found for these short-term, tangible impacts turns out to be very ambitious, with very high upside should they succeed. As detailed in our review, VillageReach isn’t just implementing a new health system – it’s advocating for this system as a new approach to health logistics all across the country of Mozambique. It has produced cost analysis concluding that the government will save money, long-term, by adopting its model; with this analysis and its superior results in hand, it has won the endorsement of the national Ministry of Health. It appears to have as good a chance at a lasting impact on how the government operates as any other charity we’ve seen.

Its impact need not even end within Mozambique. Getting supplies to hard-to-reach areas is a challenge that applies throughout the world of global health (and indeed VillageReach has been retained as a contractor in Senegal, Malawi and India).

If you’re hoping your donation has a chance to be a part of a new innovation with global impact, VillageReach gives you that chance. Of course, many other charities have equal or grander ambitions. The difference is that VillageReach is (from what we’ve seen) holding itself far more accountable, producing tangible – and cost-effective – impact as it goes.

What’s not to like?

Our process has been accused of over-focusing on tangible impact and evaluation, and thus favoring larger, more conservative organizations. This accusation has some truth to it (and we accept this bias in view of what kind of donor we’re serving). And yet, the organization we’ve found that performs best on our criteria is in fact relatively small, and highly innovative. Indeed, of all the organizations out there with big ambitions, we believe VillageReach – with its high accountability and its focus on producing results, not just hopes, in the meantime – to be the best bet to actually deliver.

Microfinance: The multi-billion-dollar aid sector that we’re just starting to learn about

(updated on: ) | by Holden

GiveWell attended the Microfinance Innovation & Impact Conference via Board member Tim Ogden and part-time employee Stephanie Wykstra. (Our full-time staff could not attend as we are all in India). Our main takeaways:

  • The people involved with this conference, and the nonprofits working with them, are producing a lot of promising, interesting, rigorous and informative work on how microfinance affects the poor and how these effects might be improved.
  • While microlending is the form of microfinance that has been most funded, celebrated and scaled up over the decades, it appears to be the least proven and least-well understood in terms of its impact on the poor. In many ways it seems that good information on how (and to what extent) it’s helpful is just starting to become available.

Content presented at the conference
There is much abstract agreement about the importance of learning from good research, but we feel that there are few concrete examples of groups actually doing it. A high concentration of those examples could be found at this conference. We have written before about how impressed we are with the work of the Poverty Action Lab and Innovations for Poverty Action (groups behind this conference), and that continues to be the case.

The research has already been summarized in a large number of posts by others (see the roundup by GiveWell Board member Tim Ogden). Our quick take:

Microsavings, microinsurance, and “ultrapoor programs” are clearly promising. High-quality studies found significant improvements on metrics like income, food security, and children’s ability to attend school when these programs – which differ markedly from the loans most people associate with “microfinance” – were rolled out. Programs included accounts intended to help people pre-commit to saving (more); intensive promotion of difficult-to-sell insurance products (more), and “ultra-poor programs” that target people with extremely low incomes and offer direct benefits such as cash/asset transfers and education, rather than financial products (more).

Unfortunately, we know of no clear vehicles through which individual donors can fund these sorts of programs. We’ve written before about the frustrating situation of seeing a good program without a good funding vehicle, and called for major funders to consider this problem. There do appear to be similarities between the “ultrapoor program” and the work of GiveWell-recommended charity Village Enterprise Fund.

The impact of microlending is, at this point, not very well understood. A new high-quality study in Morocco found no positive social impact overall, though it did find that people with existing agricultural businesses appeared to invest more/differently in their businesses (more). These results were similar to those observed in the only previous high-quality study of traditional microlending, which took place in a very different environment (Hyderabad, India) and was summarized last year by David Roodman.

There was significant discussion of the risks that people might be overborrowing (something we are quite concerned about, and something that there is apparently still little information to assess). And new data from the Philippines reinforced a point we have emphasized in the past – that microloans are more likely to finance consumption, and less likely to finance entrepreneurial investment, than most donors suppose. From Philanthropy Action’s summary:

Around 46 percent of clients used new debt to pay down old debt, and 28 percent used funds for a single large household purchase. In all, 39 percent of the money goes unaccounted for, 15 percent is used to pay down other loans, 9 percent goes into the household and 37 percent is invested in the business. Curiously, Karlan’s study got at these results by asking clients the same questions in a number of different ways. Not surprisingly, when the banks ask clients why they want the funds only 2 percent of clients admit potential uses other than the business, even after the funds are given. Only when an independent surveyor asks indirectly do any relevant percentage of people admit to non-entrepreneurial uses for credit.

Studies are finding interesting potential ways to improve the impact of microlending. One possibility is better targeting (as discussed above, microlending appears to affect different people differently). Two other possibilities raised at the conference, both with encouraging preliminary evidence behind them, were the implementation of grace periods for repayment (more) and financial education focusing on “rules of thumb” rather than formal accounting (more).

Learning how microlending works – better late than never
It appears to us that an enormous amount of philanthropic capital has gone into microfinance. Last year we looked at data from 2008 and found that $11.7 billion had gone to microfinance funding of some kind, with $2.2 billion being grants. For context, total official aid earmarked for health was about $3 billion disbursed / $8 billion committed in the same year (data from Aid Data).

My greatest fear about microfinance is that all (or nearly all) of that money is chasing a good story rather than a good program.

In this context, it’s hard for us to see how one can argue that good evaluation is too expensive. The total budget of Innovations for Poverty Action that year was just under $6 million (data from NCCS).

The process of giving: A personal story (guest post from Ian Turner)

(updated on: ) | by Guest

This is a guest post from Ian Turner about how he decided what charity to support for his most recent donation. Ian and GiveWell staff had several in-depth conversations as he worked through his decision, so we invited him to share his thought process here. Note that GiveWell has made minor editing suggestions for this post (though Ian determined the final content).

Every person who gives to charity does so for a different reason, and these reasons are as numerous and as varied as the thousands of causes and millions of charities that pursue them. I’d like to share a little about recent choices I’ve made, and how that approach to giving has developed over time.

For me, the essence of giving comes down to three distinct beliefs or realizations. These are, in turn, the limitations of money; the importance of money; and an actual desire to help others. When you combine these three beliefs with actual resources to give, one inevitably must give and must give for impact – that is, one must give well. Thus my approach. I’m not attached to any particular sector, cause, approach, country, or region. But I do look to maximize the effect of my gifts on the human condition overall.

First gifts: A rough start

In the very beginning, as a child, my father greatly influenced my thoughts about giving. He placed ideas in my head such as “don’t give to panhandlers directly, give to organizations”, and created the legitimacy, if not the expectation, of giving freely. Out of this idea, the desire to help now (and not in the unpredictable future), and out of the tradition of tithing shared by many religions, I decided even before I ever had any real earnings that I would give 10% of income (after taxes) to charity, on an ongoing basis. That is a practice that I have continued in the over a decade since.

I had no idea at the time how hard aid actually is, how unsubstantiated most of the nonprofit sector is, or how hard it is to find an effective organization to give to. My first gift was in September 1999, in the amount of $650 (10% of the income from my first summer job) to an organization picked more or less at random: The Unitarian Universalist Service Committee (UUSC). I didn’t have a clear objective other than improving the human condition generally, for which their mission seemed broad enough, and the only real thing that set them apart is that they made the right ask at the right time.

I was disappointed to receive, over the following year, a variety of communications from the UUSC, indicating that my money was being spent in all kinds of ways that (to me) didn’t make sense, from US political activism to questionable (to me) small-scale village business projects. I resolved that I would do a little better next time.

So the next year, after another (more profitable) summer job, I made a different gift. At the time I figured world hunger was a bigger problem than fair trade for cocoa farmers, but thanks to seeds planted by my father I figured there was a better way to tackle hunger than just handing out food. And that’s how I found Freedom from Hunger. Freedom from Hunger operates a large variety of international programs, the centerpiece of which claims to combine microcredit and education for poor rural women.

To evaluate the organization, my main source of information was annual reports. I went back a few years, but didn’t look at any studies or third-party information. Even with this little information or experience, however, I could tell that there were some differences between organizations and looked at a few different ones. Freedom from Hunger had what seemed to be the most impressive annual reports, which I think means intelligent copy that told a story which seemed reasonable, at least from my naïve perspective. So in 2000 I gave them $2,500.

As an exercise, I recently revisited Freedom from Hunger using my now more experienced perspective, to see what evidence of impact they really had. The answer could be described as more than most, and less than the best. Freedom from Hunger did perform a number of controlled studies examining different aspects of its flagship programs, but the results seem questionable, prone to cherry-picking, and it’s not clear to what degree the evaluated treatments match the programs actually implemented worldwide. Freedom from Hunger’s programs today are also significantly more complex than they were in 2000.

One of the biggest lessons I learned from the 2000 gift is that giving well is hard, harder than I ever would have guessed two years previous. I decided I could manage the process better if I made larger gifts less frequently. Eventually, I settled on keeping 10% of my income in a special account each year and then only actually giving it away triennially. As it turns out, I didn’t have any significant income again until 2004, and for tax reasons my next gift was not until 2006.

As an aside, I should note that the triennial 10% gift is specifically for philanthropic efforts. I also make gifts to my church, to people I know personally who are in need, and to political organizations whose causes I support. But I don’t consider that philanthropy, I don’t make the analysis that I would for philanthropic gifts (which are much larger), and I don’t count such gifts toward my 10% donation.

Seeking a better way

After 2000 I did not give again until 2006, but I was not idle during the intervening years. Through The Economist I learned about two organizations of significance, Pratham (Economist article) and Geneva Global (Economist article). Pratham is an Indian educational charity notable for its built-in evaluation approach and closed-loop feedback process. Although Givewell did not exist at the time I discovered Pratham, it has since been reviewed and recommended by Givewell. Geneva Global is a for-profit company that provides advice to donors, especially in terms of impact prediction and evaluation. For their services they charge a fee equal to 15% of the amount donated. Of course, when the time came I also had another look at Freedom from Hunger. I didn’t have any other way to cut through the million charity options, so those became by default my three options to evaluate.

Of the three, I eliminated Freedom from Hunger early on. By now I saw that there was more to impact than glossy annual reports, and I had a growing skepticism about microfinance generally. I was still looking to address root causes; education seemed like a better bet; and Pratham seemed to be making a big difference there. Geneva Global’s main advantage was an appeal to expertise: Their job was to know where to give, so the only question in my mind was whether their expertise was worth the 15% price tag.

In the end, the choice came down to my own procrastination: I didn’t start contacting organizations to ask questions until December, and by the time I found out who at Pratham could answer my questions, they were already away for the holidays. I couldn’t get any substantive information from Pratham until after the new year – not even basic financial information, let alone detailed information about programs. Geneva Global, on the other hand, was beyond responsive: They had lots of detailed information on specific projects, and best of all, a brand-new fund which was offering a 1:1 match to attract starting capital. So instead of paying 15% overhead for Geneva Global, I would end up getting a 70% uplift. In the end, on December 29, 2010, I put a check in the mail for $14,000 toward Geneva Global’s Rwanda Human Empowerment Fund, a collection of projects focused on education, health, and land rights in Rwanda.

As it turns out, I probably chose poorly in 2006. Geneva Global went through a few different purchases and restructurings before winding down the Fund and sending out a final report. This indicated that the Fund had only attracted about a third of the expected amount, but that Geneva Global had begun spending in the first year on the assumption that additional donations would continue to come in. As a result most projects were cut off or could not be continued in the second and third years of the fund’s operation.

In addition, some projects looked highly questionable to my untrained eye. For example, one school project started construction on new classroms, but then shifted midway to make the building a large $64,000 dining hall instead. The school already had a dining hall; the only benefit mentioned in the report was that all 1,000 students could dine at once rather than in shifts. Another project complained that a grantee “lacks professionalism, transparency, and honesty”, leaving the project unfinished and the funds missing. Roughly 1/3 of the total fund was invested in a land rights initative, but the report diplomatically notes that the grantee, the International Justice Mission, “faced many challenges as an outsider”. Finally, none of the projects seemed to have much in the way of quantitative evaluation; with respect to the health initative, the executive summary notes that “Though no formal study has been conducted on the impact of these improved services on infant and maternal mortality rates, we have no doubt that they consequently declined”.

My overall impression was that Geneva Global’s in-county contractor had been given some combination of too much leeway and too little training, and that a substantial amount of the money was lost to subcontractors, spent on unplanned expenses, or wasted on poorly planned projects.

The main lesson I took away from this is that even with a fair amount of effort, giving for impact is really hard! So for my next gift, made in 2009, I determined to do a better job. I started the process early in the year with only Pratham in hand, but quickly discovered resources like GiveWell and Good Intentions are Not Enough. It seems that over the course of the 2000s, a lot of people got interested in impact philanthropy all at once.

Information: A trickle becomes a torrent

GiveWell turned out to be a critical source of information for my 2009 donation. Besides pointing me to exceptional organizations like Population Services International (PSI; Givewell review) and VillageReach (Givewell review), the GiveWell blog and its archives provided a steady supply of new perspectives on charity and giving, ideas which I had never encountered before. Sure, I had long suspected that there were large differences between charities, and had discovered through my own mistakes that giving was hard, but I had never thought about things like offsetting impacts, the complexities of microfinance interest and repayment rates, the costs of receiving aid, or room for more funding. GiveWell suggested some great organizations and then gave some useful benchmarks by which to measure them.

In the end, I settled on a shortlist of Pratham, PSI, and VillageReach. The latter two I discovered through GiveWell. PSI is a large international health charity with some innovative ideas about how to leverage markets to improve distribution and use of health products and services like condoms, bed nets, circumcision, and vaccination. VillageReach is a US charity originally operating exclusively in Mozambique which is notable for having one of the most complete and inarguable demonstrations of impact among charities of any size.

GiveWell also recommended a few other organizations that didn’t pique my interest as much. The Against Malaria Foundation (AMF; Givewell review) interested me less than PSI primarily because of the latter’s inclusion of reproductive health as an essential part of health care delivery. And the Stop TB Partnership (Givewell Review) is an organization which seems to be very effective, but which I heard about too late to consider for a 2006 donation. In addition, giving to the Stop TB partnership would be significantly more difficult, since it is not a US organization and since my 2009 donation was mostly in the form of appreciated stock.

I contacted all three organizations (Pratham, PSI, and VillageReach) and engaged each of them with a battery of questions regarding their programs, evaluations, use of marginal funds, and much more. Owing to the earlier date (and perhaps larger donation size), I had a very different experience compared to 2006: All three organizations were very much forthcoming with both information and time. I read a lot of reports, came back with more questions, and spent hours on the phone, both with charities and with folks from GiveWell. This was a tough decision, because the organizations are fairly different from each other but each with a compelling case.

Pratham was the first to be disqualified. Essentially, the majority of Pratham’s spending goes to two types of programs: Field programs, where Pratham operates remedial education sessions directly on a village-by-village basis, and Read India, where Pratham partners with local governments to improve the quantity and quality of education in public schools. In an attempt to reach the massive scale required for nationwide impact in India, Pratham has focused most of its growth on Read India. But the best case and evidence, both in terms of program design and in terms of third-party evaluation, is for the field programs. In late 2009, Pratham was in the midst of conducting rigorous controlled trials of a few Read India implementations. I have no doubt that they’ll make a strong showing for my 2012 gift, but to me there was not enough evidence on Read India to justify a gift in 2009.

That left PSI and VillageReach: Both health charities, though with very different scale and approaches, making for a difficult comparison. My decision was made more difficult by the fact that VillageReach was in the middle of its own strategic planning process while my evaluation was underway. Indeed, I had originally decided against VillageReach, but gave it new consideration according to new information provided at the last minute.

PSI is a large organization and a relatively known quantity. They provide a fairly advanced quantitative model attempting to translate any and all health interventions into Disability-Adjusted Life Years (DALYs), then allocating funds to the most cost-effective projects, as measured in cost per DALY. Of course, a large organization like PSI receives a lot of restricted funding, so the organization operates some projects even if they have very high cost per DALY. Nonetheless, PSI prepares detailed and easy-to-read reports on both impact and cost-effectiveness. The questions are likewise fairly straightforward: Is the model which translates pregnancies avoided into DALYs a correct one? Are the projects sustainable? Is PSI simply replacing health products and services that would be provided by free markets or governments, or is it expanding markets to consumers that would not be reached otherwise? PSI’s programs are clear but determining true impacts must unfortunately leave some of these questions unanswered.

VillageReach

VillageReach is another story. In order to understand the organization, you need to understand its history. VillageReach aims to improve health outcomes by making distribution systems for health products, especially vaccines, work more efficiently and effectively. The organization started its programs in 2002 with a pilot project in the Cabo Delgado province of Mozambique. VillageReach created a new approach to vaccine distribution, shifting the burden from health clinic workers to a set of staff and equipment dedicated to distribution. In addition to delivery personnel, VillageReach created a new job position to monitor the inventory of vaccines and the effectiveness of the delivery schemes. All these people nominally worked for the government, and all this equipment nominally belonged to the government, but VillageReach paid all the costs, whereas costs for pre-existing systems had been covered by the government. In order to make all this work, VillageReach had to create a whole new distribution company for propane gas, which was previously prohibitively expensive to purchase in Cabo Delgado. VillageReach reports that the company, named VidaGas, is now self-sustaining.

The pilot program ended in 2007, at which point VillageReach handed control over to the government, which promptly dismantled VillageReach’s work and returned to the previously extant approach. But in the meantime, VillageReach prepared two significant reports: One on impact and the other on cost. The two reports compared the cost and effectiveness of the VillageReach approach to the one the government had been using previously, by comparing vaccination rates in the trial province of Cabo Delgado to those in the neighboring province of Niassa. The conclusion is as powerful as it is inescapable: When compared to its neigbor, Cabo Delgado saw much higher rates of vaccination at significantly lower total cost. In other words, the VillageReach approach was lower-cost, not only on a per-vaccination basis but on an overall basis. Indeed, the vaccination program was so cheap that as a donor it would be considered a bargain even considering that VillageReach had to displace all the government spending.

This is the history I had in hand when I started talking to VillageReach in 2009. And when I asked the question, “what’s next?”, I was concerned about the answer. Of course, the vaccination program in Cabo Delgado had already been shut down for two years, and in its place VillageReach talked about a variety of consulting and advocacy programs. But I had no evidence about VillageReach’s effectiveness as council in any country, let alone in those far away from Mozambique. Essentially, given the future plan VillageReach had presented to me, I had to reject them: All the evidence of effectiveness was in relation to a program that was no longer operating.

So I decided to give to PSI. But I didn’t want to give them a free pass, so I resolved to evaluate them as thoroughly as I could, quickly compare that to some other organizations’ claims, and if PSI seemed adequate to choose them.

But, at the last minute, due in part to urging by GiveWell, I gave VillageReach one last phone call, wherein they explained a new agreement with the government of Mozambique to roll out the Cabo Delgado vaccination distribution approach nationwide. This seemed to me to be much closer to VillageReach’s proven pilot project and thus much likelier to be a success. The pilot project had been so successful that I couldn’t say no to a nationwide deployment, and so I gave my $35,000 2009 contribution to VillageReach. I later found out, much to my surprise, than this gift was and is their largest ever donation from an individual.

Back to the Future

Which brings us to the end of the story (for now). In an adventure spanning a bit over a decade, I’ve learned a lot of surprising things: Doing good is harder than it looks. Good intentions are not enough. Helping others is rewarding. And maybe, with the careful application of analytic thought, giving money really can make a difference. Like science, it’s a process of discovery. You try something out, you look to see how it goes, you learn something, and then you repeat the process, and hopefully your little extra knowledge and experience lets you see something deeper the next time around. It’s not easy, but it is possible.

As I look forward to the future of philanthropy, both mine and others’, it’s an exciting picture. There will be new discoveries about the nature of giving, and as the nonprofit sector grows daily more interested in measuring impact, the number of opportunities to give effectively grows with it. And I’m confident that my own skills at giving will continue to grow as well. Charity can change the world: Just make sure you look before you give.

Acknowledgements

This essay would not have been possible without the thoughts, ideas, inspiration, proofreading, and support of a surprisingly large group of friends and family and of the GiveWell staff. You have provided me with essential feedback for which I am unremittingly grateful.

Thanks also to the many people and organizations that have provided me with time, information and ideas throughout my philanthropic career. A donor can accomplish nothing without a place to give, and I am grateful for each of you, even when I have directed funds elsewhere. Our field is in an interesting time at the moment, and is well-poised for significant new accomplishments. Let’s make it happen.

Engineers Without Borders earns “Notable” distinction

| by Holden

Engineers Without Borders is a group we’ve never encountered before, focused on “enabling rural Africans the opportunity to access clean water, generate an income from humble farms, and access critical infrastructure and services.” (Here’s a list of its projects in Africa.)

This group deserves recognition for doing the simplest thing guaranteed to improve GiveWell’s view of it: disclosing a failure.

Its 2009 and 2010 failure reports are publicly and prominently published. These aren’t “quasi-failures” along the lines of “Didn’t raise enough money for X”; they involve staff misreading their environments, both in terms of culture (poorly anticipating how a project’s timeline would be affected by holiday schedules) and in terms of interactions between aid institutions (nearly losing funding for a project by what reads to me like hurting a funder’s feelings). I encourage you to read the reports themselves rather than relying on our summary, since they are eye-opening.

Without knowing anything else about the work of Engineers Without Borders, we commend it for clearly and publicly discussing its own shortcomings, and thereby creating more public information on the complexities of aid.

Hat tip: Aid Watch