VillageReach currently holds the top slot in our top charities list. While I will be donating to the Stop Tuberculosis partnership (discussed yesterday), Elie’s donation this year went to VillageReach.

VillageReach consults on health system logistics in high-poverty, remote areas to help life-saving supplies get to those who need them. It may not be the most attention-grabbing story (“improving operational efficiency” doesn’t have the same ring to it as giving a cow or fixing a smile), but

• It’s directly addressing a key issue – limited health system capacity – that may limit the amount of good donations to any health-related cause are able to do.
• Immunizations in sub-Saharan Africa are a top global health priority (ranked #4 by the Copenhagen Consensus, for example and have been estimated to avert infant deaths for ~$200 each (estimate includes all costs).

In other words, VillageReach would be accomplishing tremendous good if it could improve logistics to deliver more vaccines. Can it?

Pilot project

VillageReach assessed its pilot project in Mozambique (Cabo Delgado province) with more thoroughness than we’ve seen almost anywhere outside of JPAL and IPA. It is a great demonstration that just because a charity is relatively small, and experimental, doesn’t mean it can’t collect substantive and compelling evidence regarding its impact.

VillageReach collected (and has now published) data on trends in vaccine/supply deliveries, “stock-outs” of vaccines (the single most compelling indicator, in our view) and vaccination coverage rates. It also conducted vaccination coverage surveys in both the province where it worked and a nearby province for comparison. After examining this evidence along with some independent analysis with our own, we’ve concluded that the case for impact (despite not including a randomized controlled trial) is quite compelling: VillageReach caused a significant jump in vaccination rates, which we believe can be fairly extrapolated to lives saved at well under $1000 each.

VillageReach hoped to transfer responsibility for the modified logistics system (which has a dedicated team responsible for delivering supplies, rather than each clinic responsible for its own collection) to the government, but available evidence suggests that the transfer failed. The fact that vaccination rates have since fallen is further evidence that VillageReach made a difference while they were there, but obviously discouraging relative to what they had hoped for. We have expressed doubts in the past about donors’ getting their hopes up for “sustainability” in aid, and think it’s worth noting that even VillageReach’s project – with its strong case for impact as well as a case that it actually would save the government money – has so far failed at the ambitious goal of changing government practices.

More details at the “Does it work?” section of our review.

The future of VillageReach

There are a lot of questions about the future of VillageReach. As a small organization, it sees frequent changes in its plans and engagements. It has taken on a variety of consulting projects, some of which we have little information about and could be significantly different from where it has succeeded before.

However, it has provided an unusual amount of clarity and specificity about the likely use of additional donations. We feel that VillageReach can productively absorb up to $2.5 million in unrestricted donations over the next year and that these funds will be used for work very similar to its pilot project: reactivating its program in the original province, in addition to possibly expanding to one or more neighboring provinces.

Pros and cons

VillageReach is very different from the “ideal charity” we had in mind early in our international aid research. (Our “ideal charity” is much closer to the charity I discussed yesterday, the Stop Tuberculosis Partnership). It is relatively small and still represents something of an experiment, which has both pros and cons.

VillageReach is a small charity that has struggled at times to raise enough funding, not a large charity built to absorb huge sums of money.

• This adds a certain kind of uncertainty to its situation, as its plans hinge heavily on how fundraising goes as well as other factors (such as government cooperation) outside its control.
• But being small also comes with advantages for a donor. The need for more funding is extremely concrete and some extra funding could make a huge difference to VillageReach’s ability to perform up to its potential. In addition, it is feasible – if difficult – to have a complete view of VillageReach’s work. Where a large charity like Stop TB Partnership relies heavily on audits and spot-checks – leaving open a lot of room for unobservable error – VillageReach is measuring, and accountable for, the results of everything it does.

VillageReach has only one demonstrable success to its credit. It is still establishing the potential of its model.

• VillageReach is “experimental” in a sense. It doesn’t have the same consistency, accomplishing similar results in many times and places, as Stop TB Partnership.
• However, we think this is a very different level of “experiment” from an organization with no demonstrated impact. We think there is enough information about VillageReach’s past impact for an individual donor to be confident that its future activities are a “good bet.”
• And with the risk of an “experiment” comes upside. If VillageReach continues to demonstrate success, it could affect the way health logistics works all over the world, through the spread of information rather than through its own funds. The problem we have with most “experiments” in international aid is that we aren’t convinced that they will produce the information needed to (a) determine whether they’ve succeeded and (b) bring about widespread adoption if they have. By contrast, VillageReach has demonstrated a willingness and ability to produce credible evidence of impact.

VillageReach does not fully share GiveWell’s priorities.

• We are most interested in “doing more of something that demonstrably improves lives.” VillageReach has expressed great interest – and allocated funds to – the more ambitious goal of “advocacy” (broadly defined), changing the way governments and larger funders operate.
• Yet even as it aims for these more ambitious goals, we believe that VillageReach is in fact saving lives at a level of cost-effectiveness that rivals that of any other charity. So we see its more ambitious goals almost as a “bonus” – if it succeeds at them, we’ll have helped fund a truly incredible level of impact, and if it fails at them, we still expect to have gotten our “money’s worth” (lives saved for under $1000 each).

Bottom line

The pros and cons listed above are largely mirror images of each other. Stop TB Partnership is an outstanding “large charity” – with both the advantages and concerns that accompany great size – and VillageReach is an outstanding “small charity,” with a different set of advantages and concerns. Both give good answers to all our major questions, leaving only the kind of concerns that are likely inevitable with charities of their size.

Which you have more confidence in, and prefer to donate to, is a judgment call. We have ranked VillageReach as our top charity because Elie and I happen to agree that it seems like a “better bet” overall (to approach the impact implied by the academic cost-effectiveness estimates) – we think the ability to hold it fully accountable outweighs the risks that come with being small and experimental.

Elie and I do part ways when it comes to the ultimate donation decision. I am giving to Stop TB Partnership for the personal values-related reasons discussed yesterday. Elie doesn’t share the values I mentioned and has donated to VillageReach.