The GiveWell Blog

Health system strengthening + sustainability + accountability

(updated on: ) | by Holden

We’ve written a lot about our top-rated charity, VillageReach (in particular, see our official review and our 2009 blog post). Consistent with our focus on individual/casual donors, we generally emphasize effectiveness over ambition, and so we have mostly focused on VillageReach’s fit with our criteria: its strong case for effectveness, cost-effectiveness, and room for more funding.

However, VillageReach has some other major strengths as well – strengths that make it a good fit not just for individual donors looking to help people cost-effectively, but also for more ambitious donors hoping to contribute to transformative change.

Health system strengthening – with tangible results

One of the ongoing debates/discussions over health aid involves “vertical” vs. “horizontal” interventions. I recommend William Easterly’s Can The West Save Africa? (start on page 57) for an overview of the issue, but briefly:

  • “Vertical” programs focus on global large-scale campaigns addressing a particular health condition, often focusing on the supply of appropriate medical materials (for example, vaccines and antiretroviral drugs for treating HIV/AIDS) while “horizontal” aid tries to strengthen the health system in general in a particular region.
  • “Vertical” programs have had some major demonstrable successes (for example, most of the cases in Millions Saved). However, many believe that once the major opportunities have been capitalized on, these sorts of initiatives hit diminishing returns, as medical supplies outstrip the health system’s ability to deliver them. Over-funding a particular campaign may even get in the way and undermine other health efforts.
  • “Horizontal” programs have the worthy goal of making the health system stronger as a whole, including reaching those who are hardest to reach. However, perhaps because these programs often involve more complex goals and more leeway for the funded, these programs do not have the same track record of demonstrable success, and arguably have inferior mechanisms for accountability (i.e., making sure that money is spent well and produces actual results). See this Aid Watch post for more on these problems.

VillageReach’s program, focused on health system logistics, is fundamentally more “horizontal” than “vertical” – it’s focused on improving health system capacity. For example, one of the major changes its pilot project brought about was creating a specialized team for delivery of vaccines, which relieved local health system personnel of the responsibility to collect their own vaccines. The aim was not only to get supplies out to the villages in a more reliable way, but to reduce demands and constraints on the many local health system personnel. In the future it may expand this model to other health supplies. It’s not just using donations to purchase supplies – it’s reorganizing the system to improve overall efficiency and even reduce total costs.

Yet unlike many “health system strengthening” initiatives, VillageReach’s program holds itself accountable with specific deliverables, promising – for example – a drastic drop in the frequency with which health centers run out of vaccines. It has run surveys of vaccination rates, comparing the province it worked in to a nearby province. Looking at vaccination rates cannot capture all of the benefits VillageReach aims to provide, but it provides a strong reality check on the model and suggestive evidence for whether it is indeed improving efficiency.

The program thus addresses one of the thornier, and difficult-to-evaluate, areas of global health, while maintaining strong accountability and demonstrable results.

Ambitions of transformation and sustainability – with tangible results in the meantime

Two other constant tensions we perceive in health initiatives are between (a) short- and long-term impact; (b) tangible and transformative impact. In neither case are the two goals clearly contradictory, yet in practice there often seem to be tradeoffs.

For our part, we have focused on short-term, tangible impact. As we lay out in our process overview, we are focused on serving individual/casual donors, and we feel these donors should look first and foremost for accountability and results. A chance at “hitting the jackpot” (creating a permanent and/or far-reaching change) ought to be considered a bonus, not a requirement, especially when the short-term tangible impacts are as good as saving a life for every $1,000 (or less) spent.

Yet the best charity we’ve found for these short-term, tangible impacts turns out to be very ambitious, with very high upside should they succeed. As detailed in our review, VillageReach isn’t just implementing a new health system – it’s advocating for this system as a new approach to health logistics all across the country of Mozambique. It has produced cost analysis concluding that the government will save money, long-term, by adopting its model; with this analysis and its superior results in hand, it has won the endorsement of the national Ministry of Health. It appears to have as good a chance at a lasting impact on how the government operates as any other charity we’ve seen.

Its impact need not even end within Mozambique. Getting supplies to hard-to-reach areas is a challenge that applies throughout the world of global health (and indeed VillageReach has been retained as a contractor in Senegal, Malawi and India).

If you’re hoping your donation has a chance to be a part of a new innovation with global impact, VillageReach gives you that chance. Of course, many other charities have equal or grander ambitions. The difference is that VillageReach is (from what we’ve seen) holding itself far more accountable, producing tangible – and cost-effective – impact as it goes.

What’s not to like?

Our process has been accused of over-focusing on tangible impact and evaluation, and thus favoring larger, more conservative organizations. This accusation has some truth to it (and we accept this bias in view of what kind of donor we’re serving). And yet, the organization we’ve found that performs best on our criteria is in fact relatively small, and highly innovative. Indeed, of all the organizations out there with big ambitions, we believe VillageReach – with its high accountability and its focus on producing results, not just hopes, in the meantime – to be the best bet to actually deliver.

Microfinance: The multi-billion-dollar aid sector that we’re just starting to learn about

(updated on: ) | by Holden

GiveWell attended the Microfinance Innovation & Impact Conference via Board member Tim Ogden and part-time employee Stephanie Wykstra. (Our full-time staff could not attend as we are all in India). Our main takeaways:

  • The people involved with this conference, and the nonprofits working with them, are producing a lot of promising, interesting, rigorous and informative work on how microfinance affects the poor and how these effects might be improved.
  • While microlending is the form of microfinance that has been most funded, celebrated and scaled up over the decades, it appears to be the least proven and least-well understood in terms of its impact on the poor. In many ways it seems that good information on how (and to what extent) it’s helpful is just starting to become available.

Content presented at the conference
There is much abstract agreement about the importance of learning from good research, but we feel that there are few concrete examples of groups actually doing it. A high concentration of those examples could be found at this conference. We have written before about how impressed we are with the work of the Poverty Action Lab and Innovations for Poverty Action (groups behind this conference), and that continues to be the case.

The research has already been summarized in a large number of posts by others (see the roundup by GiveWell Board member Tim Ogden). Our quick take:

Microsavings, microinsurance, and “ultrapoor programs” are clearly promising. High-quality studies found significant improvements on metrics like income, food security, and children’s ability to attend school when these programs – which differ markedly from the loans most people associate with “microfinance” – were rolled out. Programs included accounts intended to help people pre-commit to saving (more); intensive promotion of difficult-to-sell insurance products (more), and “ultra-poor programs” that target people with extremely low incomes and offer direct benefits such as cash/asset transfers and education, rather than financial products (more).

Unfortunately, we know of no clear vehicles through which individual donors can fund these sorts of programs. We’ve written before about the frustrating situation of seeing a good program without a good funding vehicle, and called for major funders to consider this problem. There do appear to be similarities between the “ultrapoor program” and the work of GiveWell-recommended charity Village Enterprise Fund.

The impact of microlending is, at this point, not very well understood. A new high-quality study in Morocco found no positive social impact overall, though it did find that people with existing agricultural businesses appeared to invest more/differently in their businesses (more). These results were similar to those observed in the only previous high-quality study of traditional microlending, which took place in a very different environment (Hyderabad, India) and was summarized last year by David Roodman.

There was significant discussion of the risks that people might be overborrowing (something we are quite concerned about, and something that there is apparently still little information to assess). And new data from the Philippines reinforced a point we have emphasized in the past – that microloans are more likely to finance consumption, and less likely to finance entrepreneurial investment, than most donors suppose. From Philanthropy Action’s summary:

Around 46 percent of clients used new debt to pay down old debt, and 28 percent used funds for a single large household purchase. In all, 39 percent of the money goes unaccounted for, 15 percent is used to pay down other loans, 9 percent goes into the household and 37 percent is invested in the business. Curiously, Karlan’s study got at these results by asking clients the same questions in a number of different ways. Not surprisingly, when the banks ask clients why they want the funds only 2 percent of clients admit potential uses other than the business, even after the funds are given. Only when an independent surveyor asks indirectly do any relevant percentage of people admit to non-entrepreneurial uses for credit.

Studies are finding interesting potential ways to improve the impact of microlending. One possibility is better targeting (as discussed above, microlending appears to affect different people differently). Two other possibilities raised at the conference, both with encouraging preliminary evidence behind them, were the implementation of grace periods for repayment (more) and financial education focusing on “rules of thumb” rather than formal accounting (more).

Learning how microlending works – better late than never
It appears to us that an enormous amount of philanthropic capital has gone into microfinance. Last year we looked at data from 2008 and found that $11.7 billion had gone to microfinance funding of some kind, with $2.2 billion being grants. For context, total official aid earmarked for health was about $3 billion disbursed / $8 billion committed in the same year (data from Aid Data).

My greatest fear about microfinance is that all (or nearly all) of that money is chasing a good story rather than a good program.

In this context, it’s hard for us to see how one can argue that good evaluation is too expensive. The total budget of Innovations for Poverty Action that year was just under $6 million (data from NCCS).

The process of giving: A personal story (guest post from Ian Turner)

(updated on: ) | by Guest

This is a guest post from Ian Turner about how he decided what charity to support for his most recent donation. Ian and GiveWell staff had several in-depth conversations as he worked through his decision, so we invited him to share his thought process here. Note that GiveWell has made minor editing suggestions for this post (though Ian determined the final content).

Every person who gives to charity does so for a different reason, and these reasons are as numerous and as varied as the thousands of causes and millions of charities that pursue them. I’d like to share a little about recent choices I’ve made, and how that approach to giving has developed over time.

For me, the essence of giving comes down to three distinct beliefs or realizations. These are, in turn, the limitations of money; the importance of money; and an actual desire to help others. When you combine these three beliefs with actual resources to give, one inevitably must give and must give for impact – that is, one must give well. Thus my approach. I’m not attached to any particular sector, cause, approach, country, or region. But I do look to maximize the effect of my gifts on the human condition overall.

First gifts: A rough start

In the very beginning, as a child, my father greatly influenced my thoughts about giving. He placed ideas in my head such as “don’t give to panhandlers directly, give to organizations”, and created the legitimacy, if not the expectation, of giving freely. Out of this idea, the desire to help now (and not in the unpredictable future), and out of the tradition of tithing shared by many religions, I decided even before I ever had any real earnings that I would give 10% of income (after taxes) to charity, on an ongoing basis. That is a practice that I have continued in the over a decade since.

I had no idea at the time how hard aid actually is, how unsubstantiated most of the nonprofit sector is, or how hard it is to find an effective organization to give to. My first gift was in September 1999, in the amount of $650 (10% of the income from my first summer job) to an organization picked more or less at random: The Unitarian Universalist Service Committee (UUSC). I didn’t have a clear objective other than improving the human condition generally, for which their mission seemed broad enough, and the only real thing that set them apart is that they made the right ask at the right time.

I was disappointed to receive, over the following year, a variety of communications from the UUSC, indicating that my money was being spent in all kinds of ways that (to me) didn’t make sense, from US political activism to questionable (to me) small-scale village business projects. I resolved that I would do a little better next time.

So the next year, after another (more profitable) summer job, I made a different gift. At the time I figured world hunger was a bigger problem than fair trade for cocoa farmers, but thanks to seeds planted by my father I figured there was a better way to tackle hunger than just handing out food. And that’s how I found Freedom from Hunger. Freedom from Hunger operates a large variety of international programs, the centerpiece of which claims to combine microcredit and education for poor rural women.

To evaluate the organization, my main source of information was annual reports. I went back a few years, but didn’t look at any studies or third-party information. Even with this little information or experience, however, I could tell that there were some differences between organizations and looked at a few different ones. Freedom from Hunger had what seemed to be the most impressive annual reports, which I think means intelligent copy that told a story which seemed reasonable, at least from my naïve perspective. So in 2000 I gave them $2,500.

As an exercise, I recently revisited Freedom from Hunger using my now more experienced perspective, to see what evidence of impact they really had. The answer could be described as more than most, and less than the best. Freedom from Hunger did perform a number of controlled studies examining different aspects of its flagship programs, but the results seem questionable, prone to cherry-picking, and it’s not clear to what degree the evaluated treatments match the programs actually implemented worldwide. Freedom from Hunger’s programs today are also significantly more complex than they were in 2000.

One of the biggest lessons I learned from the 2000 gift is that giving well is hard, harder than I ever would have guessed two years previous. I decided I could manage the process better if I made larger gifts less frequently. Eventually, I settled on keeping 10% of my income in a special account each year and then only actually giving it away triennially. As it turns out, I didn’t have any significant income again until 2004, and for tax reasons my next gift was not until 2006.

As an aside, I should note that the triennial 10% gift is specifically for philanthropic efforts. I also make gifts to my church, to people I know personally who are in need, and to political organizations whose causes I support. But I don’t consider that philanthropy, I don’t make the analysis that I would for philanthropic gifts (which are much larger), and I don’t count such gifts toward my 10% donation.

Seeking a better way

After 2000 I did not give again until 2006, but I was not idle during the intervening years. Through The Economist I learned about two organizations of significance, Pratham (Economist article) and Geneva Global (Economist article). Pratham is an Indian educational charity notable for its built-in evaluation approach and closed-loop feedback process. Although Givewell did not exist at the time I discovered Pratham, it has since been reviewed and recommended by Givewell. Geneva Global is a for-profit company that provides advice to donors, especially in terms of impact prediction and evaluation. For their services they charge a fee equal to 15% of the amount donated. Of course, when the time came I also had another look at Freedom from Hunger. I didn’t have any other way to cut through the million charity options, so those became by default my three options to evaluate.

Of the three, I eliminated Freedom from Hunger early on. By now I saw that there was more to impact than glossy annual reports, and I had a growing skepticism about microfinance generally. I was still looking to address root causes; education seemed like a better bet; and Pratham seemed to be making a big difference there. Geneva Global’s main advantage was an appeal to expertise: Their job was to know where to give, so the only question in my mind was whether their expertise was worth the 15% price tag.

In the end, the choice came down to my own procrastination: I didn’t start contacting organizations to ask questions until December, and by the time I found out who at Pratham could answer my questions, they were already away for the holidays. I couldn’t get any substantive information from Pratham until after the new year – not even basic financial information, let alone detailed information about programs. Geneva Global, on the other hand, was beyond responsive: They had lots of detailed information on specific projects, and best of all, a brand-new fund which was offering a 1:1 match to attract starting capital. So instead of paying 15% overhead for Geneva Global, I would end up getting a 70% uplift. In the end, on December 29, 2010, I put a check in the mail for $14,000 toward Geneva Global’s Rwanda Human Empowerment Fund, a collection of projects focused on education, health, and land rights in Rwanda.

As it turns out, I probably chose poorly in 2006. Geneva Global went through a few different purchases and restructurings before winding down the Fund and sending out a final report. This indicated that the Fund had only attracted about a third of the expected amount, but that Geneva Global had begun spending in the first year on the assumption that additional donations would continue to come in. As a result most projects were cut off or could not be continued in the second and third years of the fund’s operation.

In addition, some projects looked highly questionable to my untrained eye. For example, one school project started construction on new classroms, but then shifted midway to make the building a large $64,000 dining hall instead. The school already had a dining hall; the only benefit mentioned in the report was that all 1,000 students could dine at once rather than in shifts. Another project complained that a grantee “lacks professionalism, transparency, and honesty”, leaving the project unfinished and the funds missing. Roughly 1/3 of the total fund was invested in a land rights initative, but the report diplomatically notes that the grantee, the International Justice Mission, “faced many challenges as an outsider”. Finally, none of the projects seemed to have much in the way of quantitative evaluation; with respect to the health initative, the executive summary notes that “Though no formal study has been conducted on the impact of these improved services on infant and maternal mortality rates, we have no doubt that they consequently declined”.

My overall impression was that Geneva Global’s in-county contractor had been given some combination of too much leeway and too little training, and that a substantial amount of the money was lost to subcontractors, spent on unplanned expenses, or wasted on poorly planned projects.

The main lesson I took away from this is that even with a fair amount of effort, giving for impact is really hard! So for my next gift, made in 2009, I determined to do a better job. I started the process early in the year with only Pratham in hand, but quickly discovered resources like GiveWell and Good Intentions are Not Enough. It seems that over the course of the 2000s, a lot of people got interested in impact philanthropy all at once.

Information: A trickle becomes a torrent

GiveWell turned out to be a critical source of information for my 2009 donation. Besides pointing me to exceptional organizations like Population Services International (PSI; Givewell review) and VillageReach (Givewell review), the GiveWell blog and its archives provided a steady supply of new perspectives on charity and giving, ideas which I had never encountered before. Sure, I had long suspected that there were large differences between charities, and had discovered through my own mistakes that giving was hard, but I had never thought about things like offsetting impacts, the complexities of microfinance interest and repayment rates, the costs of receiving aid, or room for more funding. GiveWell suggested some great organizations and then gave some useful benchmarks by which to measure them.

In the end, I settled on a shortlist of Pratham, PSI, and VillageReach. The latter two I discovered through GiveWell. PSI is a large international health charity with some innovative ideas about how to leverage markets to improve distribution and use of health products and services like condoms, bed nets, circumcision, and vaccination. VillageReach is a US charity originally operating exclusively in Mozambique which is notable for having one of the most complete and inarguable demonstrations of impact among charities of any size.

GiveWell also recommended a few other organizations that didn’t pique my interest as much. The Against Malaria Foundation (AMF; Givewell review) interested me less than PSI primarily because of the latter’s inclusion of reproductive health as an essential part of health care delivery. And the Stop TB Partnership (Givewell Review) is an organization which seems to be very effective, but which I heard about too late to consider for a 2006 donation. In addition, giving to the Stop TB partnership would be significantly more difficult, since it is not a US organization and since my 2009 donation was mostly in the form of appreciated stock.

I contacted all three organizations (Pratham, PSI, and VillageReach) and engaged each of them with a battery of questions regarding their programs, evaluations, use of marginal funds, and much more. Owing to the earlier date (and perhaps larger donation size), I had a very different experience compared to 2006: All three organizations were very much forthcoming with both information and time. I read a lot of reports, came back with more questions, and spent hours on the phone, both with charities and with folks from GiveWell. This was a tough decision, because the organizations are fairly different from each other but each with a compelling case.

Pratham was the first to be disqualified. Essentially, the majority of Pratham’s spending goes to two types of programs: Field programs, where Pratham operates remedial education sessions directly on a village-by-village basis, and Read India, where Pratham partners with local governments to improve the quantity and quality of education in public schools. In an attempt to reach the massive scale required for nationwide impact in India, Pratham has focused most of its growth on Read India. But the best case and evidence, both in terms of program design and in terms of third-party evaluation, is for the field programs. In late 2009, Pratham was in the midst of conducting rigorous controlled trials of a few Read India implementations. I have no doubt that they’ll make a strong showing for my 2012 gift, but to me there was not enough evidence on Read India to justify a gift in 2009.

That left PSI and VillageReach: Both health charities, though with very different scale and approaches, making for a difficult comparison. My decision was made more difficult by the fact that VillageReach was in the middle of its own strategic planning process while my evaluation was underway. Indeed, I had originally decided against VillageReach, but gave it new consideration according to new information provided at the last minute.

PSI is a large organization and a relatively known quantity. They provide a fairly advanced quantitative model attempting to translate any and all health interventions into Disability-Adjusted Life Years (DALYs), then allocating funds to the most cost-effective projects, as measured in cost per DALY. Of course, a large organization like PSI receives a lot of restricted funding, so the organization operates some projects even if they have very high cost per DALY. Nonetheless, PSI prepares detailed and easy-to-read reports on both impact and cost-effectiveness. The questions are likewise fairly straightforward: Is the model which translates pregnancies avoided into DALYs a correct one? Are the projects sustainable? Is PSI simply replacing health products and services that would be provided by free markets or governments, or is it expanding markets to consumers that would not be reached otherwise? PSI’s programs are clear but determining true impacts must unfortunately leave some of these questions unanswered.

VillageReach

VillageReach is another story. In order to understand the organization, you need to understand its history. VillageReach aims to improve health outcomes by making distribution systems for health products, especially vaccines, work more efficiently and effectively. The organization started its programs in 2002 with a pilot project in the Cabo Delgado province of Mozambique. VillageReach created a new approach to vaccine distribution, shifting the burden from health clinic workers to a set of staff and equipment dedicated to distribution. In addition to delivery personnel, VillageReach created a new job position to monitor the inventory of vaccines and the effectiveness of the delivery schemes. All these people nominally worked for the government, and all this equipment nominally belonged to the government, but VillageReach paid all the costs, whereas costs for pre-existing systems had been covered by the government. In order to make all this work, VillageReach had to create a whole new distribution company for propane gas, which was previously prohibitively expensive to purchase in Cabo Delgado. VillageReach reports that the company, named VidaGas, is now self-sustaining.

The pilot program ended in 2007, at which point VillageReach handed control over to the government, which promptly dismantled VillageReach’s work and returned to the previously extant approach. But in the meantime, VillageReach prepared two significant reports: One on impact and the other on cost. The two reports compared the cost and effectiveness of the VillageReach approach to the one the government had been using previously, by comparing vaccination rates in the trial province of Cabo Delgado to those in the neighboring province of Niassa. The conclusion is as powerful as it is inescapable: When compared to its neigbor, Cabo Delgado saw much higher rates of vaccination at significantly lower total cost. In other words, the VillageReach approach was lower-cost, not only on a per-vaccination basis but on an overall basis. Indeed, the vaccination program was so cheap that as a donor it would be considered a bargain even considering that VillageReach had to displace all the government spending.

This is the history I had in hand when I started talking to VillageReach in 2009. And when I asked the question, “what’s next?”, I was concerned about the answer. Of course, the vaccination program in Cabo Delgado had already been shut down for two years, and in its place VillageReach talked about a variety of consulting and advocacy programs. But I had no evidence about VillageReach’s effectiveness as council in any country, let alone in those far away from Mozambique. Essentially, given the future plan VillageReach had presented to me, I had to reject them: All the evidence of effectiveness was in relation to a program that was no longer operating.

So I decided to give to PSI. But I didn’t want to give them a free pass, so I resolved to evaluate them as thoroughly as I could, quickly compare that to some other organizations’ claims, and if PSI seemed adequate to choose them.

But, at the last minute, due in part to urging by GiveWell, I gave VillageReach one last phone call, wherein they explained a new agreement with the government of Mozambique to roll out the Cabo Delgado vaccination distribution approach nationwide. This seemed to me to be much closer to VillageReach’s proven pilot project and thus much likelier to be a success. The pilot project had been so successful that I couldn’t say no to a nationwide deployment, and so I gave my $35,000 2009 contribution to VillageReach. I later found out, much to my surprise, than this gift was and is their largest ever donation from an individual.

Back to the Future

Which brings us to the end of the story (for now). In an adventure spanning a bit over a decade, I’ve learned a lot of surprising things: Doing good is harder than it looks. Good intentions are not enough. Helping others is rewarding. And maybe, with the careful application of analytic thought, giving money really can make a difference. Like science, it’s a process of discovery. You try something out, you look to see how it goes, you learn something, and then you repeat the process, and hopefully your little extra knowledge and experience lets you see something deeper the next time around. It’s not easy, but it is possible.

As I look forward to the future of philanthropy, both mine and others’, it’s an exciting picture. There will be new discoveries about the nature of giving, and as the nonprofit sector grows daily more interested in measuring impact, the number of opportunities to give effectively grows with it. And I’m confident that my own skills at giving will continue to grow as well. Charity can change the world: Just make sure you look before you give.

Acknowledgements

This essay would not have been possible without the thoughts, ideas, inspiration, proofreading, and support of a surprisingly large group of friends and family and of the GiveWell staff. You have provided me with essential feedback for which I am unremittingly grateful.

Thanks also to the many people and organizations that have provided me with time, information and ideas throughout my philanthropic career. A donor can accomplish nothing without a place to give, and I am grateful for each of you, even when I have directed funds elsewhere. Our field is in an interesting time at the moment, and is well-poised for significant new accomplishments. Let’s make it happen.

Engineers Without Borders earns “Notable” distinction

| by Holden

Engineers Without Borders is a group we’ve never encountered before, focused on “enabling rural Africans the opportunity to access clean water, generate an income from humble farms, and access critical infrastructure and services.” (Here’s a list of its projects in Africa.)

This group deserves recognition for doing the simplest thing guaranteed to improve GiveWell’s view of it: disclosing a failure.

Its 2009 and 2010 failure reports are publicly and prominently published. These aren’t “quasi-failures” along the lines of “Didn’t raise enough money for X”; they involve staff misreading their environments, both in terms of culture (poorly anticipating how a project’s timeline would be affected by holiday schedules) and in terms of interactions between aid institutions (nearly losing funding for a project by what reads to me like hurting a funder’s feelings). I encourage you to read the reports themselves rather than relying on our summary, since they are eye-opening.

Without knowing anything else about the work of Engineers Without Borders, we commend it for clearly and publicly discussing its own shortcomings, and thereby creating more public information on the complexities of aid.

Hat tip: Aid Watch

LAPO: Case study on due diligence by microfinance funders

(updated on: ) | by Holden

Updated 10/19/10 to reflect new information, submitted by a Grameen Foundation representative, regarding encouraging developments on LAPO since mid-year. To be clear, we stand by the main message of this post, which is not about LAPO’s current situation but about its funders’ and partners’ behavior over the last several years, prior to the public controversy that occurred in late 2009 and early 2010.

We’ve recently released research aiming to identify microfinance institutions (MFIs) with a strong focus on social impact. We have chosen to focus on finding individual MFIs largely because of our concerns about large microfinance-funding charities – specifically, that their due diligence seems focused on financial performance to the exclusion of social impact – i.e., on scale and revenue rather than effects on borrowers’ lives.

A controversy from earlier this year, over a Nigerian MFI called Lift Above Poverty Organization (LAPO), provides a good example of what we’re concerned about. LAPO has been funded and celebrated by many of the big names in microfinance, yet for years there have been many causes for concern about its social (as opposed to financial) performance. From what we’ve seen, it is not clear that these concerns have been on the radar screen of LAPO’s funders and partners.

LAPO’s funders/partners

LAPO’s funders/partners have included:

Controversy and reaction

In August 2009, MicroRate was the first to hint at concerns about LAPO, stating in a press release: “MicroRate notes that the integrity of the information provided to it by LAPO, as well as LAPO’s financial disclosures since the rating, have come into question. As a result, MicroRate’s rating of LAPO is no longer valid.” (MicroRate 2009)

In December 2009, Planet Rating released a “C+” rating report for LAPO that raised substantial concerns about LAPO’s legal licensing, governance, and data integrity, as well as noting an effective annual interest rate in excess of 100%. (Details below.)

In April 2010, The New York Times published an article citing the Planet Rating report on LAPO’s licensing issues and interest rates, while also noting the expiration of the MicroRate rating. Within weeks of this article’s running, both Kiva and MicroPlace had suspended their relationship with LAPO. (See Kiva’s page on LAPO, which discusses the suspension – Kiva’s loans through LAPO appear to have ended in early May – and Microplace’s discussion.) However, the Schwab Foundation award came after the article.

Update 10/19/10: between May 2010 and the present, several encouraging changes at LAPO are reported by a Grameen Foundation representative (via this comment and a followup email on the specifics of dates):

  • May 2010: LAPO “hired Deloitte and Touche to audit its 2010 financials and review the audit that was conducted in 2009.”
  • June 2010: LAPO “received its license from the Nigerian Central Bank and also hired a new Chief Financial Officer, with extensive experience in microfinance management, through the UNDP Africa Management Services Company (AMSCO).”
  • June-September 2010: LAPO “reconstituted its Board of Directors, which now comprises seasoned microfinance, banking and economics professionals from Nigeria and Benin.”
  • October 2010: LAPO “retained the services of consulting firm MicroFinance Transparency (headed by noted expert Chuck Waterfield) to review its interest rates and related policies.”

Below we discuss some details of the concerns, and why we feel they are relevant to our concerns about the due diligence done by LAPO’s funders/partners.

Forced savings and savings without the appropriate license

These issues were a major focus of the Times article, which stated:

    LAPO, considered the leading microfinance institution in Nigeria, engages in a contentious industry practice sometimes referred to as “forced savings.” Under it, the lender keeps a portion of the loan. Proponents argue that it helps the poor learn to save, while critics call it exploitation since borrowers do not get the entire amount up front but pay interest on the full loan.
    LAPO collected these so-called savings from its borrowers without a legal permit to do so, according to a Planet Rating report. “It was known to everybody that they did not have the right license,” Ms. Javoy said.

It appears to us that LAPO has been putting off getting the appropriate license for several years and that its funders have not held it accountable in this regard (though to be clear, it seems possible to us that this failure did not literally constitute breaking the law – we are not sure based on the information we have).

  • The 2005 MicroRate report stated that LAPO was planning to (and should be planning to) become licensed as a Microfinance Bank: “by law, [Community banks] will have to transform into a Microfinance Bank (‘MFB’) by December 2007 … As yet there is no deadline for the transformation of NGOs. However pressure from the central bank is expected and LAPO will have to transform sooner rather than later … The MFI is fully committed to doing so and plans are in place to convert into a private company by the proposed deadline” (page 3). Thus, at this point LAPO appears to have been targeting December 2007 for obtaining its license.

 

  • A letter from the Calvert Foundation concerning its investment in LAPO says the license is hoped for by year-end 2009: “we have been working with the Creditor Taskforce to encourage the transformation of LAPO into a depository institution regulated by the Central Bank as soon as possible. LAPO has received initial approval by the Central Bank for their application to transform into a ‘Microfinance Bank.’ Their goal is to secure the banking license by year-end 2009.“
  • The Planet Rating report, in December 2009, is clear that LAPO still did not have the license at that time, and that it planned to get one in January 2010, a plan that Planet Rating did not find realistic (Planet Rating 2009, Pg 7). Planet Rating stated that “LAPO does not have the appropriate legal structure to … disburse credit or collect savings … Although illegal, this has been so far tolerated by the [Central Bank of Nigeria]” (Pg 7).
  • LAPO still apparently did not have the license as of Kiva’s April 2010 update. This is the most recent discussion we can find of this issue.High interest rates

    The other point emphasized in the Times article is the high rates of interest charged by LAPO, which seem to contradict the stated goals of some of its partners:

      Under outside pressure, LAPO announced in 2009 that it was decreasing its monthly interest rate, Planet Rating noted, but at the same time compulsory savings were quietly raised to 20 percent of the loan from 10 percent. So, the effective interest rate for some clients actually leapt to nearly 126 percent annually from 114 percent, the report said. The average for all LAPO clients was nearly 74 percent in interest and fees, the report found.

    …

    Until recently, Microplace, which is part of eBay, was promoting LAPO to individual investors, even though the Web site says the lenders it features have interest rates between 18 and 60 percent, considerably less than what LAPO customers typically pay.

    …

    At Kiva, which promises on its Web site that it “will not partner with an organization that charges exorbitant interest rates,” the interest rate and fees for LAPO was recently advertised as 57 percent, the average rate from 2007. After The Times called to inquire, Kiva changed it to 83 percent.

    We don’t have much to add on this point. The Planet Rating report specified an effective annual interest rate of 123.9% (Planet Rating 2009, Pg 6).
    We have argued against reading too much into high interest rates, but funders and partners ought to be clear on what these rates are and whether the rates are consistent with their own values. We feel it is very important that anyone funding or partnering with an MFI do the full due diligence required to understand the true effective interest rate, from the beginning of the relationship.

    The two issues raised by the Times article – concern over LAPO’s license and over its interest rates – are both valid issues, and both issues could be easily identified years before the controversy came up.

    Other concerns

    We note other concerns about LAPO’s impact not mentioned in the Times article:

    • Integrity of governance and audits may be compromised by family relationships and other issues. The 2009 Planet Rating report states,
      • “Although the Memorandum of Association states that BOD [Board of Directors] members are to be reelected every year renewed every two years (three years for the chairman), all BOD members have been in the BOD for at least four years” (Pg 4)
      • “One of the [Board of Directors] members is related to the external auditor, creating a risk of lack of transparency. Family relations within the management team create another conflict of interests that have not yet been mitigated by appropriate policies” (Pg 7)
      • “External auditors are not sufficiently independent and do not have enough knowledge on the risks specific to microfinance” (Pg 10)
    • Data is unreliable.
      • “Loan tracking and accounting systems are not integrated and the system is prone to error” (MicroRate 2005, Pg 2)
      • The Planet Rating report stated that information management left room for mistakes and manipulation (Pg 8-9), and that “A sample of six branches by Planet Rating resulted in inconsistencies of up to a 6% difference in the amounts of PAR, arrears and number of clients (as of September 2009)” (Footnote 22). The report warned that “Due to insufficient data reliability, Planet Rating’s opinion on LAPO’s credit risk and credit risk coverage is subject to reserves” (Pg 11).
    • LAPO may lack the tools to assess, and create incentives based on, its social as opposed to financial performance. The Planet Rating report states:

      Group discipline is generally sufficiently ensured. However, for Regular Loans, the evaluation of the borrower’s capacity is not always complete and the actual use of the loan rarely formally monitored. Moreover, LAPO has not defined clear rules for the use of identification papers, which will be necessary to prevent multiple lending as the microfinance market matures and given the multiplication of MFBs … Moreover, the incentive system for Credit Officers mostly relies on their caseload, which creates a risk of excessive disbursements at the expense of portfolio quality. (Page 11)

     

  • High dropout rates. This is the issue that most worried us when we expressed concern about LAPO in December 2009. We cited its 49% dropout rate; as early as 2005, MicroRate stated, “client attrition remains unacceptably high at around 27%” (MicroRate 2005, Pg 5).Bottom line

    We aren’t sure whether/to what extent

    • LAPO’s funders/partners have been largely unaware of/indifferent to the concerns raised above (in some cases, possibly due to prioritizing financial over social returns).
    • LAPO’s funders/partners have been aware and concerned, but have had other, positive information on LAPO’s social impact that they have felt outweighs the concerns.
    • LAPO’s funders/partners have been aware and concerned, but have made a strategic decision to prioritize building sustainable, profitable financial institutions over focusing directly on social impact.

    We feel there is at least some evidence for the first possibility. Two partnerships were suspended in the immediate wake of the Times article, whose major concerns could easily have been identified years ago; and the only public record of due diligence we’re aware of, USAID’s discussion from 2007 (see page 5), discusses only financial/”efficiency” indicators, with no mention of concerns like those listed above.

    The possibility that social performance is essentially being overlooked seems strong and worrisome enough to us that, for the time being, we are more comfortable with the idea of giving directly to MFIs that are clearly focused on their social performance. We are open to changing this view, if and when major microfinance organizations become more open about what factors and concerns they are weighing and how they are conducting their due diligence.

    Sources

 

More on charity ratings and GiveWell’s mission

(updated on: ) | by Holden

Last week we wrote that we are likely going to stop giving zero-to-three-star ratings to charities. Some praised the decision, while others offered suggestions for how to resolve the problems we listed with ratings (we responded to the suggestions in the thread).

While we went into a lot of detail on the problems with ratings, I’ve realized that we didn’t talk enough about the benefits of ratings – and how those benefits don’t fit with GiveWell’s core audience and mission as we’re thinking about it now.

As we mentioned last week, we aren’t getting rid of our charity evaluations, or recommendations, or rankings of recommendations. We’re still going to disclose which charities we like best, and in what order. We’re still going to say what we think of each charity we’ve looked at, whether extensively or briefly.

What we’re getting rid of is the ability to get a quick, quantified rating of any charity we’ve looked at. In some ways that’s a big thing to lose; it’s arguably Charity Navigator‘s greatest strength in attracting broad interest and attention. A lot of the people I’ve talked to about GiveWell aren’t particularly interested in a charity recommendation; they want to know “how good” is the charity their friend is running a marathon for, or the charity that sends them mail. As the Money for Good study puts it, they’re seeking “to validate their donation, not to choose between organizations” (main study, page 40). And sometimes, they also seem to be excited by the possibility of a scandal, i.e., revealing that a particular charity is “bad.”

GiveWell’s mission has never been about serving these people. We seek to drive money to outstanding charities, and in so doing, to change incentives and allow some charities to raise money by doing demonstrably great work (instead of just by telling a great story).

We feel that if all you want from a charity evaluator is to check whether the charity that contacted you is “bad or OK,” you’ve already thrown away most of the opportunity to do as much good as possible with your donation.

For these reasons, our work has always been a poor fit for what a lot of donors want, and there has always been a tension between fulfilling our mission directly (which means focusing obsessively on charities we find promising and ignoring other charities) and doing things that might attract more attention and broaden our audience (such as publishing the star ratings that many want, or “digging up dirt” on big-name charities).

We originally introduced star ratings because we thought they would broaden our reach. The reactions we’ve gotten, particularly regarding how we rate the vast majority of charities that share no substantive information (the toughest aspect of designing an appropriate ratings system), have implied to us that there is actually very little to gain in this way. People who come to us for validation on their existing choice of charity are usually going to see that we don’t find the charity promising. As a result, rather than become interested in the work we do, they’re generally going to be disappointed and even upset.

In many ways, we’re better off making it clear from the beginning that we don’t have what these people are looking for, and focusing exclusively on the donors who do fit what we’re about: coming to their donation decisions with a lack of pre-commitments and an intent to give to the best charity possible.