David Roodman is a research fellow at the Center for Global Development and author of David Roodman’s Microfinance Open Book Blog, where he his sharing his notes on (and content of) a book on microfinance.

We have consistently found his discussions to be evenhanded, thorough, and clear, especially in terms of what should and shouldn’t be considered evidence of impact. (See, for example, his article on evidence for microfinance’s impact, as well as his accessible discussion of research on the “macro-level” effects of international aid).

I spoke with him about what to look for in a microfinance charity. As we do, Mr. Roodman appears to see microfinance less as a tool to expand businesses than as a basic tool of empowerment that helps people manage their financial lives. He sees risks in lending – even when repayment rates are high (in fact there may be such a thing as “too high” repayment rates) and is interested in greater expansion of services to help the poor not just borrow but save money.

A rough transcript (with minor edits after the fact) follows.

---

GiveWell: What would you look for in a microfinance organization if you were trying to find one to donate to?

David Roodman: I would look for an organization that has a track record in innovation and scaling up successes. I would also look for a learning organization.

Microcredit has gone very far and has much farther to go. It also has certain risks which are pretty intuitive to us these days. And so a well-recognized challenge is, “how can we push ahead more on other financial services, especially savings?”

I would be interested in using my money to support the creation of institutions that can hold people’s savings and earn their trust. That attracts me because it doesn’t create the same risks that you have with credit. But it can be less attractive for a donor because what’s so attractive about credit is that you can see that you’re giving your money directly to beneficiaries; the middleman almost disappears. With savings it’s very different because “your” money isn’t going to clients – it’s about building an institution that can take other people’s money and hold it. It could be that your money is being used for very prosaic things.

GiveWell: Isn’t it a similar case with loans? The loans themselves are usually getting paid back with interest, but a lot of the costs of making loans are in building institutions. Even if you earmark your money “for loans,” it could be fungible with an organization’s other funds.

David Roodman: Roughly speaking, sure. Money is fungible, and so in most microcredit organizations it’s hard to pin down exactly what support is being used for.

GiveWell: You said you’d look for an organization that has innovated and scaled up successes. What exactly do you mean by success? You’ve written that success in the sense of social impact can be very hard to evaluate and there aren’t any MFIs that have conclusively demonstrated that kind of success.

David Roodman: This raises a couple of big issues about what you mean when you say something works. In the really big picture most poverty reduction has occurred as part of an economic growth process, and economic growth is not just “things getting bigger” – it’s really a societal transformation. There was a commission funded by the World Bank to write on the causes of economic growth, and the commission more or less concluded that no one knows the right recipe [http://growthcommission.org].

That means that, by and large, we don’t know how to explain most poverty reduction. We don’t know how to engineer all the poverty reduction that we want. If we don’t know what causes development, how do we act? That’s the question you’re confronting directly and very admirably. One approach is to focus on an activity where the benefit is immediate and powerful, as with vaccinations or agricultural technology.

I sometimes resort to theories about how development has happened. One of them is Amartya Sen on freedom as development: the essence of development is increasing control over one’s circumstances. From this point of view democracy is development; higher education is development; these different kinds of freedom are mutually supporting. If you can give people some kind of empowerment in any one aspect of their lives – whether education, health or income – it will probably spill over into other aspects of their lives. So even though I can’t prove to you that microfinance is raising growth rates 10 years later, if I can show you that it is giving people more control over how they manage money, there are some reasonable grounds for hope that it will contribute to these other goals.

For me, economic empowerment is a broader notion than you’re using right now. You’re free to define terms however you want. If you want to define the term literally, it’s about power over one’s circumstances. One thing I got from Portfolios of the Poor and its spiritual predecessor The Poor and Their Money is that the income of the very poor is not just low but unpredictable and volatile. And on the spending side they’ve got more uninsured risks. Their economic lives are just far less predictable.

Arguably the poor need financial services more than the rich. Financial services are inherently about that. In that sense, savings and credit and insurance are inherently about economic empowerment. It’s not the kind of economic empowerment where the person rises out of poverty through microentrepreneurship, but it gives them some more control over their lives.

The other theory is about development as institution building. What has made the US such a rich country over the past 200 years is its capacity for the development of new institutions. We don’t get stuck in a rut for very long. What’s most important to focus on in microfinance is building long-lasting, large, innovative institutions that are hopefully competing with one another and that are therefore somewhat accountable to the customer, that are into the development of new ideas, and that are upending established ways of doing things.

I think it’s important to think through these kinds of stories of how development happens, and there may be others. There’s the “private sector is the key” story. William Easterly says we should look for “searchers,” people looking for something that works. He concludes that Muhammad Yunus is such a searcher.

At the end of the day it really is hard for one person coming from the outside with a small amount of money to permanently reduce people’s poverty. Building innovative self-sustaining institutions is arguably pretty good by the standards of most public and private aid.

GiveWell: Microfinance charities often point to traditional metrics such as the repayment rate, average loan size, and number of loans made as evidence that they’re having impact. Do you feel these metrics, by themselves, constitute strong evidence of impact?

David Roodman: In the case of microcredit, we need more information. We need more information about the people who drop out. That’s a very hard thing to pin down. How many people are leaving the program each year, and why are they leaving? I don’t know if that can be pinned down.

MixMarket reports portfolio-at-risk figures, which measure default. But there may be such a thing as too high a repayment rate. It can indicate that the creditor has so much power over the borrower that they’re able to extract full repayments under dire circumstances. If you were to look at similar numbers for American banks, repayment rates would be a bit lower.

With savings and insurance, I’d be less concerned about judging by the sorts of numbers you usually see. Still, there are some questions there: for example, there might be a system that looks good on paper, but perhaps on the ground there’s so much paperwork that things are never processed or that savings are embezzled.

GiveWell: Where does the goal of profitability fit in? Is profitability or sustainability evidence that an institution is having a positive impact on people’s lives?

David Roodman: In general, profitability seems like a good thing. It means that donations are leveraged.

On the other hand, there’s always the question of whether microfinance institutions are charging too much. There’s research suggesting that if you charge even a tiny sum for a bednet that that will reduce use significantly—in other words, it might be a good thing for the bednet business to take a loss covered by subsidies. But credit is something people seem pretty willing to pay for, so the market test is more appropriate

It is case by case. When do we trust people to exercise good judgment about what they should be spending their money on? Cigarette companies are profitable.

I have somewhat lowered expectations about what you can do with microfinance, relative to the claims that are often made about it, and I’m largely interested in giving people more control over their lives. But if I’m going to give people more control over their lives, then I want to know who these people are. And it’s hard to get a lot of information about that.

Procredit is a group of banks operating in 20-30 countries. The headquarters are in Germany; the group does both savings and credit; it operates in some pretty difficult places like the DRC; it’s very much in the business of finance for ordinary people. It may not be reaching the poorest of the poor, but its philosophy is that the best way to make a contribution is to build self-sustaining businesslike banks that do both savings and credit.

GiveWell: We’ve found that we often can’t get much specific information about the demographics of clients, but perhaps there are times when it can safely be assumed that clients are low-income based on the region. Do you think that’s true?

David Roodman: If you’ve got an organization in Laos, you can figure that the people you’re reaching are pretty low-income. The same is true for most of Africa outside of South Africa. If they’re reaching some respectable scale, if they’re not just getting it to the top landowners in the village.

That said, Congress passed a law here some years ago requiring that half of U.S. microfinance aid reach people below the poverty line of about $1/day (purchasing power parity adjusted – a very low poverty line). That created quite a large demand for information about the demographics of borrowers so I think that there is more and more of that kind of information.

GiveWell: Your thoughts on other areas of economic empowerment focused charity, such as agricultural programs?

David Roodman: I don’t know much about these areas. I worry about when a charity starts deciding what recipients need – why not just give them the money and let them decide? This always should be a question in the background, and does actually point to virtue of microfinance, which helps people manage money for their own goals.

One reason we don’t give them the money is that we hope there’s some leverage point where we can multiply the value of our charity. But as soon as you get into distribution of goods, you’re not multiplying value of your charity.

GiveWell: What about development of agricultural technology?

David Roodman: This is an area I’m not very familiar with. In Africa, there isn’t a good track record. If you look at the developing world as a whole, there have been huge gains in crop yields, in part due to philanthropy. Not without negative side effects.

GiveWell: Is there any particular area of economic empowerment focused charity, outside of microfinance, that you find particularly promising?

David Roodman: Support for medium enterprises in intriguing because it is not looked at much. I haven’t looked at it much. Enterprises that are a bit larger are important. In the end, most very poor entrepreneurs are entrepreneurs out of necessity. They want jobs. And somewhat larger enterprises create them as they grow.

GiveWell: The way we’re leaning is that if we can tell that an institution is working with people in need, from there what we’d most like to see is a strong track record of starting self-sustaining initiatives; another thing we’d like to see is social impact over time (even if not demonstrated with great rigor). Probably third on our list is a straight transfer of wealth to demonstrably low-income people.

That all sounds very reasonable.

I see Yunus as a Henry Ford type figure. He didn’t really invent microfinance, but what he did amazingly well was experiment and develop business solutions to the problem of how an institution can provide financial services to the poor without losing its shirt. He created a large-scale business. A lot of what characterizes modern microcredit comes from wanting to control costs. Part of what group credit is about is shifting a lot of the bankers’ jobs onto the borrowers, including collecting info about who the borrowers are and how well-off they are. It’s a characteristic of the business to not know clients very well. Asking for information about clients means asking for a business model that has higher transaction costs and must therefore charge higher interest rates or serve richer people. Perhaps charitable funds can help cover that cost, but it’s a difficult tradeoff.

GiveWell: One of our leading contenders for a grant right now is the Village Enterprise Fund. It focuses not on loans but on grants, along with general advice and mentoring. It serves extremely low-income people (below the poverty line) and regularly and systematically collects data about their standards of living (it doesn’t attempt to gauge “income” but instead looks at quality-of-life factors such as access to clean water.) It has a recent study showing major changes in standards of living over time, although there is no comparison group.

David Roodman: Sounds reasonable. It sounds like a high transaction cost operation.

I find it refreshing that the focus is on grants. With all the talk of credit and savings, giving people money and oversight and advice is a refreshing change.

There’s always potential for selection bias in a study like the one you mentioned. It’s possible that clients come to the program as they’re about to succeed for other reasons.

The kind of standard-of-living assessment they’re doing is increasingly common in microfinance as well. In fact I think it may even be the norm. There’s a guy named Mark Schreiner who has developed scoring formulas that people can quickly apply to estimate “income” based on other quality of life measures.

GiveWell: Do you know where to get that data?

David Roodman: Check microfinance.com. The reports are called Poverty Scorecards.

GiveWell: Who else would you suggest that we talk to?

David Roodman: One person I would recommend is a colleague of mine, Vijaya Ramachandran, who has done a lot of primary research on small enterprises in Africa and what helps them grow or prevents them from growing. Another person who comes to mind is Dennis Whittle of GlobalGiving, which is largely about “letting the market choose” in philanthropy.

In general, it sounds like you’re looking for something that you can have high confidence in. There’s another approach one can take, the venture capital approach of taking risks with big potential payoffs. That might make more sense for much larger pools of funds, though, as opposed to giving away $250,000 and making recommendations to individual donors.

GiveWell: In addition to microfinance, you seem to be interested in the general topic of methodology in the social sciences. You’ve written a couple of papers that were very focused on methodology, which I liked. So I thought I would get your thoughts on a general pattern I’ve noticed in my reaction to impact studies.

I’ve found that whenever I’m looking at a case-control study that is only based on a single point in time – comparing clients of a program to non-clients, at a single time – I am almost always extremely skeptical, very worried about selection bias, and it’s very hard for the paper to make me accept the result. However, when I look at a study that examines changes over time, when the changes are big and coincide with the implementation of the program, I’m much more likely to see this as suggestive evidence of impact – even without a control group. (The Village Enterprise Fund study, for example.) Reflecting on this pattern, I think it’s reasonable to have this bias, but what are your thoughts?

David Roodman: I’m not sure how to give a general answer. I think you understand: if there’s a modest change over time, then there could be all sorts of alternative explanations for the change. If it’s an extraordinarily large change, then yes, it becomes harder to explain away. But it still can be selection bias. Perhaps the people who were about to succeed were the ones who found the program.

There are some large-scale improvements in health that have been attributed to vaccines and to ORT. One could argue that we don’t know whether this impact is real or coincidental, but the change over time is just so huge that it seems like it has to be real.

All in all, I think you have a good point. A given person is more like himself/herself (at a different point in time) than like someone else (at the same point in time). This makes changes over time harder to explain away.