Continued from Part I, these are my thoughts from my recent visit to two of our top charities in Africa.
Diverting skilled labor looks like a real concern.
The COO of SEF stressed that one of SEF’s biggest challenges is human resources (i.e., continually finding good people to staff it). I can easily see how this would be. As I mentioned in Part I, I found that the nonprofits I visited were employing capable, impressive people with a combination of local background and well-above-average educational credentials and command of English.
On one hand, seeing these staffers made me feel good about the organizations we were recommending. At the same time, it highlighted one of the most universal and hardest to evaluate concerns we have about nonprofit work: diversion of skilled labor from other potentially productive pursuits.
Adding to this concern was a general impression I got (reinforced by Leah from VillageReach) that nonprofit jobs are among the best-paying and most prestigious jobs for African locals. It looks like we have a situation where:
- Many of the people hired by nonprofits could also be potentially very helpful to their communities if they were doing for-profit work.
- They work instead for nonprofits, partly because nonprofits are out-bidding the for-profits for their services.
- Within a for-profit framework, there is often (not always and never perfectly) a connection between the value of a job and the salary, which creates a (imperfect) tendency for talented people to end up in roles where they can do more good.
- I have no sense of how (or whether) nonprofits are attempting to calibrate salaries and value, and I fear that they could be “overpaying for” (and thus misusing) local talent simply because they want the best people available and they have the donor-supplied funds to get them.
More on this idea in a future post. Though we have no great methods for quantifying the losses from “diversion of labor,” we do believe that this concern reinforces the importance of demanding that nonprofits be accomplishing as much good as possible and not merely some good.
Getting basic info about people’s standard of living seemed fairly straightforward.
I understand that estimating people’s incomes can be a very complex endeavor, but in the areas I visited, it seemed possible to get a sense very quickly for how “poor” one area was relative to another. I asked basic questions at the village level: where the nearest water source was, who was responsible for maintaining it, where the nearest school was, what the school fees were, etc. I walked around and observed how many of the dwellings were made of mud vs. concrete . And when talking to individual clients, I asked straightforward questions like “Do you have a TV?”, “Do you have electricity?”, “What do you eat?” and “When was the last time you had a fever and what did you do about it?” Answers were fairly consistent in a given area, but varied dramatically across charities (more below).
Throughout our investigations into international aid, I’ve been frustrated by the fact that most charities seem either unable or unwilling to produce data on clients’ standards of living. Because I don’t tend to trust stylized stories, and I haven’t had what I consider credible data on standards of living, I’ve constantly felt very unclear on who is being helped and how. I now find it less likely that this problem stems from prohibitive costs of data collection; I find it more likely that it stems from (a) the fact that donors rarely (if ever) ask for data on clients’ standards of living; (b) the possibility that some charities may not want to reveal that their clients are anyone but the “poorest of the poor” (even when their clients are still quite poor).
The three areas I visited were very different in terms of standards of living.
- Small Enterprise Foundation (SEF) clients: I visited two villages, one in the Microcredit program (SEF’s original program) and one in the Tšhomišano Credit Program (targeted more directly at the poorer people in a village). In both villages, at least half the buildings I saw were made of concrete, and everyone I spoke to reported convenient access to running water, electricity, a fairly well-stocked local market, and public transportation to larger cities. Living spaces appeared fairly cramped (they were larger than in the other areas I visited, but when I asked who slept where it quickly became clear that there wasn’t much space per person); clients reported eating meat “only when they could afford it.”
- VillageReach clients: infrastructure was much, much worse in these areas. The town of Macomia, where we spent the night, had no running water and no electricity except for generators; it took hours to reach (in a truck) from Pemba, which I believe was the closest area with reliable electricity and running water. The one village we visited took over an hour (of alert driving on very bad roads) to reach from Macomia, and the only concrete structures I saw there were the health center, a closed shop, and the school. I was told that other nearby villages were even harder to reach (in some cases impossible in a truck) and that access to water was a major problem. In terms of both standard of living and life opportunities, these areas appeared fundamentally worse than SEF areas.
- Soweto: I took a quick tour through a poor area of Soweto (urban). It was generally filthy (literally strewn with trash) and extremely crowded, with tiny steel shacks next to each other. It seemed to me like a much more unpleasant place to live than either of the other two areas, although on the flip side, people in Soweto appeared to have access to public transportation, electricity, good schools, etc. as they were very close to much wealthier residences.
One of the reasons Small Enterprise Foundation stood out to us is that it appears more diligent about targeting the poor than other organizations. Even so, its clients – while poor – appear to be substantially better off (in fundamental infrastructure-related ways, not ways that can be attributed to program effects) than VillageReach’s clients. This doesn’t make me less supportive of SEF (it’s largely consistent with my existing suspicion that microfinance clients are rarely if ever the poorest of the poor), but it’s an important thing to keep in mind that I feel better informed about now than before.
Are you looking to help people in the worst situation, and with the most basic needs, possible? Or are you interested in helping people who are better off to begin with, in the hopes that a little assistance might go a longer way with them? To me there’s no clear right answer, but it’s a decision donors are likely making constantly without knowing it.
More thoughts coming in Part III.
Comments
Hi Holden,
Thanks for traveling to Africa to look into these matters – I’ve been reading your posts with interest. It’s discouraging to hear that diversion of skilled labor looks to be a serious concern. What does the ratio (number of lives saved by VillageReach’s vaccination efforts)/(number of local Village reach employees) look like?
Interesting point about the African locals and the jobs they’re drawn to (by a number of forces, as you note).
A friend of mine working on a project in Uganda found a particular piece of appropriate technology that was reliably paying great returns for farmers who used it to shell their coffee (he did a robust economic analysis to prove this), but when he spoke with several local leaders about getting a manufacturing facility set-up and focusing it on coffee, they declined, saying that NGOs were only interested in peanuts. It’s only one story, but it seems similar to your experience.
On the standard of living… it strikes me that it’d be pretty easy to train a group of college students to “ask the right questions” for standard of living and send them over to a part of the world, pay for their travel and have them survey a number of communities… heck, if you framed it correctly, you may even be able to get them to pay you to do it. Even if charities didn’t publicize this information, it’d be really helpful for them to know if they were raising an areas standard of living (assuming one charity has that power, which I doubt they do).
Grameen Foundation has already developed a tool that assesses standard of living based on easily identifiable household characteristics, the Progress out of Poverty Index (PPI). The tool has been developed for over 20 countries, with more on the way. It is meant to be used to a) help organizations measure the poverty levels of their clients, and thus adjust their products and targeting based on who they want to reach; and b) assess changes in those poverty levels over time (though in itself the PPI does not prove a causal relationship between program interventions and outcomes). Here’s how it works: Starting with comprehensive national household surveys (which include lots of detailed information including household income) analysts identify the top ten indicators that correlate most closely with household poverty. This list varies from country to country, but may include things like whether the household has electricity, a latrine, owns land, has children in school, etc. The range of possible scores is then linked to a range of percentages indicating the probability that the household is poor (again, these probabilities are calculated based on the national household survey data set). The probabilities are determined for three categories: above the poverty line, top half below the poverty line, and bottom half below the poverty line. So basically you are looking at whether a household is likely to be nonpoor, poor, or very poor. This is made into a 10 question scorecard that is easily used by staff on the ground – on the spot they can tell how likely it is that the household falls above or below the national poverty line. It doesn’t answer all our impact questions, but it provides great data for targeting, product design, and making educated guesses about what works for clients. I think it’s one of the best things GF has done. See http://www.progressoutofpoverty.org for more information on how it works.
Kimberly: we are familiar with the Progress out of Poverty Index, and in theory it sounds to us like a good tool. The issue is that we haven’t seen charities sharing the output of such investigations, even when doing so was a clear condition of being considered for a decent-sized grant. It’s possible that they aren’t using the tool, or that they are but don’t want to share the output.
As a side note, I’m personally more interested in the inputs into the Progress out of Poverty Index than in the index itself. I.e., I would like to know whether a charity’s clients have running water, electricity, healthy diets, etc. and find this disaggregated info more helpful than the final single number that comes out of it. (The case is somewhat analogous to that of the DALY metric, although I believe the Progress out of Poverty Index is more meaningful and less opaque than the DALY.)
Jonah:
I’m not sure how much light all of this sheds on anything. I imagine that people vary extremely widely in the “expected good they would do otherwise” and that we have very little way of quantifying this. It does seem to me that VillageReach is more “lightweight” than most charities in terms of how many top-tier local staff they are employing, since most of what they are doing is trying to reorganize the public health system rather than put in new services from scratch.
Good points, Holden. Grameen has published a couple of case studies, but that’s it. While Grameen makes it clear which countries have PPI tools available, it is not clear who is using the PPI and to what effect. I happen to know that 22 MFIs in Latin America are in some stage of using the PPI. But it’s up to the individual organizations to decide what to do with the information coming out of the PPI. And that’s if they get beyond the pilot into full implementation and regular usage. It’s certainly not a top priority for all MFIs (that’s who is mostly using the PPI as far as I know). This type of information isn’t required by many investors or donors, and the MFI business case for it is being grasped slowly. (Side note, the credit boom that was channeled into microfinance – a cause celebre that fit the times exceedingly well – diluted financial and social lending standards and requirements. Investors did not want to be rejected by an MFI because their requirements were too onerous. That’s what happens when your main yardstick of success is $$ lent – I think it was a real setback for the cause of social performance transparency). There is also the fear of revealing to the public that in reality only 30% of your clients are poor (or whatever the facts are), though I have seen at least a couple of organizations admit to less than stellar poverty targeting. So maybe the stigma about that is shrinking – at least one can hope. Stigma prevents change.
On a related note, the Mix Market has a social performance report through which MFIs can choose to report on a number of issues, including poverty measurement along the lines of what the PPI generates. 165 MFIs filed those reports for 2008, and you can find some interesting information there. But most of the ones I checked do not report poverty measurements (though a few mentioned that they were testing or considering the PPI). There is no place to report the inputs of the PPI, I suspect because the goal was to have a single metric across countries. I agree with you that the PPI inputs may have potential for guiding the design and targeting of products and services. But in some cases, the inputs do not seem particularly useful in this way. For example, in the updated Bolivia PPI Scorecard the only indicators I see that organizations or donors might want to act on the are number of children in school and type of cooking fuel. If you want information about running water, sanitation, or nutrition, you’ll have to look elsewhere in this case. The PPI is not one tool to rule them all (sorry, I’m reading LOTR right now). And I think that’s for the best. The PPI does one thing – measuring poverty likelihood – and it does that simply and well, which makes it more likely to be adopted. Of course, if one doesn’t like the income definition of poverty, or if that is not useful for targeting the intervention of choice, one may not be so happy about this tool.
Anyway, I think progress has been made – 5 years ago these methods of collecting and sharing social performance information didn’t exist. But there is a lot of work ahead to persuade organizations to do it.
Hi Holden,
You’re not alone in the concern about labor diversion:
“Is philanthropy killing business in Africa?
http://chrisblattman.com/2010/07/28/is-philanthropy-killing-business-in-africa/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+chrisblattman+%28Chris+Blattman%29
Comments are closed.