Comments on: Thoughts from my visits to Small Enterprise Foundation (South Africa) and VillageReach (Mozambique), Part II http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/ Exploring how to get real change for your dollar. Thu, 17 May 2012 08:09:07 +0000 http://wordpress.org/?v=2.0.7 by: Geoff Murphy http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-148106 Wed, 28 Jul 2010 16:53:22 +0000 http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-148106 Hi Holden, You're not alone in the concern about labor diversion: "Is philanthropy killing business in Africa? http://chrisblattman.com/2010/07/28/is-philanthropy-killing-business-in-africa/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+chrisblattman+%28Chris+Blattman%29 Hi Holden,

You’re not alone in the concern about labor diversion:

“Is philanthropy killing business in Africa?

http://chrisblattman.com/2010/07/28/is-philanthropy-killing-business-in-africa/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+chrisblattman+%28Chris+Blattman%29

]]> by: Kimberly Vora http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-113250 Wed, 17 Mar 2010 00:32:07 +0000 http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-113250 Good points, Holden. Grameen has published a couple of case studies, but that's it. While Grameen makes it clear which countries have PPI tools available, it is not clear who is using the PPI and to what effect. I happen to know that 22 MFIs in Latin America are in some stage of using the PPI. But it's up to the individual organizations to decide what to do with the information coming out of the PPI. And that's if they get beyond the pilot into full implementation and regular usage. It's certainly not a top priority for all MFIs (that's who is mostly using the PPI as far as I know). This type of information isn't required by many investors or donors, and the MFI business case for it is being grasped slowly. (Side note, the credit boom that was channeled into microfinance - a cause celebre that fit the times exceedingly well - diluted financial and social lending standards and requirements. Investors did not want to be rejected by an MFI because their requirements were too onerous. That's what happens when your main yardstick of success is $$ lent - I think it was a real setback for the cause of social performance transparency). There is also the fear of revealing to the public that in reality only 30% of your clients are poor (or whatever the facts are), though I have seen at least a couple of organizations admit to less than stellar poverty targeting. So maybe the stigma about that is shrinking - at least one can hope. Stigma prevents change. On a related note, the Mix Market has a social performance report through which MFIs can choose to report on a number of issues, including poverty measurement along the lines of what the PPI generates. 165 MFIs filed those reports for 2008, and you can find some interesting information there. But most of the ones I checked do not report poverty measurements (though a few mentioned that they were testing or considering the PPI). There is no place to report the inputs of the PPI, I suspect because the goal was to have a single metric across countries. I agree with you that the PPI inputs may have potential for guiding the design and targeting of products and services. But in some cases, the inputs do not seem particularly useful in this way. For example, in the <a href="http://www.progressoutofpoverty.org/system/files/PPI_Design_Documentation_for_Bolivia_2009.pdf" rel="nofollow">updated Bolivia PPI Scorecard</a> the only indicators I see that organizations or donors might want to act on the are number of children in school and type of cooking fuel. If you want information about running water, sanitation, or nutrition, you'll have to look elsewhere in this case. The PPI is not one tool to rule them all (sorry, I'm reading LOTR right now). And I think that's for the best. The PPI does one thing - measuring poverty likelihood - and it does that simply and well, which makes it more likely to be adopted. Of course, if one doesn't like the income definition of poverty, or if that is not useful for targeting the intervention of choice, one may not be so happy about this tool. Anyway, I think progress has been made - 5 years ago these methods of collecting and sharing social performance information didn't exist. But there is a lot of work ahead to persuade organizations to do it. Good points, Holden. Grameen has published a couple of case studies, but that’s it. While Grameen makes it clear which countries have PPI tools available, it is not clear who is using the PPI and to what effect. I happen to know that 22 MFIs in Latin America are in some stage of using the PPI. But it’s up to the individual organizations to decide what to do with the information coming out of the PPI. And that’s if they get beyond the pilot into full implementation and regular usage. It’s certainly not a top priority for all MFIs (that’s who is mostly using the PPI as far as I know). This type of information isn’t required by many investors or donors, and the MFI business case for it is being grasped slowly. (Side note, the credit boom that was channeled into microfinance - a cause celebre that fit the times exceedingly well - diluted financial and social lending standards and requirements. Investors did not want to be rejected by an MFI because their requirements were too onerous. That’s what happens when your main yardstick of success is $$ lent - I think it was a real setback for the cause of social performance transparency). There is also the fear of revealing to the public that in reality only 30% of your clients are poor (or whatever the facts are), though I have seen at least a couple of organizations admit to less than stellar poverty targeting. So maybe the stigma about that is shrinking - at least one can hope. Stigma prevents change.

On a related note, the Mix Market has a social performance report through which MFIs can choose to report on a number of issues, including poverty measurement along the lines of what the PPI generates. 165 MFIs filed those reports for 2008, and you can find some interesting information there. But most of the ones I checked do not report poverty measurements (though a few mentioned that they were testing or considering the PPI). There is no place to report the inputs of the PPI, I suspect because the goal was to have a single metric across countries. I agree with you that the PPI inputs may have potential for guiding the design and targeting of products and services. But in some cases, the inputs do not seem particularly useful in this way. For example, in the updated Bolivia PPI Scorecard the only indicators I see that organizations or donors might want to act on the are number of children in school and type of cooking fuel. If you want information about running water, sanitation, or nutrition, you’ll have to look elsewhere in this case. The PPI is not one tool to rule them all (sorry, I’m reading LOTR right now). And I think that’s for the best. The PPI does one thing - measuring poverty likelihood - and it does that simply and well, which makes it more likely to be adopted. Of course, if one doesn’t like the income definition of poverty, or if that is not useful for targeting the intervention of choice, one may not be so happy about this tool.

Anyway, I think progress has been made - 5 years ago these methods of collecting and sharing social performance information didn’t exist. But there is a lot of work ahead to persuade organizations to do it.

]]> by: Holden http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-112870 Mon, 15 Mar 2010 17:46:31 +0000 http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-112870 <strong>Kimberly</strong>: we are familiar with the Progress out of Poverty Index, and in theory it sounds to us like a good tool. The issue is that we haven't seen charities sharing the output of such investigations, even when doing so was a clear condition of being considered for a decent-sized grant. It's possible that they aren't using the tool, or that they are but don't want to share the output. As a side note, I'm personally more interested in the <em>inputs</em> into the Progress out of Poverty Index than in the index itself. I.e., I would like to know whether a charity's clients have running water, electricity, healthy diets, etc. and find this disaggregated info more helpful than the final single number that comes out of it. (The case is somewhat analogous to that of the <a href="http://www.givewell.org/international/technical/additional/DALY" rel="nofollow">DALY</a> metric, although I believe the Progress out of Poverty Index is more meaningful and less opaque than the DALY.) <strong>Jonah:</strong><ul> <li>There are currently about 15 Seattle staff (Americans). <li>There are currently 2 major local staff, plus a translator/IT consultant. <li>The planned expansion will involve hiring one more local staff per province, to train Field Coordinators. <li>In each province VillageReach expects about 10-20k additional immunizations per year (details forthcoming). At about 15 immunizations per life saved (the approximation we use), that's 700-1500 lives saved per province per year. <li>Expansion will also involve other reallocation of government workers to roles as Field Coordinators, as <a href="http://www.givewell.org/international/Site-Visits-Feb-2010#Part17LeahHasselbeck" rel="nofollow">I discussed with Leah</a>. </ul> I'm not sure how much light all of this sheds on anything. I imagine that people vary extremely widely in the "expected good they would do otherwise" and that we have very little way of quantifying this. It does seem to me that VillageReach is more "lightweight" than most charities in terms of how many top-tier local staff they are employing, since most of what they are doing is trying to reorganize the public health system rather than put in new services from scratch. Kimberly: we are familiar with the Progress out of Poverty Index, and in theory it sounds to us like a good tool. The issue is that we haven’t seen charities sharing the output of such investigations, even when doing so was a clear condition of being considered for a decent-sized grant. It’s possible that they aren’t using the tool, or that they are but don’t want to share the output.

As a side note, I’m personally more interested in the inputs into the Progress out of Poverty Index than in the index itself. I.e., I would like to know whether a charity’s clients have running water, electricity, healthy diets, etc. and find this disaggregated info more helpful than the final single number that comes out of it. (The case is somewhat analogous to that of the DALY metric, although I believe the Progress out of Poverty Index is more meaningful and less opaque than the DALY.)

Jonah:

  • There are currently about 15 Seattle staff (Americans).
  • There are currently 2 major local staff, plus a translator/IT consultant.
  • The planned expansion will involve hiring one more local staff per province, to train Field Coordinators.
  • In each province VillageReach expects about 10-20k additional immunizations per year (details forthcoming). At about 15 immunizations per life saved (the approximation we use), that’s 700-1500 lives saved per province per year.
  • Expansion will also involve other reallocation of government workers to roles as Field Coordinators, as I discussed with Leah.

I’m not sure how much light all of this sheds on anything. I imagine that people vary extremely widely in the “expected good they would do otherwise” and that we have very little way of quantifying this. It does seem to me that VillageReach is more “lightweight” than most charities in terms of how many top-tier local staff they are employing, since most of what they are doing is trying to reorganize the public health system rather than put in new services from scratch.

]]> by: Kimberly Vora http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-112292 Fri, 12 Mar 2010 18:48:54 +0000 http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-112292 Grameen Foundation has already developed a tool that assesses standard of living based on easily identifiable household characteristics, the Progress out of Poverty Index (PPI). The tool has been developed for over 20 countries, with more on the way. It is meant to be used to a) help organizations measure the poverty levels of their clients, and thus adjust their products and targeting based on who they want to reach; and b) assess changes in those poverty levels over time (though in itself the PPI does not prove a causal relationship between program interventions and outcomes). Here's how it works: Starting with comprehensive national household surveys (which include lots of detailed information including household income) analysts identify the top ten indicators that correlate most closely with household poverty. This list varies from country to country, but may include things like whether the household has electricity, a latrine, owns land, has children in school, etc. The range of possible scores is then linked to a range of percentages indicating the probability that the household is poor (again, these probabilities are calculated based on the national household survey data set). The probabilities are determined for three categories: above the poverty line, top half below the poverty line, and bottom half below the poverty line. So basically you are looking at whether a household is likely to be nonpoor, poor, or very poor. This is made into a 10 question scorecard that is easily used by staff on the ground - on the spot they can tell how likely it is that the household falls above or below the national poverty line. It doesn't answer all our impact questions, but it provides great data for targeting, product design, and making educated guesses about what works for clients. I think it's one of the best things GF has done. See www.progressoutofpoverty.org for more information on how it works. Grameen Foundation has already developed a tool that assesses standard of living based on easily identifiable household characteristics, the Progress out of Poverty Index (PPI). The tool has been developed for over 20 countries, with more on the way. It is meant to be used to a) help organizations measure the poverty levels of their clients, and thus adjust their products and targeting based on who they want to reach; and b) assess changes in those poverty levels over time (though in itself the PPI does not prove a causal relationship between program interventions and outcomes). Here’s how it works: Starting with comprehensive national household surveys (which include lots of detailed information including household income) analysts identify the top ten indicators that correlate most closely with household poverty. This list varies from country to country, but may include things like whether the household has electricity, a latrine, owns land, has children in school, etc. The range of possible scores is then linked to a range of percentages indicating the probability that the household is poor (again, these probabilities are calculated based on the national household survey data set). The probabilities are determined for three categories: above the poverty line, top half below the poverty line, and bottom half below the poverty line. So basically you are looking at whether a household is likely to be nonpoor, poor, or very poor. This is made into a 10 question scorecard that is easily used by staff on the ground - on the spot they can tell how likely it is that the household falls above or below the national poverty line. It doesn’t answer all our impact questions, but it provides great data for targeting, product design, and making educated guesses about what works for clients. I think it’s one of the best things GF has done. See www.progressoutofpoverty.org for more information on how it works.

]]> by: James Edward Dillard http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-112145 Thu, 11 Mar 2010 22:55:00 +0000 http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-112145 Interesting point about the African locals and the jobs they're drawn to (by a number of forces, as you note). A friend of mine working on a project in Uganda found a particular piece of appropriate technology that was reliably paying great returns for farmers who used it to shell their coffee (he did a robust economic analysis to prove this), but when he spoke with several local leaders about getting a manufacturing facility set-up and focusing it on coffee, they declined, saying that NGOs were only interested in peanuts. It's only one story, but it seems similar to your experience. On the standard of living... it strikes me that it'd be pretty easy to train a group of college students to "ask the right questions" for standard of living and send them over to a part of the world, pay for their travel and have them survey a number of communities... heck, if you framed it correctly, you may even be able to get them to pay you to do it. Even if charities didn't publicize this information, it'd be really helpful for them to know if they were raising an areas standard of living (assuming one charity has that power, which I doubt they do). Interesting point about the African locals and the jobs they’re drawn to (by a number of forces, as you note).

A friend of mine working on a project in Uganda found a particular piece of appropriate technology that was reliably paying great returns for farmers who used it to shell their coffee (he did a robust economic analysis to prove this), but when he spoke with several local leaders about getting a manufacturing facility set-up and focusing it on coffee, they declined, saying that NGOs were only interested in peanuts. It’s only one story, but it seems similar to your experience.

On the standard of living… it strikes me that it’d be pretty easy to train a group of college students to “ask the right questions” for standard of living and send them over to a part of the world, pay for their travel and have them survey a number of communities… heck, if you framed it correctly, you may even be able to get them to pay you to do it. Even if charities didn’t publicize this information, it’d be really helpful for them to know if they were raising an areas standard of living (assuming one charity has that power, which I doubt they do).

]]> by: Jonah S. http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-111984 Thu, 11 Mar 2010 00:02:39 +0000 http://blog.givewell.org/2010/03/10/thoughts-from-my-visits-to-small-enterprise-foundation-south-africa-and-villagereach-mozambique-part-ii/#comment-111984 Hi Holden, Thanks for traveling to Africa to look into these matters - I've been reading your posts with interest. It's discouraging to hear that diversion of skilled labor looks to be a serious concern. What does the ratio (number of lives saved by VillageReach's vaccination efforts)/(number of local Village reach employees) look like? Hi Holden,

Thanks for traveling to Africa to look into these matters - I’ve been reading your posts with interest. It’s discouraging to hear that diversion of skilled labor looks to be a serious concern. What does the ratio (number of lives saved by VillageReach’s vaccination efforts)/(number of local Village reach employees) look like?

]]>