Comments on: Agriculture charity evaluation: incomes boosted are not the same as lives changed http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/ Exploring how to get real change for your dollar. Thu, 17 May 2012 07:15:04 +0000 http://wordpress.org/?v=2.0.7 by: Bryan http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-116896 Mon, 29 Mar 2010 23:44:54 +0000 http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-116896 Analytically weak. There are troves of information about the Millennium Villages out there and that givewell has decided to base their evaluation on a couple 600 opeds with obvious bias is unfortunate. Though this is somewhat dated for those of you who don't like reading: http://video.google.com/videoplay?docid=-2617281089086064784&ei=KDqxS9amFZ6yqQLL1KHiBg&q=millennium+villages+cross+cutting+initiatives&view=3# Otherwise, check out the ODI report, the peer reviewed stuff and the real literature on this. Don't hold things to unreasonable burdens of proof without doing any substantive research into this, that's not productive or fair. Analytically weak. There are troves of information about the Millennium Villages out there and that givewell has decided to base their evaluation on a couple 600 opeds with obvious bias is unfortunate.

Though this is somewhat dated for those of you who don’t like reading: http://video.google.com/videoplay?docid=-2617281089086064784&ei=KDqxS9amFZ6yqQLL1KHiBg&q=millennium+villages+cross+cutting+initiatives&view=3#

Otherwise, check out the ODI report, the peer reviewed stuff and the real literature on this. Don’t hold things to unreasonable burdens of proof without doing any substantive research into this, that’s not productive or fair.

]]> by: Holden http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-96604 Wed, 06 Jan 2010 00:03:47 +0000 http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-96604 Sam, speaking off the cuff, I'd say the types of evidence we've seen collected fall into the following broad categories:<ul> <li>Agricultural output (as discussed in this post) <li>"Income," sometimes inferred using a relatively complex methodology, in PPP-adjusted dollar terms. <li>Standard-of-living and quality-of-life measures, which can include asset accumulation but also access to water, enrollment of children in school, health & nutrition measures, and more (example: <a href="http://www.grameen-info.org/index.php?option=com_content&task=view&id=23&Itemid=126" rel="nofollow">Grameen Bank's 10 Indicators</a>).</ul> The last category is the one we find most compelling. Sam, speaking off the cuff, I’d say the types of evidence we’ve seen collected fall into the following broad categories:
  • Agricultural output (as discussed in this post)
  • “Income,” sometimes inferred using a relatively complex methodology, in PPP-adjusted dollar terms.
  • Standard-of-living and quality-of-life measures, which can include asset accumulation but also access to water, enrollment of children in school, health & nutrition measures, and more (example: Grameen Bank’s 10 Indicators).

The last category is the one we find most compelling.

]]> by: Sam Lee http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-95646 Fri, 01 Jan 2010 23:42:43 +0000 http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-95646 Holden, On economic empowerment long term success metrics, in addition to typical income gain, and asset accumulation that you prefer, what other indicators have you found be used in practice? Holden,

On economic empowerment long term success metrics, in addition to typical income gain, and asset accumulation that you prefer, what other indicators have you found be used in practice?

]]> by: Holden http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-83614 Tue, 27 Oct 2009 14:23:25 +0000 http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-83614 Ian, I think it's hard to tell whether Millennium Villages has had "success" in any area, and harder to break it down as you describe. I have much more confidence that lives were improved if I see asset accumulation than if I see a one-time bump in incomes/crop yields, though less confidence than if I see a major health improvement (particularly a life-saving one) or - as you say - a long-term, sustained improvement in living standards. Ian, I think it’s hard to tell whether Millennium Villages has had “success” in any area, and harder to break it down as you describe.

I have much more confidence that lives were improved if I see asset accumulation than if I see a one-time bump in incomes/crop yields, though less confidence than if I see a major health improvement (particularly a life-saving one) or - as you say - a long-term, sustained improvement in living standards.

]]> by: Ian Turner http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-83345 Sat, 24 Oct 2009 23:07:14 +0000 http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-83345 Holden, Suppose that you take someone who is earning $150 per year and give them an extra $100 per year in cash for five years. Wouldn't you expect them to build up assets every year during those five years? But in the sixth year, when you stop giving, this person will survive on $150 plus whatever assets they saved in the prior five years. By the time you get to year 15, is there any reason to think that the person's life will be much better off than it was in year zero? Perhaps, but it's far from obvious to me. My problem with the millennium villages is that it's impossible to tell what part of their success comes from injecting $100 per person into the local economy, and what part of their success comes from improving crop yields, reducing malaria, etc. Cheers, --Ian Holden,

Suppose that you take someone who is earning $150 per year and give them an extra $100 per year in cash for five years. Wouldn’t you expect them to build up assets every year during those five years? But in the sixth year, when you stop giving, this person will survive on $150 plus whatever assets they saved in the prior five years. By the time you get to year 15, is there any reason to think that the person’s life will be much better off than it was in year zero? Perhaps, but it’s far from obvious to me.

My problem with the millennium villages is that it’s impossible to tell what part of their success comes from injecting $100 per person into the local economy, and what part of their success comes from improving crop yields, reducing malaria, etc.

Cheers,

–Ian

]]> by: Holden http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-83018 Thu, 22 Oct 2009 22:37:36 +0000 http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-83018 Ian, it depends somewhat on what is meant by "sustainability." I agree that your test is the only one for long-term impact of a project on a community. However, I believe we can fairly infer a major change in an individual's life from less. Asset accumulation could be misleading in many ways, but I believe the measure has fewer pitfalls than the measures discussed in this post. Ian, it depends somewhat on what is meant by “sustainability.” I agree that your test is the only one for long-term impact of a project on a community. However, I believe we can fairly infer a major change in an individual’s life from less. Asset accumulation could be misleading in many ways, but I believe the measure has fewer pitfalls than the measures discussed in this post.

]]> by: Ian Turner http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-82865 Wed, 21 Oct 2009 18:32:55 +0000 http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-82865 Also, regarding sustainability, there is really only one measure that you can use to find out if a project is making a sustainable difference: Withdraw the funds and see if the subject thrive or wither. Even checking to see if roofs are built doesn't tell you whether or not they will be maintained. These facts are unfortunate, partly because withdrawing funds is hard, and partly because it takes years before you get usable data. Also, regarding sustainability, there is really only one measure that you can use to find out if a project is making a sustainable difference: Withdraw the funds and see if the subject thrive or wither. Even checking to see if roofs are built doesn’t tell you whether or not they will be maintained.

These facts are unfortunate, partly because withdrawing funds is hard, and partly because it takes years before you get usable data.

]]> by: Ian Turner http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-82864 Wed, 21 Oct 2009 18:24:20 +0000 http://blog.givewell.org/2009/10/21/agriculture-charity-evaluation-incomes-boosted-are-not-the-same-as-lives-changed/#comment-82864 Regarding the Millenium Villages, it should be noted that the project is adding roughly $100 (*) per person per year to villages' local economy. Note that GDP per capita in these villages would be somewhere around $150 (*) per year, again raising the specter of how these changes would compare to just handing out cash. (*) Note that figures named in this comment are measured at market exchange rates, not PPP. Regarding the Millenium Villages, it should be noted that the project is adding roughly $100 (*) per person per year to villages’ local economy. Note that GDP per capita in these villages would be somewhere around $150 (*) per year, again raising the specter of how these changes would compare to just handing out cash.

(*) Note that figures named in this comment are measured at market exchange rates, not PPP.

]]>