The GiveWell Blog

Helping people: Easy or hard?

I wonder how much of the difference between our approach to charity and others’ approach comes from this very simple fact:

We think that improving people’s lives is really hard.

It might be easy to brighten someone’s day, even their week. But to get someone from poverty and misery to self-sufficiency and a world of opportunity … you have to change not just their resources, but their skills, often their attitude, and always their behavior. You can try to do this early in life or late; either way, it’s an uphill battle.

As we’ve been over and over, that’s why we think “money spent” is a poor proxy for “good accomplished,” and tend not to get as excited as others over dollars raised or dollars spent. But as we get deeper into reading apps, I’m finding there’s more and more to this rift.

I’m very skeptical of any program that claims great effects with relatively low amounts of intervention, whether it’s a one-time class on condom use or a once-a-week tutoring session. I think about how easy it is for the people I know to sit through some class, walk out full of ideas, and forget them a week later, and I think – if you’re doing anything meaningful for people with this little investment, you must be some sort of sorceror.

I trust survey data about as far as I can throw it (note: data cannot be thrown). Katya’s post today gives a sniff of the argument that “people are notoriously bad predictors of their behavior”; there’s actually an enormous amount of literature out there ramming home this point (if you really want to see it, let me know). We’ve seen a lot of survey data along the lines of “97% of participants felt the program improved their ability to ___,” which would be great if wishes were horses and horses were great.

I tend to feel the same way about anecdotes and personal experiences. They can be useful to get a picture of a program, but as evidence that it works? People are easy to change in the moment and incredibly hard to change in the long run. How can you possibly tell from watching a group of children for a year that they’re headed for better lives?

And in the end, though I keep reminding myself that evidence of effectiveness can come in all different forms, I can’t keep down that longing for randomized studies, or at least studies that employ some sort of comparison group. These are very rare in the applications we’re receiving: charities tend to look only at their clients. While this clearly saves them a lot of money and hassle, it leaves me wondering whether they’re really helping people … or just picking out the ones who are going to succeed anyway (you’re probably familiar with this question in the context of Ivy League schools). I’m sure this is heresy, but if we started a Placebo Foundation that sought out the “most motivated” poor people as clients, did absolutely nothing for them, and then examined their outcomes, would we find the same “successes” that many charities claim?

And yet every study I’ve read from applicants – even when limited to survey data, even when lacking a comparison group of any kind – concludes that the program in question was a success.

Unless there’s willful deception going on here, this implies to me that the people who work at charities think helping people is really, really easy; that there’s no need to worry about all the questions above; that the flimsiest and most perfunctory of evidence is good enough to walk away from feeling that people in need have been truly helped.

Maybe we’re wrong. Maybe helping people is that simple. If that’s what you think, keep writing those checks to whomever sends you mail, and make sure they aren’t blowing a penny more than they have to on salaries. If you’re concerned as I am, though, I’ve got some tough news from you: aside from GiveWell, I don’t think you have much company.

Comments

  • Laura Wutzke-Lucas on December 20, 2007 at 5:20 pm said:

    I doubt if you will read my reply. But I must comment on the fact that you have left out a lot of organizations in your analysis because they didn’t fit your narrow parameters. I understand the benefit of cost analysis, I really do, but there are strong arguments that this method doesn’t accurately evaluate the services you are talking about. Also, I worry what impact givewell’s list will have on giving to those organizations that aren’t on your list. Charities are diverse and I would argue that charitably given money should be diversely given. I don’t think this is necessarily best reflected when givewell chooses only a dozen or so qualifying organizations. Also, I have to say I think your organization’s underlying real-world knowledge about charitable endeavours, especially those in Africa, is seriously lacking. And finally, I wonder if you have considered WHY people give. I believe its not always b/c a charity is most cost effective but b/c of our personal connections to the people involved in that charity or being helped by that charity. And though the wealthy may give large endowments and take solace in your work, I think the average small giver like me (1000 or so a year) looks at the charities they give to in a much more complex, personal way.

  • I am finishing up a Ph.D. in applied economics, and the issue of impact analysis is a big one right now for larger government-run programs. Many of the journals in my field are packed with impact evaluations relating to specific programs, and larger organizations such as the World Bank and governments such as that in Mexico (and even, recently, that of New York) are building evaluations into their programs. Of course, as you suggest, there is still a positive bias in terms of outcomes, but occasionally a program will be deemed a failure, such as on http://econ.worldbank.org/external/default/main?pagePK=64165259&piPK=64165421&theSitePK=469372&menuPK=64216926&entityID=000094946_03051404103233

    Thanks for applying the same standards to charities, and I hope that one day maybe governments will make available matching grants or something so that all of these programs can be evaluated and thereby improved. In the meantime, we’ll have to keep working with whatever information can be provided!

  • “I hope that one day maybe governments will make available matching grants or something so that all of these programs can be evaluated and thereby improved”

    Government does do this already, and has been doing this for decades. “All” programs are not evaluated, because it would be enormously expensive. But it would be great if GiveWell donated money for this sort of research.

Comments are closed.