The GiveWell Blog

Two questions about education

| by Holden

Researching Cause 4 has been frustrating. On one hand, I immediately ran right into a pretty terrific overview of what’s known about how to improve high schools. On the other hand, I have a couple burning questions that I’m struggling to answer, and I’d appreciate any thoughts.

1. How does the achievement gap break down by year in school?

What I’d really like to know is what % of students is below grade level in reading and math as of each year (entering K, entering 1st grade, etc.), broken down by income (and race, if possible). I’ve seen scattered statements along these lines, but what I’m describing seems so essential for deciding where intervention is most needed.

2. What’s the connection between education and later life outcomes?

Initially, I hadn’t planned to spend much time on this question – it seems intuitively obvious that a college education leads to much more opportunity than a high school diploma, which in turn is much better than dropping out. But as I look through reports like the one above, I’m starting to question whether targeting academic performance is the same as targeting better outcomes. For example, it looks like one of the best ways to improve test scores is through extremely intensive instruction … is someone’s likelihood of becoming a doctor rather than a drug dealer really more impacted by their knowledge of math after having it drilled into them, or by, say, mental health? (I’d still guess that it’s the former, actually, but I’d like to see some evidence – especially because I’m trying to decide on the relative value of getting dropout risks through high school vs. getting non-dropout risks ready for college.)

Anonymity rocks

(updated on: ) | by Holden

I’ve been flooding our readers lately, so today I just want to brag a little bit about an aspect of our application process.

We provide a public online discussion forum as one way for our applicants to ask questions and give feedback. One of the advantages of that it lets our applicants criticize us anonymously – and unlike with a survey, that criticism doesn’t have to be after-the-fact and one-way, but can be a conversation while still retaining the anonymity.

Yesterday and today, we got our first concrete benefit from this particular policy. As you may have figured out, I am a pretty blunt person and can sometimes come across as nasty, sarcastic, flippant, etc. without meaning to. I got criticized for this by an applicant who, completely understandably, didn’t want to disclose their identity. My first reaction to the criticism was dismissive: I didn’t believe I’d actually said anything mean, just terse, and I was ready to write off the complaint as hypersensitivity (and I would have, if we’d collected the complaint through a traditional survey). But when I asked for examples, the anonymous party responded with good ones, and I realized s/he was right and I was wrong. I’m too used to my old workplace environment, where everyone I write to has met me, and won’t misinterpret what I say.

I can easily fix this, I just needed it pointed out. The result will be more appropriate communications with applicants, and that matters. And if not for the anonymous discussion, I’d just keep making the same mistake. Think about that. To others working in grantmaking (where those you work with often hesitate to be critical): you may be better with tone than I am, but what you-specific mistakes might you be making, and what are you doing to catch them?

My favorite cause

(updated on: ) | by Holden

This is a response to the Giving Carnival topic I posted a couple days ago.

I’m passionate about all of the 2007 Clear Fund causes, and about many others as well, but the one that revs me up the most is inner-city education in wealthy societies. The reasons for this are complicated and tough to unravel, but here goes. This cause appeals to my:

  • Universal-humanitarian values. Although the Clear Fund is focusing for now on NYC, I have never felt anything more for the people who share my geography or religion or ethnicity than for the people who don’t. (And if someone I care about suffers from a particular disease, it makes me want to fight suffering, not fight the disease.)
  • View of the good life. Gotta be brief about this one. I don’t value happiness, or even the absence of pain, as much as most people do. What I really value is giving someone the opportunity to reach their full potential as a world-shaper, not just -citizen; I want people to experience, learn from, compete with, or even be the best in the world at what they care most about. I see value in things like hospice care and the Make-a-Wish foundation, but they excite me way less than enabling a life with no ceiling. And even though you can save an African life for much less than a U.S. one, I think a true shot at all the opportunities the world has to offer is easier and even cheaper to provide in the U.S.
  • Desire to make a real, short-term, definite difference. I’d love to help invent a great cure or technology, or a new version of capitalism where everything is perfect. But no matter what the “expected value,” I can’t stomach the idea of pouring everything I have into one giant losing bet, and failing to impact a single person’s life for the better. We might not have the formula for helping kids succeed, but at a minimum, giving them more supportive adults to interact with is going to make a difference in someone’s life, statistically significant or not.
  • Desire to work on a large-scale problem. Despite the smack I’ve talked about “root causes,” I also can’t be 100% satisfied just exchanging dollars for lives. I believe that well-spent charity in this area will support organizations that are trying new things and documenting them; we may never get “the answer,” but we’ll be learning from our mistakes and creating more knowledge about programs that can be replicated at other charities and even possibly by the government. To me it’s a great compromise between tackling a grand problem and helping individuals, because unlike a purely political endeavor, you’re building evidence as you go – and the evidence is real people really helped.
  • Wish to fight bad guys. Sure, malaria and diarrhea are bad guys, but this cause, for me, has a much more human and tangible (and thus detestable, and thus motivating) bad guy as well. It’s the people who believe that economics is justice, that our current society is perfectly fair, and that – by implication – the hordes of poor/unproductive/criminal people in the inner city must be that way because they were born that way. People who think this aren’t just being hypercapitalist; they’re logically committed to racism too, since the achievement gap persists as a racial phenomenon even when you control for economics.

    My hatred for these people goes back to childhood, when I loved Les Misérables and couldn’t believe that the smug suburban sheep around me thought they actually deserved to be better off than Jean Valjean. I desperately want to see them be wrong, and I don’t need a worldwide revolution for this to come about. Every poor minority student who grows up to be a successful businessman/politician/writer is one more slap in the face of people who think the world as of 2007 is a reflection of justice.

  • Beliefs about race issues. Some feel that racism is the true “root cause” of the achievement gap, and that may be true, but I think it also works the other way around. As long as certain communities (particularly black ones) are held back from the start, they’re going to end up less successful and more criminal than the rest of us, and that just encourages the people (the same jerks from above) who look for reasons to write them off as inferior. Again, every person helped is another salvo in this battle.

A lot of the above isn’t in our criteria, because it isn’t strictly in line with our declared values, and it isn’t strictly rational. I shouldn’t care about proving jerks wrong, for example; I should just care about helping people. But I’m human, and what motivates me isn’t quite that simple, and I can be honest about that and still do my job well. I hope others will participate in similar fashion.

Giving Carnival 8/07: Bare your soul

(updated on: ) | by Holden

This blog is hosting the Giving Carnival this week. The Giving Carnival is a horrible name for the following: the host chooses a topic, anyone who wants to writes/submits a post on that topic, and the host posts links to the ones he wants to (in a space this small, generally everyone) with commentary. It’s like a periodical, but with the advantage that it’s much more of a pain in the neck to read. Get pumped!

This week’s topic is: what charitable cause are you personally most passionate about?

Cancer research? Feeding the homeless? Fighting malaria?

Are you a US kinda person? Or global? Or Topeka, KS?

Please post a comment to this post by midnight of 8/4/07, either linking to your post or containing your response. If you have your own blog, link to your post and include a summary. If you don’t, or want to be anonymous, or just don’t want this on your blog, you can fully participate by comment. I will publish my own roundup the next week, but this way everyone will be able to see the unedited version.

A few requests:

  1. Get personal. Whatever “hats” you wear at your organization or as a blogger, take ’em off, along with your clothes. Tell us what you care about and why.
  2. Be specific. I’m sick of debating things in the abstract like “Should nonprofits make every effort to run as efficiently as possible, while also leaving room for their human side?” No. Next time, maybe. This time, I want to learn something about you and your values.
  3. Fundraisers: please participate! You chose your organization – and with it, your cause – over every other choice in the world. Why? What drives you? If your answer ends up being a plug for your organization, that’s totally fine. Just be personal about what excites you, not what excites others.
  4. No meta-charity or other copouts. Don’t talk about a cause centered on getting others to give more or give better. (I won’t be talking about GiveWell.) I know that technically answers the question, but it won’t be about your values. So if you do work for one of these organizations, write about your favorite sub-cause that your cause helps fund.

Can you tell I’m worried about cop-outs? I think people in this space love to say things that “can’t be argued with,” and they also have a tendency to talk about what we can sell rather than what we should sell. For this carnival/thing, I want you to put your personal values right out there in the open. I want to see passion, values, your bleeding heart laid bare. If there’s some professional reason you can’t do this, just participate anonymously (via comment). (And all non-bloggers are invited to participate via comment as well.)

Wanna get pumped about GiveWell?

| by Holden

Read the debate over at Tactical Philanthropy, on the following question: Clarity in charity – good or bad? (My wording.)

It reminds me of when I used to read really old arguments for democracy and free speech (I’m talking about Locke, John Stuart Mill, etc.) and think, “Huh, there were people who disagreed with this stuff?” That’s what the above debate will look like 100 years from now. People will read it and go “Wow, look at those crazy people saying there are ‘pros and cons’ to transparency. Is this real? Did people really think that?”

Hangover

(updated on: ) | by Holden

That last post was a doozy, wasn’t it. Please read it, though, if you get a chance, because this is really important. It’s my first attempt in a while to do what I’ve been saying forever that we can do: put the reasoning behind our decisions in writing, so that you can see the basis for everything we say and everything we decide. I’d really appreciate your thoughts on the following:

  • Readability and usability – are you able to use the writeup to understand what I’ve found and what I think?
  • Fairness and transparency – is the context, reasoning and evidence for every judgment call I make clear?
  • Coverage – I can’t be comprehensive, but am I making intelligent decisions about what lines of research to pursue?
  • Quality of analysis – are my conclusions reasonable? Do they make you feel I’ll make good bets on helping people?

It is a very preliminary, rough draft – 2 days from “What’s Head Start?” to “Oog, OK, finished the blog post. My head hurts.” – but the sooner I start getting feedback on these things and improving my approach, the better.

A few other thoughts that have come up in my first stabs at research:

  • I’m probably (no promises!) going to be posting more often on the blog, and when Elie joins us full-time on August 3, I’ll be asking him to as well. This time around, knowing I had to write a blog post really focused my thoughts and stopped me from forming impressions that I can’t explain; that’s a good thing, and I want to keep it up. My goal is to create a public document of everything that goes into our decisions – the more often I’m writing down where I stand, the better I’ll force myself to keep track of why I think what I think (and the earlier my progress will be out there to be critiqued).
  • If you think it’s best to forget about measurement and spend all our money on “getting things done” … take a nice careful look at that writeup and its sources and please think again. Many of the Head Start studies found strong and immediate positive effects on children’s IQ … effects that were gone two years later. Meanwhile, the St. Pierre study observed positive changes in just about every aspect of its clients’ lives … but putting them alongside against a control group, the researchers realized that everyone’s lives were improving (maybe a strong economy, maybe just the fact that they were hand-picking people at their worst point) … and their program was having no impact at all.Next time you hear someone say “The smiles on the children’s faces are the only data I need,” think about this. The fact is, as a local service provider, you simply have no way of getting around problems like what’s above. No amount of direct observation and experience with your clients can tell you if you’re having a real and lasting impact on their lives. A long, tedious, dry study can. It’s annoying but true.
  • I believe that there is a monstrous and unfortunate disconnect between charity and academia. Here’s why.
    • Reading academic papers was a whole different universe from reading charities’ self-evaluations (as I did last year). With the latter, I was constantly trying to figure out how the study was run and what kind of problems might be present; the papers themselves seemed entirely unaware of selection bias issues, making conclusions like “Test scores rose over period X, so the program was a success.” The academic papers on Head Start, on the other hand, were crystal clear – much clearer than I ever was in my head – about how each study was designed and what potential problems (or lack thereof, for randomized studies with low attrition) might be involved. Those that didn’t employ randomization apologized up front. It’s like people have been thinking about these issues for years and years, and have formed a whole culture around them … and many nonprofits and their evaluators seem totally oblivious.
    • I can’t get freaking decent library access. This is crazy. Sorry, but the New York Public Library is crap – spotty collections, closed stack, no borrowing research materials. Meanwhile, Columbia won’t let me within 10 feet of them because I’m not in academia. NYU wants $225 to get in the door and another $600 for borrowing privileges, which we might do, but still … in the world of research libraries, my nonprofit affiliation is worth about as much as my talent for whistling. Why hasn’t this come up a million times, for every foundation trying to research their issues? I gotta say it – probably because they’re not doing that.
    • It’s just unbelievable how excited academics seem to get about anything that has public policy implications. Witness the enormous amount of research related to Head Start. And it’s no surprise – every academic paper begins by explaining why it’s important, and for many papers this is the weakest section. The Head Start-related papers are nothing less than gleeful in proclaiming their relevance to the debate, and thus to actual current events. Well, that’s great – but what about the question of, you know, what we should do with our own money? I haven’t seen a single paper justify its existence by saying it’s relevant to the question of where foundations, or individuals, should give charitably. Why not???

Look. Academia and nonprofits are a match made in heaven. Nonprofits are always complaining that they have no capacity for evaluation; academics complain that no one cares about their work. Academics have the resources, knowledge, and patience to do ridiculously long and diligent studies; nonprofits, at least potentially, have the real-world significance to make it matter. Guys, I can’t watch you ignore each other like this.