Monday I visited Year Up, our #1 ranked charity in the cause of employment assistance. Although they quite understandably declined my request to videotape the visit, I’m going to share as many of my impressions as I can here.

Although I think informal/intuitive evidence is generally overrated (relative to facts about life outcomes), I do still think it’s valuable. So here are my thoughts. I’ve made no attempt to organize them, or to support them empirically in the way we usually try to do. This is soft evidence - these observations are nothing but loose impressions and entirely personal opinions, so take them as such.

First of all, I think it was a valuable way to learn about the organization, and in the future I think we should change our plan and do site visits in between Round 1 and Round 2 (rather than at the end of the process). It certainly was far from “immersion” - closer to a fundraising event - but talking to the staff and students gave me a sense of what they think is most important about the program, which, on reflection, is a better place to start our investigation than with our own questions.

The area was small but nice - clean and crisp with a generally positive, upbeat vibe. (Because of the visit? I don’t really think so. The students had energy when they were in the central space between classes - you can’t reasonably fake that, or at least, you wouldn’t bother to for little me.)

Staff were 100% in “selling mode.” All smiles, all positive. I tried continually to ask questions/bring up topics that would lead to meaty conversation, acknowledging challenges/weaknesses of the organization as well as strengths, but the fish just weren’t biting. So although there were things I learned, I didn’t find the conversation to be very substantive, and it certainly wasn’t a good basis for getting to know the staff in any way or getting a good sense for what they’re really like (how they approach problems, whether I’d want them on my team, etc.)

(To be clear, this isn’t necessarily a criticism. It’s possible that being 100% sunny is the way to get money from most funders, and you do what you have to do. But I feel that we were unable to interact in an authentic way, and I find that unfortunate.)

Students were also in “selling mode.” It isn’t necessarily because they were actively trying to sell me - their attitudes just reminded me of members’ attitudes toward the summer camp I went to, or the college clubs I joined, or even to some extent the companies I’ve worked for. I believe people have a strong bias to believe that whatever they’re doing is the greatest thing to be doing, particularly if it’s something they (a) chose to do, and represents a commitment on their part; (b) involves education (nobody likes to think they’re studying anything other than the most important things you would want to know). So, no question that there was a lot of enthusiasm, and this made me pretty confident that the program does not bore them or antagonize them; at the same time, this is a long way from convincing me that they’re headed for better lives than they would be without Year Up’s help.

(I would certainly get a stronger feeling on this - though not anything definitive - from talking to a former student who is now working. Year Up told me they can set this up at a later time.)

I am very unclear on exactly what skills people are picking up in class. I asked the students what they’re learning about, but it took a long time to get off of things like “believing in myself” and onto anything related to the actual skills/knowledge they pick up. (Of course both can be important; I wanted to hear about both.) One talked about taking a computer apart, and another talked about learning how companies manage their risk, but neither was specific enough for me to get a sense that they are learning specific things that will be needed for specific activities. Cold-hearted as I am, I believe that if you’re going to try to change something about a young adult in a lasting way, changing their job skills is a better bet than changing their self-image and attitude. So seeing an apparent focus on the latter was a little disconcerting, though nothing big (I’d still bet that students are learning specific skills at some point during their program and internship, just given the nature of the jobs they’re going into).

I learned some interesting tidbits:

• Students meet once a week to publicly give each other feedback on strengths and weaknesses. This idea has enormous appeal to me personally - I just think taking and giving feedback is an underrated skill, keeping weaknesses “private” is overrated, and I’m glad to see Year Up sharing these values of mine in a pretty significant way. I asked if anyone was comfortable sharing a weakness, and the responses were on the generic side (”I used to not have confidence but now I do”), so I don’t get a great sense for what the feedback they share is actually like … but the exchange was pretty unnatural (funder asking students to share a weakness) so I don’t think it tells me very much on the whole.
• Students are assessed on a “point system”: being late to class means being docked a small amount of their stipend and losing “points” that can eventually lead to dismissal. Along with the contract they sign at the beginning of the program, this is intended to get them used to being on time every day, no excuses.

My view of why Year Up is good is very different from the staff’s view of why Year Up is good. I believe that it is a smart program because it finds people who not only need help but are capable of being helped - people who already have the motivation and intelligence to hold down a reasonably well-paying job, but aren’t doing so for a fairly straightforward reason like not having the training or connections to get it. Year Up agrees with me about the importance of connections, but that’s about it. They think their program design (particularly the two bullet points above) is key - and they emphatically do not agree that their selection of students is of major importance.

I asked them point-blank why they think particular other organizations, also serving disconnected youth in intensive, high-expectations programs, end up getting their clients jobs paying ~$10/hr, as opposed to Year Up’s ~$20/hr. Year Up’s answers were all about the high expectations they have for their students, the confidence they place in them, and the passion with which they do their job. I don’t deny that any of these things are true of Year Up - but I would guess they’re also largely true of at least some of the other organizations, and I have a really, really hard time believing that they can explain a 2x difference in incomes, especially when there is another perfectly good explanation at hand: Year Up has different entry/selection requirements than other programs.

As you know, I am very willing to believe that some charities are much better than others, but I wouldn’t bet that Year Up - or anyone - can take the same people that earn $10/hr from other programs and get them to $20/hr. I think their students are different people; consistent with our triage principle, I consider this a major advantage of the program.

Well, there you are, a bunch of impressions. Key takeaways? Seems like a good place with positive energy; my confidence in Year Up as #1 certainly hasn’t eroded. On the other hand, the “sell” aspect really made it hard for me to get a true feel for much, though I did the best I could; I’d much rather do a site visit as a spy than as a funder.

Our research for saving lives in Africa comes out soon. While we’ve mostly stuck to finding the best organization - rather than generalizing about “how to save lives” - we’ve formed a couple informal opinions along the way, and this seems like a good time to share.

First off, I think bed nets are a little overrated as a cheap way to save lives. You may have heard sales pitches like this: “With just $10 you can send a bed net to stop mosquitoes in their tracks. Send a net. Save a life.” (From the Nothing but Nets Campaign.) Or, from Nicholas Kristof: “For $5 you can buy a family a large mosquito net and save several people from malaria.”

The thing is, while it’s true that $5-10 buys a net, that’s a very long way from saving a life. We’re looking at PSI’s net-selling program, which costs between $5-10/net including distribution, marketing, etc., and we’re finding the following things need to be considered:

• Relatively few children die from malaria. That means you have to give out a lot of nets to make a difference in a few lives. By our estimates, you need to distribute about 25 nets to reach a child who would have died. And, some nets are likely distributed to areas where the mortality rate is even lower.
• Not all nets that are distributed are eventually used properly. Distributing something for free is great because everyone gets it, but it’s also likely that many people choose not to use it, and if they do, may use it for something far different than what it’s intended for. Since PSI sells rather than gives nets, we’d guess the concern is smaller than usual with them, but it’s still a concern.
• Nets only save lives if at-risk people sleep under them. Malaria largely kills children under the age of five (as well as pregnant women). If the wrong family members are under the nets - or they’re just not using the nets, period - the nets won’t do any good. Proper use, awareness of who’s at risk, etc. can’t exactly be taken as given, although unlike other distribution campaigns we’ve seen, at least PSI has some data on how often the nets actually get used (about 70% of those who own them use them on a given night, in the region PSI studied).
• Sleeping under a net only reduces your risk of contracting malaria by 50%. You can still get bitten during the day, when you go to the bathroom, etc.
• A net doesn’t last forever. Nets tear, and some need to be retreated with insecticide to remain effective.

When we do all the math (not available yet but will be this coming Monday), we estimate that you don’t end up saving a life per net - you end up saving a life for every 70 or so nets at best, and maybe even more like 300 (so around $500 to $2000). Even PSI’s own estimate of lives saved comes out close to around 70 nets per life saved. And that’s looking at an organization whose customers purchase nets, defraying the cost somewhat and also probably reducing the number of nets that go to waste. Giving nets, though it may ultimately be effective, may involve even higher expenses.

Too expensive? Of course not, $2000 for a life is still a ridiculous deal. We just think you can do better.

For example, I think condoms are pretty underrated as a way of saving lives from HIV/AIDS. A lot of the focus in HIV/AIDS is on antiretroviral therapy, an extremely expensive form of continual treatment for existing AIDS patients, but promoting safe sexual behavior of any kind can stop AIDS before it starts for many people, and has many additional benefits as well. Looking at the same organization (PSI), we think their condom marketing is saving lives for more like around $250-1000 a pop - and that isn’t including other benefits, such as:

• Reducing unwanted pregnancies (which also means reducing deaths in childbirth, actually one of the leading killers of African adults).
• Reducing sexually transmitted diseases aside from HIV/AIDS (also a killer and generally a pain in the neck).
• Slowing the spread of HIV/AIDS.

Of course, these numbers and claims rely on a lot of assumptions; our full research will be available within the week, so you’ll get the chance to check it all out then. These are just informal observations. But even though our estimates are rough, it seems to me that all things considered, a successful condom marketing program (or any program that increases safe sexual behavior) is a pretty good life saver, and probably better than the good old nets. I’ll take either one in a heartbeat over antiretriviral therapy, which was all the rage for a long time despite the fact that it’s one of the most expensive and complex ways to help people in a region littered with cheap and simple opportunities.

Then there are things I think might be even more cost-effective, but don’t know much about yet:

• Immunizations. They’re great because they fully protect a child from a disease for the rest of his life (unlike nets which only last for a few years), and as far as we know, the actual vaccine costs little. Measles, for which an effective vaccine already exists is still one of the leading causes of death among children in the developing world. We don’t know why vaccines don’t reach those children - it’s certainly plausible that the areas they live in are so hard to reach that the cost of vaccinating them is extremely high - but we’d like to know more about this, because it’s hard to find a simpler way to save a life.
• Vitamin A supplementation. Research suggests that providing vitamin A supplements (a product that costs less than a quarter) to children under five can reduce child mortaility by pretty huge amounts. We have some questions about the research, as well as the ease of expanding coverage (as in vaccinations), but it’s possible that (unsexy though it may be) reducing Vitamin A deficiency is one of the most promising ways to lower infant mortality and improve general health.

Thoughts?

Elie and I get all the headlines, and the ladies, but GiveWell is a lot more than the two of us. This seems like a good time to thank:

Our donors, mostly former coworkers who believed in us and stepped up to make GiveWell possible, even when it was nothing but an idea. Now, of course, it’s more.

Teel and Kendall, our best volunteer researchers. They’ve each put in a huge amount of time - and brainpower - helping us get a better handle on relevant literature from microfinance to tropical disease burdens.

Melissa and Ross, our volunteer Marketing Department. Both have put significant time and effort into various ways of getting our idea to people who can help us, and it’s paying off already.

Kendall (same Kendall), Vartges, Eric, and Leyla, our best volunteer editors. They’ve read our reviews, checked them over thoroughly (including the footnotes), and discussed them with us at length, helping them reach the state of perfection you see now.

Jordan of Fresh Milk Design, who put together the logo on our new site in a hurry (and on a volunteer basis).

Nicolas Borda and Suro, the website team. Not volunteers, but they’ve done great work for us, quickly and for good prices. Suro has mostly been working on the back-end functionality of our “Right of response” features, which allow charities access to a response space on their review pages. Nicolas is the man behind the visual layout of the site.

These aren’t the only people who’ve been a huge help to us, but they’re the ones who have invested the most of their own time and/or money to date. We’re lucky to be working on a project that brings out the passion and energy of such great people.

Well, one more thank-you:

My friends, family, and especially sister, who have all had to deal with my becoming a hermit, and have been incredibly supportive.

Eduwonkette gives a clear examination of data that is generally anything but.

This post is more about public policy than charity, but it shows - at a glance - a lot of the problems with the traditional approach to charity (giving as an act of the heart without the brain; trusting charities that mean well, without carefully studying their outcomes). The fact is that we don’t know what works in education. Blind faith won’t cut it, and neither will flimsy data.

A favorite saying of foundation people is, “You can’t be afraid to fail. You have to be ready to take bold risks.”

I’ve thought about it, and I think they’re going too easy on themselves. I put it to you: we need both funders of innovative projects and funders who focus on what already works. But right now, the latter is the one we need more of.

I believe that foundations today almost exclusively focus on high-risk, high-reward, unproven, innovative projects. Exclusively, to the point where there is no mechanism for proven, effective, scalable projects to get as big as they should be. My evidence is not as concrete as I’d like it to be, because foundations are too busy cowering behind locked doors to tell anyone anything about what they do. But here it is anyway.

• Exhibit A: from the horse’s mouth. Joel Orosz, Insider’s Guide to Grantmaking (pg 19): “Most foundations focus on encouraging innovation rather than on supporting the ongoing programs of nonprofit organizations.” Orosz reasons that individual donors, who give far more, will pick up the slack of funding what works. Problem is, they have no way of doing so. Take it from a guy who tried.

  As I’ll argue below, figuring out what works is actually far harder than figuring out what might work. That’s where foundations’ staffs and resources are most needed.

• Exhibit B: from the charities. Check out our Round 1 instructions, especially these parts:
  • “We know that helping people can be difficult to measure, but we have a strong preference for methods that we see strong reason to believe have helped people in the past.” (Pg 2)
  • “we suggest that you pick an existing, established, well-documented program for which you have thorough information readily available.” (Pg 3)
  • “Please focus on your existing programs and activities … [we] will likely not award programs for which little precedent exists. Rather, we are looking for organizations whose activities have worked well …” (Pg 4, emphasis in original)

  Now check out the Round 1 submissions (available under each cause on www.givewell.net). Note that even though we worked exclusively with large established charities and asked for projects that have worked before, we still received huge numbers of proposals for completely new projects. The only explanation I can think of is that charities are that accustomed to sending in new ideas, and that unaccustomed to talking about what they already do.

  Now check out the applications that didn’t misread our instructions, and did talk about existing programs. The evaluations are, frankly, a mess, with practically no exceptions. They’re in draft form, they’re disorganized, they neglect to mention clear methodological concerns … for the most part, not even our finalists were able to show us in a clear and readable way what their programs have accomplished in the past.

  I don’t blame charities, I blame foundations. We thought our questions would be easy for charities to answer, but as it turns out, “what do you already do?” is a question that none of them seem ready for. If foundations were asking this question, I have to think they would be.

• Exhibit C: the logic of laziness. There is no pressure on foundation people to do anything - so I expect them to take the easy way out. Although many point to their “courage” in taking on high-risk projects, in reality a high-risk approach is the easy way out for a funder. When you’re looking at what might work, instead of what does work, you don’t need any analytical abilities; you don’t need to interpret data; and you don’t need to be ashamed when you fail, because after all, you were taking a risk. You can follow your gut and your personal relationships all day long, and shrug your shoulders when it doesn’t pan out. I’m not saying there’s no place for this type of grantmaking, but it sure seems less sweaty than grantmaking based in the here and now.

Since Straw Man boxing is a favorite sport in this sector, I expect many of those who read this to respond, “But what would we do if nobody funded new innovative ideas?” I’m not suggesting that nobody fund new innovative ideas. GiveWell exists because some people took a chance on an idea. So does every other charity. What I’m saying is that right now, foundations work almost exclusively in the high-risk sphere, and that we desperately need more people identifying and scaling up what already works.

We need to learn more about what already works, so we can get those low-risk dollars from individuals to fund it and help lots of people. We need funders who go beyond people they like with ideas that sound cool, and get into the messy business of learning how the world actually is. We need more people who hold themselves to a high burden of proof, and the hard work and analysis that comes with it. We need more people who are afraid to fail. Could you be one of them?

There’s a saying that I think about a lot: “If you can’t explain it to your Uncle Bob, you don’t understand it.” (Note: may be a rewording of a real saying, or not a real saying at all; I’m not sure. If it’s unclaimed, mark it down as mine, thanks.)

How many times have you sat down to write something - for a blog, for a report, heck, for an email - that you were 100% sure of, then realized as you were writing that you have to rethink things entirely? It happens. When you’re talking to no one but yourself, everything you vaguely recall and intuitively believe sounds reasonable; you’ve got no check on it. When you’re talking to your fellow Program Officer, there’s a little more of a check, but the two of you are still in the same bubble, and you can still use jargon to skip over concepts you should be reexamining. When you talk to your Uncle Bob, that’s when you have to be clear - and that means getting clear in your own head.

That’s why, as we’ve written up our reviews for our public website, our opinions have changed drastically. Because as I write a review, I’m not trying to explain my reasoning to Elie, I’m trying to explain it to Uncle Bob. I’m trying to put it down so I can literally send a link to my grandma and have her understand why I’d rather give to Year Up than St. Nick’s. And even before anyone actually reads it, this makes me think much harder about it, question all the little assumptions I’ve been making (without knowing I’m making them), recheck all the sources for things I’ve been vaguely recalling, and get really clear on what I think.

I put it to you that in terms of clarifying and improving your thoughts, there is no substitute for the process of explaining your decision to a general audience. Everyone who thinks for a living knows this. Except, perhaps, for Program Officers at foundations.

Cause what I’m saying comes down to this. I believe that what we’re doing is exactly what every foundation, everywhere, all the time, should be doing: documenting everything we decide, with links to every single material we used to decide it, and releasing it so that anyone can see it and critique it. The benefits are obvious in terms of public information sharing, but one of the lame objections that is sometimes raised is that documenting your views for the public is “too time-consuming.”

Well heck yeah, it’s time consuming. But not because of the typing - because of the thinking. We’re spending ungodly amounts of time writing up decisions we think we’ve already made … and we’re getting every second’s worth back in terms of improving our decisions, clarifying them for others and for ourselves. Even before anyone has jumped in the ring to challenge our views (and a few people have), we’ve gotten a good dose of skepticism, feedback, and improvement from Uncle Bob.

Every single foundation should be writing up its decisions in public. Uncle Bob is fine with that statement. Who wants to challenge it?

For nearly a year now, we’ve been talking about creating the world’s first truly useful donor resource: one that goes beyond naive and meaningless metrics like “how much of my money goes to program”? and instead looks at what charities do and whether it works.

It’s not an idea anymore, it’s a product. It exists now. It’s right here:

www.givewell.net

There is more content to read, learn from, and critique than 10 blog posts’ worth.

If you’d rather read my snarky ramblings than the first ever public exploration of how to help people as well as possible … come back Thursday, I guess. Please check out www.givewell.net. We want to know what you think.

When you’re donating, do you want a “safe and reputable” charity - or the best?

From fighting disease in Africa to improving inner-city education, helping people isn’t simple - for charities OR for donors.

That’s why it isn’t enough to know that “99% of your money goes to programs.” (In fact, we think it’s often a bad thing). What really matters is what the programs are, and whether they’re helping people.

That’s why we’ve been working for months trying to find a donor’s best bet, in causes from employment assistance (NYC) to saving lives (Africa).

And that’s why we’re now sharing what we’ve found. Initially, only one cause (employment assistance) will be available. We’ll be putting up more research as we complete it. Giving season is upon us, and we want to help as many donors as we can.

Our research is now available at www.givewell.net. If you’re trying to help people, don’t miss it. If you’ve got any questions, let me know.

This year, don’t just give generously. Give well.

Look, another story where a foundation is accused of violating its original funders’ values.

It’s ridiculous to accuse the foundation. The funders violated their own values, the second they handed over their money to an organization set up to exist forever. They may as well have thrown it into a lake.

This is just common sense. The world changes too fast, and people are too different from each other, to think that the 3rd generation of your foundation’s staff will have any more in common with you than your dentist.

At least you know they’ll be trying to improve the world in some way, right? No. Foundations so often focus on political advocacy, awareness, and other zero-sum “giving” that you actually have no way of knowing whether they’ll be working for what you believe in - or directly against it.

There’s one way to control what your money pays for. Give it away yourself.

This Monday, we will release the beta for the product we’ve been promising since day one: a full account of what charities we recommend, and why, in five different humanitarian causes. (Only one cause will be available at first, but the others will come online within 5-10 weeks.)

I blogged yesterday only because I had to write all that stuff to think it through well, as we make our decisions and recommendations. We promise nothing tomorrow. And if I don’t answer my email or shower this week, know that it’s because we’re working around the clock getting the site as ready as we can make it.

Get pumped.

Many moons ago, I listed the metrics we planned to use in evaluations. Well, here’s a shocker for you: when looking at actual charities’ activities, the reality is 10,000x as complicated as anything that can fit into these metrics.

We always knew we would run into the following problem: if Charity A saves 100 people from death and 1000 people from mild fever, while Charity B saves 150 people from death and 500 people from mild fever (assume the same cost), which is better? And we planned to tackle it in a slightly unusual way - rather than trying to “adjust” everything to the same terms (disability-adjusted life-years, or dollars of income gained, or some such nonsense), we would simply focus on how many lives were deeply changed. So in the case above, Charity B wins because it saved more people from death, and to heck with the fevers. Not perfect, but no solution to this problem is, and at least this approach is easy to understand and makes it easy for the donor to draw their own conclusions.

I reasoned that cases like this hypothetical should be rare anyway, as long as we’re separating charities into different causes (some aiming to educate children, others aiming to save lives, etc.) and only comparing charities when they share the same fundamental goal. For example, the charity that saves more lives is probably preventing more fevers too; when charities meaningfully diverge, it’s probably a sign that they belong in different causes.

Well, it hasn’t been even as simple as our messy expectation.

We initially wanted a cause to “save lives in Africa,” but we quickly realized that many charities are trying to prevent permanent blindness, debilitating skin disease, or a cleft lip that can lead to permanent malnutrition issues and ostracization. This isn’t exactly the same as saving a life, but doesn’t it seem pretty close? So we changed Cause 1 to “prevent death and extreme debilitation.”

Well, now we are in a pickle. What do you do when Charity A treats 10,000 cases of malaria and another performs 150 corrective surgeries? Now, bear in mind that malaria can be fatal; or it can lead to brain damage; or anemia; or it can just be a fever and stop there. And we don’t know how often it does each of these things (anyone have a source? Seriously, we can’t find it.) Now about those corrective surgeries. Some of them repair clefts (possibly, though not necessarily, life-ruining in the way described above). Others repair hand deformities or eyelid deformities, which we believe are usually cosmetic issues and not as bad as clefts (though how bad is the penalty for looking weird in these societies? We have no idea). And other surgeries are unclassified. Oh, and we don’t know for sure how many fall in each category - we just have to estimate based on past data.

How do you figure THAT one?

Here’s what I think. By default, I think the more comprehensive a charity is, the better. There are lots of things that make a community program - serving every need that everyone has - better than a narrower program (like running around distributing bednets). A more comprehensive program has tighter integration into the community, probably better relations with it, and better ability to observe it and make sure that people’s lives are improving on the whole (not trading some problems for others). On the other hand, there is exactly one thing that is worse about this approach - that you might help fewer people for the same funds. If malaria is cheap to prevent, running around and preventing everyone’s malaria could save many, many more lives than sitting in a village working on everything from AIDS to hangnail.

So, what I say is that the burden of proof is on the less comprehensive program to show that it’s getting much better results (same money) than the more comprehensive one. We’re still estimating our “dollars per life changed” metrics, using largely the same philosophy we started with - just focus on the really huge life changes - because we want to see if one program is obviously more cost-effective than another. If one program saves 10-100x as many lives as another (and I’d include cleft repair in “saving lives” for only this purpose), we’ll take our chances on it. But if it’s close at all, we’ll go with the communal/comprehensive program.

Since so much is unknown, that often makes things tough for distribution-type programs, but that matches with common sense too. If you’re trying to help people in a faraway land, you’d better be really confident to run around the countryside with nets, instead of sitting down in a village where you can see everything that’s going on.

So that’s what I’m thinking. Keep quantifying charity, as messily as we have to, knowing that we’re only going to use the wacky numbers we come up with if they show enormous differences in cost-effectiveness. Otherwise, we’re giving the edge to charities that work with one person/community at a time (instead of one problem at a time). What do you think?

Eduwonkette (if you’re interested in the cause of education, stop reading so many philanthropy blogs and subscribe to Eduwonkette) links to an exciting-sounding article about a series of innovative charter schools. Imagine how excited I get by a passage like this:

“On graduation rates, on test scores, on teacher pay — on just about anything you associate with school reform — we have kicked the district’s butt. There’s nobody in America who has taken the same kind of kids in the same kinds of areas and the same dollars and narrowed the achievement gap like we have.” …

It’s been six years since the 47-year-old Barr launched his personal variant on the charter-school formula, Green Dot Public Schools, then lured 500 kids (and their supportive parents) away from nearby — and academically disastrous — Lennox High School in Boyle Heights. To the consternation of L.A. Unified officials, Barr created Animo Leadership Charter High School with the aim of showing what he could do with $1,200 less per student than L.A. Unified and most big-city districts in California spend. His goal was to accomplish what California schools have failed to achieve for nearly 30 years: turn functionally illiterate and grossly undereducated urban freshmen into literate, math-competent, college-ready graduates who can compete with the graduates of rich-kid Harvard-Westlake.

Yeehaw! Let’s go! You’re right, I haven’t seen any other school that can convincingly demonstrate it’s pulled off this incredibly difficult task! How did you do it? Is it the hiring practices, the classroom protocols, what’s the secret?

Oh, and by the way, what is the actual evidence that you’ve closed the achievement gap for your students? From the way you talk, it sounds like you’ve just humiliated the public system - so you’ve been following your kids through college and seeing that they turn out just as well as Harvard-Westlake kids, or what?

Here it is:

So far, early returns from his 10 schools show a graduation rate double that of LAUSD’s sad results. While the data is too new to be earth-shatteringly conclusive …

Oh. Hrmm. Early returns … not conclusive … also, only graduation rates are mentioned, and of course graduation rates are one of the easier metrics to play fast and loose with …

And - oh dear -

Duffy has claimed that Barr selectively handpicks only four students out of every 10 in the neighborhoods where he opens a new school … ““We know they have the ability to [skim the cream]. We just don’t have any way of verifying what they do.”

So, we have a school that doesn’t admit all comers, we’re not sure exactly how they choose to admit students, they could be creaming and they could not be, and whether or not they are, their graduation rates might represent progress, lowered graduation requirements, or a fluke … and people are celebrating with grants and editorials about the end of the achievement gap.

In other words, I just re-read the ol’ article about NYC charters, pretty much word for word.

My favorite part of this one is this quote: “‘Eli Broad doesn’t write a check if we are marginally better,’ Barr concludes. ‘People don’t write editorials about us because we’re not successful.’” I beg to differ. Apparently, people write checks and editorials because in the absence of any school conducting true randomized trials and thorough followup, a couple preliminary and questionable stats plus some flashy rhetoric is what passes for both news and hope.

That’s where we’re going to be stuck until and unless an education charity not only figures out how to make a difference, but makes a good, methodologically sound demonstration of it. In the meantime, giving to an education charity with poor self-evaluation seems like giving to a cancer research laboratory with no pens, paper or computers.