There’s a big question in my mind about K-12 education. The question is which of the following hypotheses about helping disadvantaged children is the best bet:

1. Disadvantaged children are so far behind by age 5 that there’s nothing substantial to be done for them in the K-12 system.
2. Disadvantaged children are so far behind by age 5 that they need special schools, with a special approach, if they’re to have any hope of catching up.
3. Disadvantaged children generally attend such poor schools that just getting them into “average” schools (for example, parochial schools without the severe behavior and resource problems of bottom-level public schools) would be a huge help.

My view on KIPP vs. the Children’s Scholarship Fund, for example, hinges mostly on my view of #2 vs. #3. Of course, believing #1 would make me want to avoid this cause entirely in the future. We’ve been examining academic and government literature to get better informed on this question, but we’ve noticed a serious disconnect between what we most often want to know and what researchers most often study.

To answer our question, you’d study how students do when they change schools, focusing on school qualities such as class size, available funding, disciplinary records, academic records, and demographics. However, most academic and government studies of voucher/charter programs focus instead on whether a school is designated as “public,” “private,” or “charter.”

Three prominent examples:

• The New York City Voucher Experiment intended to examine the impact of increased choice (via vouchers) on student achievement; the papers on it (Kruger and Zhu 2003; Mayer et al. 2002; Peterson and Howell 2003) conduct a heated debate over who benefited, and how much (if at all), from getting their choice of school, but do not examine or discuss any of the ways in which the schools chosen differed from the ones students would have attended otherwise.
• “Test-Score Effects of School Vouchers in Dayton, Ohio, New York City, and Washington, D. C.: Evidence from Randomized Field Trials” (Howell et al. 2000), a review of several voucher experiments, also discusses the impact of vouchers without reference to school qualities.
• “Apples to Apples: An Evaluation of Charter Schools Serving General Student Populations” (Greene et al. 2003) performs similar analysis with charter schools, looking broadly at whether charter schools outperform traditional public schools without examining how, aside from structurally, the two differ.

To be sure, there are exceptions, such as recent studies of charter schools in New York including Hoxby and Murarka 2007. But in trying to examine the three hypotheses above, I’ve been struck by how often researchers pass over the question of “good schools vs. bad schools” (i.e., the 3 hypotheses outlined above) in favor of the question of “private vs. public vs. hybrid schools.”

When a debate is focused on government policy, it makes sense for it to focus on political questions, such as whether the “free market” is better than the “government.” But when you take the perspective of a donor rather than a politician, this question suddenly seems irrelevant. Some public schools are better than others; some private schools are better than others; and I, for one, would expect any huge differences to be driven more by the people, practices and resources of a school than by the structure of its funding (i.e., whether donors, taxes, parents or a mix are paying).

That’s why we’d like to see more studies targeted at donors, rather than politicians. But for it to happen, donors have demand it.