Here’s what I’ve learned in two days of reading up on early child care (Cause 3). This is as rough as a draft gets, but I’m still going to provide references, a summary, and a roadmap, because those things are all so useful.

In a nutshell: it’s at least possible to make an amazingly big and long-lasting difference for disadvantaged children, through preschool programs with an emphasis on learning. (Several reasonably executed studies return to beneficiaries over ten years later and still find effects; one does so at age 21!) Not all of these programs work, and there’s no clear “magic formula,” but what I’ve seen makes me expect reasonably big things from programs that are (a) time-intensive; (b) focused on education; (c) targeting at-risk and disadvantaged children. Parent outreach programs, and programs that focus exclusively on the earliest years (0-3), are less well documented, probably because they aren’t at the center of the massive policy debate over Head Start.

Roadmap

• Head Start-style programs describes programs that generally emphasize learning and focus on the years immediately before kindergarten.
  • The stakes explains why there has been so much debate and study of these programs, and thus why I spent so much more time on them.
  • Types of programs distinguishes between Head Start, Early Head Start, Universal Pre-K, and other.
  • Evidence of effectiveness: “model programs” describes some of the findings of individual, intensive preschool programs. The systematic studies that have been done give a surprising and impressive picture: by and large, it appears as though these programs have been associated with real impact on children’s lives, long after the programs end.
  • Evidence of effectiveness: Head Start describes the evidence for a larger, more broadly defined set of programs. It’s much more muddled, although to me it still looks like these less intensive, less well-funded programs are still doing more good than harm (and more good than random).
  • What works? takes, basically, a couple wild guesses at what the difference is, observing that the most impressive results seem to come from programs that (a) are more intensive (b) focus on disadvantaged and at-risk children.
• Early childcare programs briefly discusses the debate around programs that focus on the years of 0-3. There is some evidence that these programs can make a modest difference, and it’s even possible that they have more potential than Head Start programs, but as far as I can tell there isn’t a lot of compelling evidence.
• Parent outreach programs briefly discusses the not-very-strong case for programs that center on educating parents, rather than children.
• References: if you’re still awake at the end of this writeup, and want to read more … you should really consider volunteering for us.

Buckle your seatbelts, and get ready to get real mad about anything bad that happened to you when you were 4.

Head Start-style programs

The stakes

Preschool programs have been subjected to dozens of systematic studies, several of which are carried out with a level of scientific precision that frankly, with the 50 warnings I get per day about the expense and difficulty of measurement, knocked me off my chair. These studies randomize the participants, to ensure that nothing but the preschool program could possibly explain any differences, and then follow up with them regularly for 10-15 years after they complete the program. (Details below.)

The studies aren’t perfect, and the one thing all the authors seem to agree on is that we need more research (makes sense; they’re researchers). But the diligence and volume here is beyond what I’ve seen for other social benefit programs (including within this cause). And I believe the reason is the massive debate over Head Start, a federal program that gives grants for preschool education. Head Start is a large use of federal funds, but its coverage is not universal; some want to see it cut, and others want to see it extended to every child in the U.S.

This is important for a couple reasons. First off, research has really honed in on “whether it works,” rather than “when it works.” Concluding anything about best practices within a preschool program takes a lot of triangulating across studies, and I’m hesitant to do it (although a couple things to jump out, as I’ll discuss). Secondly, on the question of how to donate, there is a huge side benefit (or cost, to some) to this area: funding programs that measure themselves well can help build (or erode) the case for expanding Head Start, as well as the case for changing the way it’s carried out. If you’re strongly ideologically opposed to being a part of this, you may want to steer clear. To me, it’s a plus, although not one that we’re going to consider in grant allocations (since we’re explicitly focused on helping people directly).

Types of programs

• Head Start (official program page here) is a federal program that gives grants for preschool care. I’ve had a heck of a time pinning it down to official, explicit criteria (unlike Universal Pre-K, below); the closest thing I see to a definition, from Section 641D of the Head Start Act, is “comprehensive health, nutritional, educational, social, and other services needed to aid participating children in attaining their full potential.” In practice, most of the studies on it (see Barnett, pp. 232-236) have looked at programs that treat students at the ages of 4-5, and Barnett gives a more concrete description of what the program usually amounts to: “Head Start for many years has provided children with education services (typically a half-day program during the school year), health services (including medical, dental, hearing, and vision examinations and treatment), nutritional services (meals, snacks, nutrition education), and social services to children and their families through direct services or referrals” (222).
• Early Head Start is the leg of Head Start focused on earlier childhood, described in Section 645A of the Head Start Act. I haven’t seen nearly as many studies of this program, and its focus on earlier childhood makes it more similar to the “early childhood care” programs described below.
• Universal Pre-K is a general term for similar grantmaking programs run by individual states. Because our focus is NYC, I looked only at the New York incarnation, which is far more specific in its criteria than the federal program is. Its guidelines are available here; they are extremely comprehensive, and demand programs that address children’s emotional, social, educational, and physical (food, health) needs. Advocacy group PreKnow states that the state intended this program to be truly universal in its coverage, but has followed through with the funding.
• Model programs is a term used by many researchers (and now by me) to refer to specific preschool programs that go above and beyond the Head Start guidelines and requirements. It’s not totally clear to me what the technical line is between model and non-model, but Barnett implies that model programs tend to have been developed by researchers (225) with an explicit intent to measure whether impact is possible in the best of circumstances.

Evidence of effectiveness: model programs

Scanning all the most rigorous and thorough evaluations of model programs, there are a couple that had flat-out stunning evidence of effectiveness many years after program participation, a few with strong evidence, and a few that demonstrated nothing statistically significant. All in all, a pretty compelling picture.

The one that really knocked me off my chair was the set of several studies of the Carolina Abecedarian Project, most of which I was able to check out directly (not just via literature review). See Campbell &c 1994, 1995, 2002. This project sought out at-risk children (“based on maternal and paternal educational levels, family income, absence of the father from the home, poor social or family support for the mother, indications that older siblings had academic problems, the use of welfare, parents working at unskilled jobs, indications of low parental IQ, family members seeking counseling, and other evidence of a need for support from community agencies” – from the 1995 study); randomly separated them into groups; gave year-round, full-day preschool care to one group and not to the other (and also did a later split to test in-school care, but I won’t be discussing that), and followed up again and again. The kids who had been put in their program had higher IQ’s at ages 12 and 21; better test scores at 15 and 21; more enrollment in higher education at 21; less enrollment in special ed, and less getting stuck in their grade, at 15. Holy cow.

This isn’t just cherry-picking. A couple different lit reviews (Barnett, Currie) compiled large lists of studies and examined all the ones that are most methodologically sound, and while the other programs they looked at weren’t as overwhelming as this one, they showed statistically significant impacts on similar measures, long after preschool (sometimes to grade 5, sometimes post high school). See Barnett, 226-229, for a grid that summarizes it all (Currie’s grid, on page 6, is similar though smaller). I come away thinking that early child care can matter.

It looks to me like the programs that were year-round and full-day, as well as focusing on the many years before school (not just 4-year-olds) generally had much stronger results (see the grid). The High/Scope Perry Preschool Project (Schweinhart &c 1993) did manage to get better high school graduation rates for a single year of part-day preschool, which to me borders on implausible, but it’s one of the methodologically strongest studies. Most of the other studies of part-time one-year programs found nothing statistically significant, although some of the results still show pretty big positive impacts for the programs (for example, noticeably smaller proportions of beneficiaries in special ed) that just have too little sample size to make much of.

Evidence of effectiveness: Head Start

Two widely cited studies (Administrative History; McKey et al. 1985) apparently claim that when you look at Head Start programs in general, you see immediate impacts on children’s IQ, but that these impacts “fade out” within a few years. I haven’t been able to get my hands on either of these originals, but Barnett characterizes McKey that way and Wikipedia characterizes Administrative History that way. And the tone of pro-Head Start papers and advocates is often on the defensive against this idea, so somehow it seems to have gained at least popular traction.

Barnett’s literature review has a section on studies of larger Head Start programs (see pages 232-236 for another awesome grid), and it’s certainly less impressive than the model programs. Any statistically significant difference in test scores fades by around 3rd grade, in 12 of the 16 studies (2 of these 12 have it fading only for certain ethnicities). On the other hand, when you look at measures of performance rather than intelligence – particularly focusing on issues most relevant to struggling children, such as grade retention (i.e., getting held back) and special education, you see pretty strong and pretty persistent positive effects of these programs.

Almost all of these programs have significant methodological problems. None of them are randomized, and to me that’s a real issue with Head Start, an optional program. I find it very counterintuitive that one year, no matter when, could make all that much difference, and it’s more plausible to imagine that parents who enroll their kids in Head Start are the same parents who are likely to push harder for them to stay out of special ed and move to the next grade. Then again, in light of the similar (though stronger) results for model programs, as well as a couple of the patterns I describe below, there is some reason to take the findings at face value.

Barnett comes out of this grid feeling pretty sold on Head Start. I come out of it feeling much more mixed. With low confidence, I think there’s a chance that these programs are highly beneficial for the kids who need the most help, as I’ll explain in the next section.

What works?

Unlike with early childcare programs (below), I wasn’t able to find much on the definition of what constitutes “quality” in preschool. What follows are my own observations from Currie’s & Barnett’s grids (you can check them yourself and see if you agree):

• Results are more impressive for programs that went beyond a year, and went beyond part-day (both of which Head Start is generally restricted to). This makes sense to me. I can believe that an intensive program that takes care of a kid for his/her entire early development can affect the development of the brain and basic attitudes; I have more trouble with the idea of one year (age 4) in isolation making that much of a difference, although I suppose it could be that crucial of a year in development.
• All of these programs seemed to do a better job with things like keeping kids from requiring special education, or getting held back, than with IQ and test scores. That makes sense to me too. I’d guess that good child care has more to do with keeping kids out of bad situations than with turning them into geniuses – I think (without any evidence) that it’s easier to screw someone up than to make them go above and beyond their natural abilities.
• In light of that, it’s also striking to me that the strongest results (from the Abecedarian program) come from a program that specifically sought out “at-risk” children, rather than simply low-income children. Also, looking at the Barnett grid, it seems that the methodologically strongest studies showed the strongest effects – the opposite of how it usually is (as selection bias usually exaggerates results). Why would this be? One possible answer, to me, is that the methodologically strongest studies were the ones that used randomization rather than after-the-fact comparisons, and that means they were the most careful in picking out people to study. I didn’t get to look at the Perry preschool study, but it may have done the same sort of selection – specifically targeting at-risk children – as the Abecedarian study, whereas the weaker studies would have no good way of doing this since they didn’t pre-pick their populations.
• So, I don’t know everything or even much, but my money’s on intensive programs targeting highly disadvantaged and at-risk children. And in the end, I don’t think child care is rocket science – if I see a program that’s doing the same basic activities as the Abecedarian project, and its personnel are qualified and (from a site visit) seem to like children, I’m going to think it’s relatively reasonable to expect similar results, i.e., a real effect on children’s development.

Early childcare programs

As this article details, there has been a good amount of media coverage of the idea that the ages of 0-3 is really where it’s at, because that’s when the brain is really developing. As the article also explains, the actual science is much more mixed. I don’t have much literacy or opinion of the actual neuroscience, except I know that it would have to be pretty slam-dunk cause-and-effect reasoning to sell me without accompanying evidence of actual results, and it doesn’t appear to be slam-dunk (based on the review in the article at the beginning of this paragraph). So I look for studies of whether differences in early childhood care have actually been shown to lead to better later outcomes.

Studies on this are pretty scarce, probably because there’s no policy debate attached. The main source I can find is the NICHD, whose huge study on of early child care and youth development is available here. I haven’t read the study carefully, but I’ve scanned it and have a couple observations:

• It doesn’t look at later-in-life outcomes the way the Head Start studies do (it goes up to age 4.5), and given the “fade out” often observed by the latter, I’m hesitant to get too excited about its relatively modest demonstrations of effects.
• It also doesn’t have a randomized design. It’s a survey: they look at the connection between quality of child care and child performance along various metrics, meaning they could easily just be showing that families that care more about getting good child care are also doing other things to help their children (or just care more about them!)
• One appealing thing about the NICHD study is that it’s very specific about what constitutes “quality” child care. Unlike the Head Start studies, which are all about debating “whether it works,” the NICHD gives a nice worksheet that I’ve printed out for us, listing the qualities of good child care that it has linked to better performance, some easily quantifiable and some not.

It makes just as much logical sense to me that 0-3 child care matters as that 3-5 education-focused care matters. Just as much, no more, no less. The evidence for the latter is much stronger. So that’s where I stand for now. A particular charity with a particular approach, and good measurement, could easily change my mind.

Parent outreach programs

Teaching parents to take care of their kids seems great in theory, but the question is, are you really changing parents’ behavior or just giving them a bunch of advice they’re not taking? I’m open to seeing a parent outreach program that really has a way of changing parents’ behavior, but preliminarily, I’m more excited about the more child-focused programs.

The Gomby study is pointed to by many papers (including Schaefer, see page 6) as establishing that there isn’t much to these sorts of programs. I haven’t had a chance to look carefully at Gomby itself, and I should. Early on, it states, “In most of the studies … programs struggled to enroll, engage, and retain families. When program benefits were demonstrated, they usually accrued only to a subset of the families originally enrolled in the programs, they rarely occurred for all of a program’s goals, and the benefits were often quite modest in magnitude” (Gomby, 6). That quote in itself isn’t all that sobering to me – I’d expect any program that only addresses one factor to show only modest effects.

Much more sobering is one of the studies it references, the St. Pierre et al. study, which took an extremely thorough, completely randomized look at a large federal “experiment”, the Comprehensive Child Development Program, which centered around advising families rather than providing particular services. There is a huge amount of data here, and the program seems to have been reasonably well carried out, and the randomized design found nothing. Pending what I find in the Gomby paper, I feel like it’s reasonable to conclude that (consistent with logic) engaging parents is tricky, and it’s going to take ingenuity, not just execution of a basic plan (while I feel that Head Start-type programs can work with the latter). Again, I’m totally open to a particular charity’s showing me that it’s cracked this nut and can prove it.

References

These are just the papers that either are not available online or that I refer to repeatedly. When I just talk about a paper once and it’s available online, I just link to it (above).

• Administrative History of the Office of Economic Opportunity, Vol. I, p.252, Box 1, LBJ Library.
• Barnett, W.S. (2004). Does Head Start have lasting cognitive effects? The myth of fade-out. In E. Zigler & S. Styfco (Eds.) The Head Start Debates. Baltimore, MD: Paul H. Brookes Publishing Co.
• Campbell, F.A., and Ramey, C.T. (1994). Effects of early intervention on intellectual and acaademic achivement: A follow-up study of children from low-income families. Child Development, 65, 684-698.
• Campbell, F.A. and Ramey, C.T. (1995). Cognitive and shcool outocmes for high-risk African-American students at middle adolescence: Positive effectds of early intervention. American Educational Research Journal, 32, 743-772.
• Campbell, F.A., Ramey, C.T., Pungello, E.P., Sparling, J., & Miller-Johnson, S. (2002). Early Childhood Education: Young Adult Outcomes from the Abecedarian Project. Applied Developmental Science, 6, 42-57.
• Currie, J. (2001). Early Childhood Education Programs. Journal of Economic Perspectives V. 15 #2, pp. 213-218.
• Gomby et al., 1999. Home Visiting: Recent Program Evaluations: Analysis and Recommendations. The Future of Children, Vol. 9, No. 1.
• McKey, R.H., Condelli, L, Ganson, H., Barrett, B.J., McConkey, C., & Plantz, M.C. (1985). The impact of Head Start on children, families, and communities. (DHHS Pulication No. (OHDS) 90-31193). Washington, DC: U.S. Government Printing Office.
• Schaefer, S. and Cohen, J. Making Investments in Young Children: What the Research on Early Care and Education Tells Us. National Association of Child Advocates, Washington, D.C.
• Schweinhart, L.J., Barnes, H.V., Weikart, D.P., Barnett, W.S., and Epstein, A.S. (1993). Significant benefits: The High/Scope Perry Preschool study through age 27. Monographs of the High/Scope Educational REsearch Foundation. No. 10. Ypsilanti, MI: High/Scope Press.
• St. Pierre et al.. 1997. National Impact Evaluation of the Comprehensive Child Development Program. U.S. Dept of Health and Human Services.

Still reading?

You get 5 points.

This blog has become part of a blogging “community.” We read each other, we refer to each other, we comment on each others’ posts, and we probably think of each other as the “other charity bloggers.” I’m talking mostly about:

• Tactical Philanthropy
• Philanthropy 2173
• White Courtesy Telephone
• GiftHub (and its weird sister, Wealth Bondage)
• The Nonprofiteer

I think all of the blogs above are intelligent, sometimes entertaining, and generally worth reading for a charity nerd such as myself. I like having people to respond to and people responding to me; being part of this community is been a reliable way to get dialogue, as well as pick up readers. That’s cool. But I have two major problems with this set of people.

1. It’s just too small, and I think the people are too similar to each other. That’s been driven home to me by Sean’s recent interview with William Schambra. Read it, if someone holds a gun to your head and forces you to. In brief, the participants are acting like the fact that this guy affiliates with “conservatives” rather than “liberals” means that this affiliation is the only thing worth discussing about or around him, that every word he says must be nothing more than snake oil designed to protect conservative interests. They’ve used their stereotypes to come up with ludicrous, implausible interpretations of his motives, rather than challenging what he says to get at whatever biases he does have. There’s no recognition that conservatives sometimes think about things other than cutting taxes and oppressing minorities (and that some conservatives support neither of these things). To me, that’s a sign of insulation from other points of view.

(Can you tell I’m annoyed? Guys, if you’re reading this, I’ll get over it. Here’s an e-hug: (). But I do think you’re insulated.)

2. More importantly, none of these blogs ever talk about how to help people, which is an odd quality for a group of “philanthropy blogs.” My original vision for this blog was as a place to discuss tutoring vs. charter schools, bednets vs. water pipes. It hasn’t happened so far, mostly because we had to halt our research on these issues in order to focus on the business side of The Clear Fund. It still may not happen for a little while, because I’m just starting to learn about these issues myself, and at this point anything I wrote about them would largely be in a vacuum. But as we shift our focus to research, I expect this to change.

So, I don’t want to ditch the friends I have, but I want some new ones too. I do read a ton of other blogs, but the only ones I know of that discuss how to help people are Beyond Philanthropy and the Google.org blog (which is showing early signs of flakiness). Any suggestions?