The GiveWell Blog

I believe that the children are our future

Here’s what I’ve learned in two days of reading up on early child care (Cause 3). This is as rough as a draft gets, but I’m still going to provide references, a summary, and a roadmap, because those things are all so useful.

In a nutshell: it’s at least possible to make an amazingly big and long-lasting difference for disadvantaged children, through preschool programs with an emphasis on learning. (Several reasonably executed studies return to beneficiaries over ten years later and still find effects; one does so at age 21!) Not all of these programs work, and there’s no clear “magic formula,” but what I’ve seen makes me expect reasonably big things from programs that are (a) time-intensive; (b) focused on education; (c) targeting at-risk and disadvantaged children. Parent outreach programs, and programs that focus exclusively on the earliest years (0-3), are less well documented, probably because they aren’t at the center of the massive policy debate over Head Start.

Roadmap

  • Head Start-style programs describes programs that generally emphasize learning and focus on the years immediately before kindergarten.
    • The stakes explains why there has been so much debate and study of these programs, and thus why I spent so much more time on them.
    • Types of programs distinguishes between Head Start, Early Head Start, Universal Pre-K, and other.
    • Evidence of effectiveness: “model programs” describes some of the findings of individual, intensive preschool programs. The systematic studies that have been done give a surprising and impressive picture: by and large, it appears as though these programs have been associated with real impact on children’s lives, long after the programs end.
    • Evidence of effectiveness: Head Start describes the evidence for a larger, more broadly defined set of programs. It’s much more muddled, although to me it still looks like these less intensive, less well-funded programs are still doing more good than harm (and more good than random).
    • What works? takes, basically, a couple wild guesses at what the difference is, observing that the most impressive results seem to come from programs that (a) are more intensive (b) focus on disadvantaged and at-risk children.
  • Early childcare programs briefly discusses the debate around programs that focus on the years of 0-3. There is some evidence that these programs can make a modest difference, and it’s even possible that they have more potential than Head Start programs, but as far as I can tell there isn’t a lot of compelling evidence.
  • Parent outreach programs briefly discusses the not-very-strong case for programs that center on educating parents, rather than children.
  • References: if you’re still awake at the end of this writeup, and want to read more … you should really consider volunteering for us.

Buckle your seatbelts, and get ready to get real mad about anything bad that happened to you when you were 4.

Head Start-style programs

The stakes

Preschool programs have been subjected to dozens of systematic studies, several of which are carried out with a level of scientific precision that frankly, with the 50 warnings I get per day about the expense and difficulty of measurement, knocked me off my chair. These studies randomize the participants, to ensure that nothing but the preschool program could possibly explain any differences, and then follow up with them regularly for 10-15 years after they complete the program. (Details below.)

The studies aren’t perfect, and the one thing all the authors seem to agree on is that we need more research (makes sense; they’re researchers). But the diligence and volume here is beyond what I’ve seen for other social benefit programs (including within this cause). And I believe the reason is the massive debate over Head Start, a federal program that gives grants for preschool education. Head Start is a large use of federal funds, but its coverage is not universal; some want to see it cut, and others want to see it extended to every child in the U.S.

This is important for a couple reasons. First off, research has really honed in on “whether it works,” rather than “when it works.” Concluding anything about best practices within a preschool program takes a lot of triangulating across studies, and I’m hesitant to do it (although a couple things to jump out, as I’ll discuss). Secondly, on the question of how to donate, there is a huge side benefit (or cost, to some) to this area: funding programs that measure themselves well can help build (or erode) the case for expanding Head Start, as well as the case for changing the way it’s carried out. If you’re strongly ideologically opposed to being a part of this, you may want to steer clear. To me, it’s a plus, although not one that we’re going to consider in grant allocations (since we’re explicitly focused on helping people directly).

Types of programs

  • Head Start (official program page here) is a federal program that gives grants for preschool care. I’ve had a heck of a time pinning it down to official, explicit criteria (unlike Universal Pre-K, below); the closest thing I see to a definition, from Section 641D of the Head Start Act, is “comprehensive health, nutritional, educational, social, and other services needed to aid participating children in attaining their full potential.” In practice, most of the studies on it (see Barnett, pp. 232-236) have looked at programs that treat students at the ages of 4-5, and Barnett gives a more concrete description of what the program usually amounts to: “Head Start for many years has provided children with education services (typically a half-day program during the school year), health services (including medical, dental, hearing, and vision examinations and treatment), nutritional services (meals, snacks, nutrition education), and social services to children and their families through direct services or referrals” (222).
  • Early Head Start is the leg of Head Start focused on earlier childhood, described in Section 645A of the Head Start Act. I haven’t seen nearly as many studies of this program, and its focus on earlier childhood makes it more similar to the “early childhood care” programs described below.
  • Universal Pre-K is a general term for similar grantmaking programs run by individual states. Because our focus is NYC, I looked only at the New York incarnation, which is far more specific in its criteria than the federal program is. Its guidelines are available here; they are extremely comprehensive, and demand programs that address children’s emotional, social, educational, and physical (food, health) needs. Advocacy group PreKnow states that the state intended this program to be truly universal in its coverage, but has followed through with the funding.
  • Model programs is a term used by many researchers (and now by me) to refer to specific preschool programs that go above and beyond the Head Start guidelines and requirements. It’s not totally clear to me what the technical line is between model and non-model, but Barnett implies that model programs tend to have been developed by researchers (225) with an explicit intent to measure whether impact is possible in the best of circumstances.

Evidence of effectiveness: model programs

Scanning all the most rigorous and thorough evaluations of model programs, there are a couple that had flat-out stunning evidence of effectiveness many years after program participation, a few with strong evidence, and a few that demonstrated nothing statistically significant. All in all, a pretty compelling picture.

The one that really knocked me off my chair was the set of several studies of the Carolina Abecedarian Project, most of which I was able to check out directly (not just via literature review). See Campbell &c 1994, 1995, 2002. This project sought out at-risk children (“based on maternal and paternal educational levels, family income, absence of the father from the home, poor social or family support for the mother, indications that older siblings had academic problems, the use of welfare, parents working at unskilled jobs, indications of low parental IQ, family members seeking counseling, and other evidence of a need for support from community agencies” – from the 1995 study); randomly separated them into groups; gave year-round, full-day preschool care to one group and not to the other (and also did a later split to test in-school care, but I won’t be discussing that), and followed up again and again. The kids who had been put in their program had higher IQ’s at ages 12 and 21; better test scores at 15 and 21; more enrollment in higher education at 21; less enrollment in special ed, and less getting stuck in their grade, at 15. Holy cow.

This isn’t just cherry-picking. A couple different lit reviews (Barnett, Currie) compiled large lists of studies and examined all the ones that are most methodologically sound, and while the other programs they looked at weren’t as overwhelming as this one, they showed statistically significant impacts on similar measures, long after preschool (sometimes to grade 5, sometimes post high school). See Barnett, 226-229, for a grid that summarizes it all (Currie’s grid, on page 6, is similar though smaller). I come away thinking that early child care can matter.

It looks to me like the programs that were year-round and full-day, as well as focusing on the many years before school (not just 4-year-olds) generally had much stronger results (see the grid). The High/Scope Perry Preschool Project (Schweinhart &c 1993) did manage to get better high school graduation rates for a single year of part-day preschool, which to me borders on implausible, but it’s one of the methodologically strongest studies. Most of the other studies of part-time one-year programs found nothing statistically significant, although some of the results still show pretty big positive impacts for the programs (for example, noticeably smaller proportions of beneficiaries in special ed) that just have too little sample size to make much of.

Evidence of effectiveness: Head Start

Two widely cited studies (Administrative History; McKey et al. 1985) apparently claim that when you look at Head Start programs in general, you see immediate impacts on children’s IQ, but that these impacts “fade out” within a few years. I haven’t been able to get my hands on either of these originals, but Barnett characterizes McKey that way and Wikipedia characterizes Administrative History that way. And the tone of pro-Head Start papers and advocates is often on the defensive against this idea, so somehow it seems to have gained at least popular traction.

Barnett’s literature review has a section on studies of larger Head Start programs (see pages 232-236 for another awesome grid), and it’s certainly less impressive than the model programs. Any statistically significant difference in test scores fades by around 3rd grade, in 12 of the 16 studies (2 of these 12 have it fading only for certain ethnicities). On the other hand, when you look at measures of performance rather than intelligence – particularly focusing on issues most relevant to struggling children, such as grade retention (i.e., getting held back) and special education, you see pretty strong and pretty persistent positive effects of these programs.

Almost all of these programs have significant methodological problems. None of them are randomized, and to me that’s a real issue with Head Start, an optional program. I find it very counterintuitive that one year, no matter when, could make all that much difference, and it’s more plausible to imagine that parents who enroll their kids in Head Start are the same parents who are likely to push harder for them to stay out of special ed and move to the next grade. Then again, in light of the similar (though stronger) results for model programs, as well as a couple of the patterns I describe below, there is some reason to take the findings at face value.

Barnett comes out of this grid feeling pretty sold on Head Start. I come out of it feeling much more mixed. With low confidence, I think there’s a chance that these programs are highly beneficial for the kids who need the most help, as I’ll explain in the next section.

What works?

Unlike with early childcare programs (below), I wasn’t able to find much on the definition of what constitutes “quality” in preschool. What follows are my own observations from Currie’s & Barnett’s grids (you can check them yourself and see if you agree):

  • Results are more impressive for programs that went beyond a year, and went beyond part-day (both of which Head Start is generally restricted to). This makes sense to me. I can believe that an intensive program that takes care of a kid for his/her entire early development can affect the development of the brain and basic attitudes; I have more trouble with the idea of one year (age 4) in isolation making that much of a difference, although I suppose it could be that crucial of a year in development.
  • All of these programs seemed to do a better job with things like keeping kids from requiring special education, or getting held back, than with IQ and test scores. That makes sense to me too. I’d guess that good child care has more to do with keeping kids out of bad situations than with turning them into geniuses – I think (without any evidence) that it’s easier to screw someone up than to make them go above and beyond their natural abilities.
  • In light of that, it’s also striking to me that the strongest results (from the Abecedarian program) come from a program that specifically sought out “at-risk” children, rather than simply low-income children. Also, looking at the Barnett grid, it seems that the methodologically strongest studies showed the strongest effects – the opposite of how it usually is (as selection bias usually exaggerates results). Why would this be? One possible answer, to me, is that the methodologically strongest studies were the ones that used randomization rather than after-the-fact comparisons, and that means they were the most careful in picking out people to study. I didn’t get to look at the Perry preschool study, but it may have done the same sort of selection – specifically targeting at-risk children – as the Abecedarian study, whereas the weaker studies would have no good way of doing this since they didn’t pre-pick their populations.
  • So, I don’t know everything or even much, but my money’s on intensive programs targeting highly disadvantaged and at-risk children. And in the end, I don’t think child care is rocket science – if I see a program that’s doing the same basic activities as the Abecedarian project, and its personnel are qualified and (from a site visit) seem to like children, I’m going to think it’s relatively reasonable to expect similar results, i.e., a real effect on children’s development.

Early childcare programs

As this article details, there has been a good amount of media coverage of the idea that the ages of 0-3 is really where it’s at, because that’s when the brain is really developing. As the article also explains, the actual science is much more mixed. I don’t have much literacy or opinion of the actual neuroscience, except I know that it would have to be pretty slam-dunk cause-and-effect reasoning to sell me without accompanying evidence of actual results, and it doesn’t appear to be slam-dunk (based on the review in the article at the beginning of this paragraph). So I look for studies of whether differences in early childhood care have actually been shown to lead to better later outcomes.

Studies on this are pretty scarce, probably because there’s no policy debate attached. The main source I can find is the NICHD, whose huge study on of early child care and youth development is available here. I haven’t read the study carefully, but I’ve scanned it and have a couple observations:

  • It doesn’t look at later-in-life outcomes the way the Head Start studies do (it goes up to age 4.5), and given the “fade out” often observed by the latter, I’m hesitant to get too excited about its relatively modest demonstrations of effects.
  • It also doesn’t have a randomized design. It’s a survey: they look at the connection between quality of child care and child performance along various metrics, meaning they could easily just be showing that families that care more about getting good child care are also doing other things to help their children (or just care more about them!)
  • One appealing thing about the NICHD study is that it’s very specific about what constitutes “quality” child care. Unlike the Head Start studies, which are all about debating “whether it works,” the NICHD gives a nice worksheet that I’ve printed out for us, listing the qualities of good child care that it has linked to better performance, some easily quantifiable and some not.

It makes just as much logical sense to me that 0-3 child care matters as that 3-5 education-focused care matters. Just as much, no more, no less. The evidence for the latter is much stronger. So that’s where I stand for now. A particular charity with a particular approach, and good measurement, could easily change my mind.

Parent outreach programs

Teaching parents to take care of their kids seems great in theory, but the question is, are you really changing parents’ behavior or just giving them a bunch of advice they’re not taking? I’m open to seeing a parent outreach program that really has a way of changing parents’ behavior, but preliminarily, I’m more excited about the more child-focused programs.

The Gomby study is pointed to by many papers (including Schaefer, see page 6) as establishing that there isn’t much to these sorts of programs. I haven’t had a chance to look carefully at Gomby itself, and I should. Early on, it states, “In most of the studies … programs struggled to enroll, engage, and retain families. When program benefits were demonstrated, they usually accrued only to a subset of the families originally enrolled in the programs, they rarely occurred for all of a program’s goals, and the benefits were often quite modest in magnitude” (Gomby, 6). That quote in itself isn’t all that sobering to me – I’d expect any program that only addresses one factor to show only modest effects.

Much more sobering is one of the studies it references, the St. Pierre et al. study, which took an extremely thorough, completely randomized look at a large federal “experiment”, the Comprehensive Child Development Program, which centered around advising families rather than providing particular services. There is a huge amount of data here, and the program seems to have been reasonably well carried out, and the randomized design found nothing. Pending what I find in the Gomby paper, I feel like it’s reasonable to conclude that (consistent with logic) engaging parents is tricky, and it’s going to take ingenuity, not just execution of a basic plan (while I feel that Head Start-type programs can work with the latter). Again, I’m totally open to a particular charity’s showing me that it’s cracked this nut and can prove it.

References

These are just the papers that either are not available online or that I refer to repeatedly. When I just talk about a paper once and it’s available online, I just link to it (above).

  • Administrative History of the Office of Economic Opportunity, Vol. I, p.252, Box 1, LBJ Library.
  • Barnett, W.S. (2004). Does Head Start have lasting cognitive effects? The myth of fade-out. In E. Zigler & S. Styfco (Eds.) The Head Start Debates. Baltimore, MD: Paul H. Brookes Publishing Co.
  • Campbell, F.A., and Ramey, C.T. (1994). Effects of early intervention on intellectual and acaademic achivement: A follow-up study of children from low-income families. Child Development, 65, 684-698.
  • Campbell, F.A. and Ramey, C.T. (1995). Cognitive and shcool outocmes for high-risk African-American students at middle adolescence: Positive effectds of early intervention. American Educational Research Journal, 32, 743-772.
  • Campbell, F.A., Ramey, C.T., Pungello, E.P., Sparling, J., & Miller-Johnson, S. (2002). Early Childhood Education: Young Adult Outcomes from the Abecedarian Project. Applied Developmental Science, 6, 42-57.
  • Currie, J. (2001). Early Childhood Education Programs. Journal of Economic Perspectives V. 15 #2, pp. 213-218.
  • Gomby et al., 1999. Home Visiting: Recent Program Evaluations: Analysis and Recommendations. The Future of Children, Vol. 9, No. 1.
  • McKey, R.H., Condelli, L, Ganson, H., Barrett, B.J., McConkey, C., & Plantz, M.C. (1985). The impact of Head Start on children, families, and communities. (DHHS Pulication No. (OHDS) 90-31193). Washington, DC: U.S. Government Printing Office.
  • Schaefer, S. and Cohen, J. Making Investments in Young Children: What the Research on Early Care and Education Tells Us. National Association of Child Advocates, Washington, D.C.
  • Schweinhart, L.J., Barnes, H.V., Weikart, D.P., Barnett, W.S., and Epstein, A.S. (1993). Significant benefits: The High/Scope Perry Preschool study through age 27. Monographs of the High/Scope Educational REsearch Foundation. No. 10. Ypsilanti, MI: High/Scope Press.
  • St. Pierre et al.. 1997. National Impact Evaluation of the Comprehensive Child Development Program. U.S. Dept of Health and Human Services.

Still reading?

You get 5 points.

Comments

  • Jason R on July 25, 2007 at 7:11 am said:

    Even if you grant the likelihood that these programs have some lasting impact on children’s lives, it seems counter-intuitive to me that helping a child in a bad situation when they are 3-4 and then leaving them alone would be more helpful than say helping that same child in grades 6-12 (academic help + social help – even if you exclude actually making the school better). To me it’s not even a philosophical question of what kind of cause I am passionate about, because you can narrow the question to helping one child at different stages of their development. What will help a child more… help early in life when potentially neurons are connecting (if this is true) or help later on when the child will actually remember/learn from the experience.

  • Jason R on July 25, 2007 at 7:15 am said:

    I also want to point out a major shortcoming of this interface. You can use pictures to make your point while responders cannot (as far as I can tell), which puts us at a disadvantage and may eventually cost lives. I am much more compelling when I can use pictures (you know my work). While there is a chance that some porn will filter through, that is a risk we all take when we get out of bed every morning.

  • Holden on July 25, 2007 at 7:31 am said:

    “What will help a child more… help early in life when potentially neurons are connecting (if this is true) or help later on when the child will actually remember/learn from the experience.”

    Neither of us knows the answer to this. Most of our cause separations are around strict philosophy (what do you value), but the split between Cause 3 and 4 is an exception. That split is much more about taking a monstrous cause (education), and finding one of the questions within it that is (a) one of the hardest-to-resolve debates (one that basically requires you to lean entirely on your intuition, which will vary tremendously from person to person, usually in ways that correlate with their philosophical values anyway – i.e., those with more “conservative/libertarian” values will generally be more open to the idea of helping brain development than of later socialization); (b) a pretty clean way of separating charities.

    So, the split is what we have for now, and my concern is more with figuring out where to grant within Cause 3 than with comparing it to Cause 4. My intuition matches yours that there’s more to gain by helping kids later on in life, but within Cause 3, this is where I’m betting. What do you think?

  • John Brothers on July 25, 2007 at 10:27 pm said:

    From my family having a couple Head Start kids and having come from homelessness and poverty, my experience with what works and does not work in early childhood has to do with organizations having the capacity and where-with-all to track the development of young children. My community growing up had over 200 organizations aimed at “alliviating poverty” and each year, including early childhood indicators relating to poor children, became worse and worse. Large reason for this is the failure of networks working together to streamline all of these social efforts. Now, that was a while ago, but the current social and human service landscape is not much different.

    As a former non-profit consultant, I viewed hundreds of programs, many early-childhood, and what I saw many times would be an equivalent to babysitting. What I found most surprising is that these “babysitting” groups were often highly regarded by some of the most respected funders.

    The programs that were the best were the ones that were, at any given time, able to show that an individual child they knew by name was moving along a developmental continuum. This is hard and I would view any organization or effort that is moving in this direction as ahead of the game.

    Now, this takes a couple things. A moderately well trained staff, the more training the better obviously. Second, the internal infrastructure to track the development of these children, beyond the typical human service/social service bean counting.

  • Holden on July 25, 2007 at 10:32 pm said:

    Thanks for the comment.

    Do you happen to know of any hard evidence for this claim, that tracking children along their developmental curriculum leads to better results? If so, that would be fantastic. If not, I will take this claim as what it is: an informal observation by a self-described experienced party, whose logic makes sense to me.

  • Anonymous on July 31, 2007 at 10:30 am said:

    “I feel like it’s reasonable to conclude that (consistent with logic) engaging parents is tricky, and it’s going to take ingenuity, not just execution of a basic plan (while I feel that Head Start-type programs can work with the latter).”

    This is absolutely true, it is harder to engage families, but it results in better outcomes. You’re right, it’s not enough to give parents a bunch of advice that they won’t use. But giving parents (who are often at a loss of how to manage their children) a toolbox of parenting skills to use, and then practicing and role playing these skills, does work. Improving parenting is like giving the child an intervention 365 days a year. One evidence based parenting program that’s been used in Head Start is the Incredible Years training. I’ll send you some articles on this Holden.

  • Anonymous on July 31, 2007 at 10:33 am said:

    Furthurmore, if a program is is successful at engaging and retaining families, it is a sign that this program has a well trained staff who understands the needs of the population they are serving. This criteria could be used by Givewell to identify exemplary charities.

  • Stephanie Hope on July 31, 2007 at 10:35 am said:

    Oops, I didn’t mean to be anonymous on my comments (6 & 7). It’s me, Stephanie.

  • Margaret Simms on July 22, 2008 at 7:58 am said:

    Good afternoon from Nottinghamshire.

    Here is my contribution to the debate on quality in childcare.

    Responsibility for poor quality childcare has been directed as much at practitioners as it has at providers. Cited in defence of practitioners is a lack of training opportunities which, together with poor pay and conditions, lead to high turnover of staff {{487 Cameron, C. 1997; }}. Evidently there is a need to raise standards and improve quality of provision {{411 Office of Public Sector Information 1989; }} and practice. Quality in it purest form equates to high standard {{424 Soanes, C. 2008; }} but the baseline for quality in Early Years and Childcare provision in England has been set at good {{414 Office for Standards in Education 2001; }} and Standards have been introduced to help providers achieve good quality {{414 Office for Standards in Education 2001; }}. However, even though providers and parents agree that quality childcare comes as a result of well trained and rewarded staff who are committed to their work {{428 Daycare Trust 2004; }}, there is no reference in the Standards of the need to value and reward practitioners who display these quality characteristics.

  • Good Therapy on August 10, 2008 at 2:17 pm said:

    I agree with you, “Children are our Future” and deserve the very best. I was not sure the differences between the programs until I came across your site. There does not seem to be an overlap between the programs. We need to have all children involved in some type of program from early childhood through school. Especially if the parents decline to be involved. Other countries gaining on the United States, as we have focused on more other things instead of our children. We are a very fortunate country as we can have as many children as we want, versus other countries that limit couples to one child per couple only.

  • Mr Fairfield on March 13, 2009 at 9:14 am said:

    i am going to look into child care soon, and im glad to have some real content sites to read about now in advanced. its such a hard decision and you want the best for your child. I just wish they were less expensive, but sometimes you get what you pay for

  • Michele Rodriguez on March 20, 2009 at 3:35 am said:

    I realize you are addressing early child care here, but I wanted to point you to the Freedom Writer’s Foundation which focuses on high school intervention for disadvantaged, high risk students.

    The Foundation began accidentally by a teacher who cared. It’s an interesting case because it takes students who were classified high risk for drop out and disadvantaged and then transforms them into successful college graduates. The founder and teacher, Erin Gruwell, cared deeply enough to fully engage herself with her students and act as their advocates. This resulted in a program that promoted empowerment through reading the students related to, relative and exciting field trips, and self story telling through journal writing.

    The original results, of having students who were all but failing go on to graduate college, shows that intervention can happen as late as high school and be successful.

    I think it speaks volumes about how important it is to have the right staff (or volunteers) when designing and implementing a program. In my opinion, this holds true for any youth development program.

Comments are closed.