We are in favor of scaling up proven programs, but against the Promise Neighborhoods initiative.
As far as we know, the only evidence that the Harlem Children’s Zone (or any similar approach) has been effective is the relatively recent study showing impressive effects on test scores at its charter schools. We discussed this study in a four-part series and concluded that:
- The effect demonstrated was extremely impressive and unusual.
- There are serious questions about how “real” the effect is (to what extent did it come from narrow “teaching to the test?”), how likely it is to be sustained as opposed to temporary, and how significant it is in terms of likely effects on actual life outcomes.
- These questions aside, there are also major questions about just what aspect(s) of Harlem Children’s Zone are crucial and whether they can be replicated at all, let alone at a reasonable cost.
Given this situation, we don’t feel it’s time to attempt a replication in 20 communities at once, at a cost that seems likely to stretch into the billions if and when these replications are fully carried out.
We’re not just concerned about mis-spending money. We’re concerned about overreacting to evidence, overpromising results, and thus damaging the credibility of future proposals along these lines. We’re concerned that the funds will be allocated, the Promise Neighborhoods will be rolled out, and 10 years from now we’ll check back and see no narrowing of the achievement gap.
We hope that someday, there will be a truly replicable program with an extremely strong case that it can put a significant dent in the achievement gap. If and when that day comes, a failed Promise Neighborhoods scale-up – and any other oversold programs – will come back to haunt us.
We feel it is appropriate to pursue some replication of, and experimentation with, the Harlem Children’s Zone model. We feel a rollout of this magnitude would be a mistake.
Comments
I absolutely concur with this take on the replication. Too many variables, too much going on that cannot be effectively reproduced in another community with unique insitutions, associations and characteristics. I worked in national deliquency programs in the late 70’s and found nothing but trouble in replication projects – this one will not be an exception – just more zeros.
Correct me if I’m wrong, but reading the post from ed.gov that you posted, it doesn’t sound to me like communities will be forced to have exact replicas of the Harlem Children’s Zones. All types of local nonprofits are being invited to submit grant proposals outlining an education-centered full-continuum for improving the ability for youth to have improved health, safety, & learning. Though it also sounds like this is coupled with some ed policy reform (which may be open to critique another time).
It seems like this significant investment will help spur new ideas like Harlem Childrens Zone to help find more solutions that work.
There may be questions of how effectively ed.gov reviews & selects proposals, but that isn’t something new for them or many other government departments.
Im encouraged to see government investing in proactively investing in opportunities for community organizations to innovate and make an impact.
Kevin, that’s an interesting point.
When I read the linked post, it didn’t seem to me like it was discussing a new set of “experiments to find out what works.” There is no mention of taking diverse approaches, conducting formal evaluation, etc. – the emphasis is directly on transforming neighborhoods and improving outcomes, not generating knowledge.
That said, it’s not entirely clear – as you point out, there is no apparent attempt at “exact replication” or explicit statement that outcomes will be comparable to HCZ’s.
For reasons discussed in the post, how the project is presented seems key. If it is presented as a way to close the achievement gap, disappointing results will undermine future attempts; if it is presented as a way to experiment and gain new information, that’s a different situation. At the least, some clearer communication on the initiative’s goals seems warranted.
Seems worthwhile to point to the recent Brookings report on the Promise Academy, HCZ’s flagship charter school. Also, Geoffrey Canada’s reply (PDF). Even to the extent that Canada’s reply is correct (which I think is quite likely), there remains little to no positive evidence about the effectiveness of the non-charter school elements of the HCZ.
Comments are closed.