This year, Dr. Kevin Croke, a post-doctoral fellow at the Harvard School of Public Health, released a study that we consider an important addition to the evidence for deworming children. The study (Croke 2014) followed up on a randomized controlled trial (RCT) of a deworming program in Uganda and found higher scores on tests of literacy and numeracy in children living in treatment areas compared to the control 7 to 8 years later. This finding reinforces the findings of the only other RCT examining the long-term effects of deworming, which we had previously considered to be relatively strong but still had substantial reservations about. By providing a second data point that is free of some of our previous concerns, Croke 2014 substantially changes our view of the evidence.
Two of our top charities, the Deworm the World Initiative (DtWI) led by Evidence Action and the Schistosomiasis Control Initiative (SCI), focus on deworming. We have not yet concluded our examination of Croke 2014, but at this point we think it is likely to lead us to view these charities as more cost-effective.
Overview of Croke 2014
Croke 2014 follows up on an RCT that involved 48 parishes (the administrative level above the village and below the sub-county) in 5 districts of Uganda selected based on the prevalence of worms in the districts. Half of the parishes were randomly assigned to a treatment group and the other half to a control. In all the districts, community organizations delivered basic health services, like vitamin A supplementation, vaccination and growth monitoring, through regular Child Health Days (CHDs). Children in the treatment group received albendazole (a deworming drug) during CHDs in addition to the other services offered, while children in the control just received the usual services.
Croke analyzed surveys conducted by an education nonprofit several years later that happened to sample 22 of the parishes in the original RCT. He compared children living in the treatment parishes sampled who were 1 to 7 years old (the age group offered albendazole) at the time of the program to children of the same age living in the control parishes and found children in the treatment group had scores about 1/3 of a standard deviation higher on tests of literacy and numeracy.
Strengths and significance
Few other studies have rigorously examined the long-term effects of deworming. Up until now, we’ve relied heavily on two studies: (a) Bleakley 2004, a study of the Rockefeller Sanitary Commission’s campaign to eradicate hookworm in the American South in the early 20th century; (b) a series of studies in Kenya, in which school deworming was rolled out on a purposefully arbitrary (randomization-like) basis, and children who received more years of deworming were compared to children who had received fewer. These studies suggest the possibility that deworming children dramatically improves their productivity later in life by subtly improving their development throughout childhood. In our view, the case for deworming largely rests on these long-term, developmental effects, because the intervention seems to have few obvious short-term benefits.
Having two relatively recent RCTs from Sub-Saharan Africa increases our confidence in long-term benefits far more than having just one RCT, especially because we have had substantial reservations about the RCT in Kenya – some of which seem notably less applicable to Croke 2014. Specifically:
- The earlier RCT was a trial of “combination deworming” – treatment of both schistosomiasis (with praziquantel) and soil-transmitted helminths (with albendazole). Croke 2014 looks only at albendazole. This is particularly important because one of our current top charities – the Deworm the World Initiative – operates largely in India, where only albendazole is used.
- Regarding the earlier study, we also thought it was plausible that efforts to encourage students to attend school in order to receive treatment might have accounted for some of the effect found in Baird et al 2012 (the follow-up of Miguel and Kremer 2004). The intervention examined in Croke 2014 appears far less likely to introduce other positive changes into the treatment group, because it involves the addition of albendazole to an existing program rather than an intensive program of deworming in schools in the treatment compared to the absence of any program in the control.
- We worried that the results of Miguel and Kremer 2004 (the RCT in Kenya) might not generalize to other areas, because of extraordinary flooding caused by the El Niño climate pattern during the study and abnormally high infection rates in the study area (more). Croke 2014 appears to have had a lower (though still high) initial prevalence of infections (the programs selected districts based on high rates of worms found by Kabatereine et al 2001, which estimated that 60% of children ages 5-10 were infected, primarily with hookworm). El Niño may still have affected the study, however, because the parishes examined in Croke 2014 are very close (some within about 10 miles) to the district in which Miguel and Kremer 2004 took place. The program evaluated in Croke 2014 started about 2 years after El Niño, but we’re not sure whether this amount of lag time would lead to lower or higher infection rates.
In our current cost-effectiveness analyses for deworming, we have a “replicability adjustment” to account for the possibility that Baird et al 2012 wouldn’t necessarily hold up on replication, as well as an “external validity adjustment” to account for the fact that most deworming programs likely take place in less heavily infected areas. We will be revisiting both of these adjustments, resulting in improved estimated cost-effectiveness for deworming.
We still have some concerns about the evidence.
- El Niño may have affected the parishes examined in Croke 2014 just as it affected the schools in Miguel and Kremer 2004, potentially causing unrepresentatively high infection rates and limiting the generalizability of both studies.
- Though Croke 2014 finds a large increase in test scores, Baird et al 2012 does not.
- We worry about the sensitivity of the results to the outcome and control variables used in regressions and about selective reporting of results.
- We also worry about publication bias. Perhaps other parish-level surveys would have supplied other outcomes. We wonder if other analyses employing a similar methodology that did not find an effect would have been published.
- The study included a relatively small number of clusters. Croke 2014 reports on a few different regressions and methods of calculating the standard error of the treatment effect, which lead to different estimates of that standard error. In one more conservative analysis, for instance, the effect on the combined literacy and numeracy test scores is significant at the 90% confidence level.
- Finally, we still can’t articulate any mechanism for the long-term benefits of deworming supported by data (we haven’t seen notable impacts on weight or other health or nutrition measures).
We have not yet concluded our examination of Croke 2014, though we have looked it over closely enough to feel that it is very likely to result in a substantial positive revision of our view on deworming, and therefore of our views on two current top charities as well.
In combination with the earlier study, Croke 2014 represents a major update regarding the case for deworming; we’re very glad to see this new evidence generated, and hope that it will become a prominent part of the dialogue around deworming. We intend to do our part by updating our content for this giving season.