Summary
The Deworm the World Initiative (DtWI), led by Evidence Action, received approximately $2.3 million as a result of GiveWell’s recommendation last year. While there were some deviations, it largely allocated these funds as we expected.
DtWI now has limited room for funding; it is currently seeking to raise an additional $1.3 million to support its activities in 2015 and 2016. We expect it to allocate approximately 30% of the additional funds it receives for work related to expanding school-based, mass deworming programs and funding related operating expenses (including impact evaluation related expenses), and will allocate other funds to priorities that are less directly connected to expanding and evaluating deworming programs (investigating ways to combine other evidence-based programs with deworming rollouts, supplementing a project supported by another funder).
We currently expect to release updated recommendations by December 1st. We think it is likely that the Deworm the World Initiative will remain on our top charities list.
How did DtWI spend the money it received due to GiveWell, and how does this compare to our expectations?
GiveWell directed approximately $2.3 million to the Deworm the World Initiative since we added it to our top charities list in December 2013.
At the time of our recommendation, we expected DtWI to spend additional funds in the following ways; we did not have precise estimates for how much it would spend in each category:
- Some portion to provide reserves for DtWI, both to make the organization more resilient and to allow it to respond to high impact opportunities
- Some portion to allow DtWI to offer a lower-intensity level of assistance to regions that didn’t require its standard level of assistance
- Some portion to support expansion to new states in India
It has allocated these funds as follows (years when we expect funds to be spent in parentheses; 2014 means funds have been spent):
- $881,000 – ongoing reserves. Our understanding is that DtWI does not have plans to spend these funds in the near future. Instead, these funds make DtWI more robust as an organization: for example, it is less likely to need to significantly shift priorities in order to fundraise and it is more likely to be able to respond quickly to high-impact opportunities it identifies.
- $509,000 – expansion into new countries (2015 and 2016). This includes preliminary work in Ethiopia, Indonesia, Philippines to support possible future work and $104,000 for prevalence surveys and technical assistance to the government and partner organization in Vietnam.
- $430,000 – ongoing work in India (2014 and early 2015). This will fund a follow-up prevalence survey in Bihar to assess the impact of three rounds of deworming on worm prevalence and intensity, and enable expansion to preschool children there, as well as contribute to the third round of the Rajasthan and Delhi programs.
- $207,000 – contribution to elimination research primarily funded by the Children’s Investment Fund Foundation (CIFF) and the Bill and Melinda Gates Foundation (BMGF) (2015-2017). CIFF and BMGF provided approximately $1.6 million in funding to the Deworm the World Initiative and the London School of Hygiene and Tropical Medicine to conduct research on the feasibility and cost effectiveness of breaking transmission of soil-transmitted helminths. Breaking transmission would potentially require a different approach (likely covering more than just school-aged children) than DtWI’s standard school-based deworming model.
- $151,000 – DtWI overhead (2014). These funds support DtWI as an organization but are not directly programmed (e.g., a portion of Alix Zwane’s, the Executive Director of Evidence Action salary, Evidence Action financial staff, etc.). Note that DtWI estimated $151,000 based on allocating 15% of programmed GiveWell-sourced funding to DtWI overhead. DtWI said it could more explicitly track these funds but would be time consuming to do so. We agreed that more detailed accounting was not necessary.
- $129,000 – additional staff (2014). In 2014, DtWI hired (a) a deputy director to support its programming worldwide and (b) someone to focus on its impact evaluation. The latter hire is likely to be doing work on the breaking transmission work discussed below. We allocate some of this line item to expansion and related operating expenses and some to research.
Overall, DtWI’s funding decisions seem reasonable to us and are broadly consistent with what we anticipated.
- 46% ($1,067,000) supported expanding deworming programs and funding related operating expenses (including impact evaluation related expenses). This includes the deputy director who supports the organization as a whole but is necessary to expanded work in India and other new countries and half of the salary for the impact-evaluation-focused new staffer since he works on programmatic and technical support across DtWI.
- 38% ($881,000) supported ongoing reserves.
- 10% ($241,000) supported research that we had not anticipated (including the other half of the new staffer since he is spending a significant part of his time on this research).
- 6% (151,000) supported DtWI as a whole.
How would DtWI spend additional funds?
The Deworm the World Initiative seeks an additional $1.3 million to support its activities in 2015 and 2016. DtWI expects to spend $377,000 of the $1.3 million (29%) it seeks on work related to expanding school-based mass deworming programs and funding related operating expenses (including impact evaluation related expenses). More specifically, these activities would be:
- $230,000: staff to support expansion in India, new countries, and related operating and evaluation expenses. This line item is the salary for the deputy director and part of the salary for the impact evaluation focused staff member described above.
- $144,000: DtWI overhead (described above).
- $500,000: evaluation of new evidence-based programs that leverage deworming. We have limited detail about what this would entail. One idea that DtWI has investigated is the possibility of distributing bednets along with deworming pills in schools as an alternative distribution mechanism to national net distributions. Another is including hand-washing educational programming alongside deworming days. This line item includes $50,000 to support DtWI’s evaluation of its hygiene and deworming program funded by Dubai Cares and $50,000 to enable DtWI to hire a senior epidemiologist.
- $230,000: staff to support evaluation of DtWI’s work in Kenya. This work is primarily funded by CIFF. DtWI believes that additional resources can improve significantly the quality of the analysis done regarding the cost effectiveness of breaking transmission. This line item includes $100,000 to support the impact evaluation focused staff member described above.
- $170,000: implementation support for the integrated deworming, sanitation and hygiene education program in Vietnam, in partnership with Thrive Networks.
Why is DtWI seeking additional funds primarily to support research and evaluation rather than scale up? What changed in the past year?
In 2014, two events affected DtWI’s projection of the additional funding it would require to scale up in India:
- The Children’s Investment Fund Foundation (CIFF), a major foundation that had supported DtWI’s programs in Kenya, agreed to a 6-year, $17.7 million grant to support DtWI’s expansion to additional states in India and technical assistance to the Government of India for a national deworming program. At the end of 2013, DtWI believed it was reasonably likely that it would not receive this grant and had not anticipated how quickly it would come through. With these funds, DtWI does not require significant additional funding to support its India expansion.
- The new Indian government expressed interest in conducting a single deworming day nationally with increased national attention and resources. Advocating for such a policy and assisting the national government in creating a plan became the major focus of DtWI’s India work in 2014, which both reduced the amount of time it was able to spend generating interest in heavy DtWI involvement in new states, and also required little funding since there were few costs of that project aside from staff time. DtWI believes that the first national deworming day will likely happen in February 2015.
Together, these changes led DtWI to the conclusion that funding is no longer the bottleneck to reaching more people in India.
More broadly, we believe that if donors close both the $1.3-million 2-year funding gap of DtWI and the ~5-$8-million funding gap of the Schistosomiasis Control Initiative (SCI), another deworming organization we recommend, funding will not be the primary bottleneck to deworming programs’ scaling in general. Overall, our impression is that there is currently more funding available for scaling up deworming programs than capacity at organizations to utilize funds for scaleup.
Dr. Zwane believes that DtWI’s research agenda is important for two reasons:
- She believes it is possible that this research will demonstrate that other approaches to deworming are more cost-effective, such as eliminating worms from areas to avoid the need for mass treatments, or combining deworming with other interventions such as bednet distributions or hygiene education.
- She would like DtWI to consistently provide useful information to funders and policymakers and undertaking this research will enable it to continue doing so.
Notes on other deworming implementers and funders
It is not unlikely that GiveWell-directed donors will close the funding gaps of both DtWI and Schistosomiasis Control Initiative in the coming few months. Because of this, we also asked Alix Zwane (Executive Director of Evidence Action) about other implementers and funders working on deworming.
Implementers
Dr. Zwane told us DtWI and SCI are the two primary organizations that focus primarily on expanding countrywide deworming programs. Other organizations work on deworming but are not as directly focused on scaleup with government partners to her knowledge. There are other NGOs that work on other neglected tropical diseases (e.g., SightSavers) and school health (e.g., Partnership for Child Development), but Dr. Zwane is less familiar with the reach and scope of the service delivery they support.
Organizations that do a smaller amount of deworming implementation include UNICEF, Micronutrient Initiative and Vitamin Angels, which have begun adding deworming pills to their vitamin A supplementation programs, and WaterAid, which adds deworming to some of its water and sanitation programs.
IMA World Health, Helen Keller International, Sight Savers International, The MENTOR Initiative, and possibly others implement deworming programs supported by the funders discussed below. We have yet to speak with these organizations and have little information about their deworming programs or funding needs.
According to Dr. Zwane, the Global Network for Neglected Tropical Diseases works primarily on advocacy and does not focus on deworming, specifically, while Children without Worms coordinates partners globally and does not work on providing technical assistance for program delivery directly currently to her knowledge.
Funders
Major funders of deworming service delivery include the following: Dubai Cares, The END Fund, CIFF, the UK government’s Department for International Development (DFID), Michael and Susan Dell Foundation, and the US government’s USAID.
According to Dr. Zwane, these funders are interested in supporting scale-up, and she believes that DtWI will be in a strong position to raise funds for scale up from them if and when funding becomes a bottleneck. These funders are less likely to fund the types of activities to which DtWI has allocated GiveWell-directed funding.
A longer list of organizations working on deworming is available in this document, from a recent meeting of groups that are part of the STH Coalition.
Comments
Of the funders you list, I think it’s only the END Fund that takes donations from the general public. Giving to them might be an alternate approach to giving directly to other implementers (assuming DtWI and SCI are fully funded). Do you have a view on whether that’s a better or worse approach? Per their website, they do cover some other neglected tropical diseases (trachoma and river blindness) in addition to deworming.
Colin, I know very little about The END Fund, so it’s possible that it’s a decent option if DtWI and SCI’s funding gaps are both filled.
Two additional thoughts:
How come the articles on the top-rated charities say Schistosomiasis Control Initiative (SCI) needs “around $1.23 per child” to deworm a child and Deworm the World Initiative (DtWI), led by Evidence Action, needs “a total of about $0.30 per child, or $0.09 per child excluding the value of teachers’ and principals’ time spent on the program”. Is this a typo or is DtWI really several times as cost-effective as SCI?
Hi Stephen — good question. It’s not a typo, but we’re also not entirely sure what drives the difference in cost. I’ve given a brief answer below. Before I posted this comment, I ran it by the GiveWell analysts who led our work on the SCI and DtWI’s cost estimates. That discussion adds some additional color, so I’ve pasted that chat in below the line.
For reasons laid out laid out here, we include all costs of a program in our cost estimates not just those paid for by the charity, so for both DtWI and SCI, we include government costs in addition to the charity’s costs.
We have very limited information about the costs governments provide for SCI-supported programs. We base our estimate on a single study in Nigeria estimating that government costs account for approximately 30% of total program costs. (More detail on this here.)
One difference we do know about is that DtWI and SCI treat different parasitic infections that require different pills. DtWI requires albendazole which costs approximately 3.5 cents per treatment; SCI requires praziquantel which costs approximately 30 cents per treatment.
Note that albendazole is almost always donated by drug companies and praziquantel is often (though not always donated). We include these costs in our calculations for reasons laid out here, though this decision is debatable.
Even if it is the case that DtWI spends less per treatment than SCI, it is still not necessarily the case that it is more cost-effective. SCI tends to work in areas where people suffer from multiple worm infections (schistosomiasis, as well as soil-transmitted helminths). DtWI works in locations that are not affected by schistosomiasis. Taking into account costs and the likely impacts of treating its targeted infections, our best (though extremely rough) guess is that DtWI-supported programs are 1-2x as cost-effective as SCI’s. You can see our cost-effectiveness models here and read more about how we think about cost-effectiveness analysis here.
Sean
25c difference is drug costs, and a little of the difference is government costs (but not much – government costs is most of DtW’s 30c costs and for SCI its ~40c). So the remaining difference is SCI’s incurred costs – which are 50c – vs DtW’s – which are 3c.
That seems to come from:
Is that from
a) DtW’s program really is hugely different since they are just doing technical advocacy
b) We’re letting DtW not include some central costs
c) What is SCI spending money on?
I’m guessing it’s mostly a) with some b)
I might be doing this estimating wrong but, we directed DtW $2.3 million last year, and if we’re saying that for every $237,000 that DtW has spent that they get out 10.8 million treatments, then for 2.3 million they should’ve gotten out >100 million treatments? I don’t think they’ve done that many? Seems like the DtW-only cost/treatment should be higher (ie, what we do for SCI is just do total money they’ve gotten and spent/total treatment’s they’ve caused to get the SCI-only costs)
Tim
@sean re: the number of treatments we’d expect, we didn’t expect all of that money to go to scaleup; much was used for core reserves: https://blog.givewell.org/2014/11/21/deworm-the-world-initiative-led-by-evidence-action-update/. That said, I don’t think we’ve settled the question of how to deal with money used for core reserves in our CEAs; I would argue that if an org wants 10% of annual expenses in core reserve, then 10% of scaleup costs (post-overhead) should be included in the ‘costs’ since it forces a larger core reserve. In this case we were also making up for a dearth of core reserves due to (arguably over-) restricted funding from other donors; I would argue that back-fill core reserves should be counted as roughly equivalent value to treatments as it is high organizational value and might be appropriately thought of as ‘leveraging’ other restricted donations. (If we adopted that approach we would leave back-fill reserves out of both costs and benefits side of our CEA).
Tyler
@sean the other things that came to mind for me when i poked around were (to continue your list):
d) is DtWI working in areas with way higher population density that gives them a large cost-effectiveness benefit (can treat way more people for the same costs)
e) are the norms for per diems in india very different than africa? it looked to me that it was possible that dtwi paid fewer per diems (for training but not the actual mda) and perhaps they are significantly smaller than what sci pays.
Tim
I’m not totally sure which drug cost number you want to use, but JTBC, for DtWI, the drug-only costs were 3.5c in 2013-2014, and the drug+drug management costs were 3.7
I would consider making a hat tip to what I imagine is making the largest cost difference, ~”SCI is funding governments to run the program, whereas DtWI is not putting any money in the hands of governments; instead only spending money to play a support role alongside the government run programs. It is not clear what the source of discrepancy in our estimates of total costs represents; it could be that pre-existing government structure and other external factors in India allow for health programs to be carried out through schools at lower cost than in the countries SCI works, that funding governments to run programs is much less cost-efficient than supporting them from the outside (without cash support), or that our cost estimates in India fail to capture major hidden government costs (that are covered by SCI’s cash gifts to governments in their cases).”
@sean Yeah we should do the same calculation for DtWI that we do for SCI (pretending we have less info than we do) and see what we come out with. I think you’re right that the number will be significantly higher for DtWI than it is now; we’re letting them not count things because they identify them as non-state program related, but we don’t allow SCI the same luxury.
Sean
Is that something you could do easily just to check what it comes out as?
I think for all the rest of the charities we try to count all costs (including making up a salary for Rob Mather which isn’t even a real cost) so it seems like we should be counting central costs for DtW if we’re not already (I know it gets a little weird because some of EA’s money goes on DSW and we’ve had discussions with them in the past in the context of DSW on what deserves to get counted and not counted towards the program, but we should try to get a best guess of DtW+EA costs that’s equivalent to our other charities)
Tim
Yes DtWI’s estimates are supposed to include central costs, but it gets tricker because they’re excluding Kenya (separate program) and exploratory activities for future rounds (since they do it on a per-executed round basis).
Sean
The Kenya thing is tricky. The exploratory costs I think we should be including, since I think we include similar things for SCI, and possibly AMF (I think we include all of AMF’s central costs but maybe there’s some we leave out? Natalie would know), and I think GD. By “similar things” I mean that I think we include all costs incurred for those, including sometimes costs that involve planning for the future, which exploratory costs seem to be.
Also arguably the way we’re doing for DtW is better, we just should be consistent if we’re not already.
Tim
Actually no, we don’t have the numbers we need to make this alternate calculation easily. What I want is “total 2013 DtWI costs” (arguably excluding Kenya, but even without); then I would credit them with the 3 recent distributions happening that year (even though Bihar was early 2014) and see what the rough breakdown was. But we don’t have that kind of over-arching budget data from DtWI; their financials have been a bit of a mess as they’ve been expanding rapidly and switching from IPA to Evidence Action.
Re what to include: I agree the consistency is the critical factor here, separate from which method is ideal:
Anyway, it’s clear that (as we’ve already noted) we need to do more to ensure consistency between these estimates. I would add to my earlier “hat tip” suggestion, “or that DtWI is failing to include organizational costs in its distribution-specific budgets that we are including for SCI (since we haven’t yet seen DtWI’s organization-wide expenses for 2013)”
Sean
Exploratory cost in year of actual program is probably fine in theory – but we’re not counting those at all at the moment anyway I think. Not including Kenya probably makes sense.
Going off numbers I have available easily (2015 budget):
So in 2015 they’re spending 6 million in India (plus maybe .8m centrally? plus .8m on new countries, which we should probably count in some year). So that’s $7.6m in costs. If there are 30 million treatments that’s an extra 25c/treatment which is almost a 2x change. (I have no idea how many treatments there are – does Rajasthan’s 10m represent one of many regions or is the biggest of the few they’ve worked in. Also some of those costs we could be convinced don’t matter)
Do we not have a similar budget for 2013? (or, a year in which we have data on # of treatments). I think this seems worth someone (Tim? Tyler? Me?) putting more time into
Tim
30M treated was true for the 2013-2014 round; they are planning to expand rapidly next year with a major CIFF grant. Even if their actual treatments stay at ~30M it seems like counting their total predicted expenses would make their cost per treatment seem unusually high due simply to an inflection point in scale-up (I’m guessing we’ve thought about/dealt with this issue in our GD estimates; though I don’t know how). That said my best guess is they get ~1 more state in 2015, bringing it to (Very roughly) 40M children (if we wanted a more precise estimate I would look at school-age population of Madhya Pradesh) and they certainly hope for much more than that in the next year or two (while plateauing at ~$10M budget for a year or two).
I’m actually surprised that taking their scaled-up 2015 expenses and dividing by their 2013-2014 children reached only creates a factor 2x difference with our current estimate. That makes me think (a) this possible problem is not as big as I was worried about earlier today, but still worth caveating in our response, and (b) I agree it’s worth Tyler/someone double-checking the consistency/thinking a bit more about what DtWI numbers we can scrap together to compare with our SCI estimate as a double-check to our existing methodology for DtWI CEA (which still seems like a reasonable best guess but may not be consistent with our SCI estimate methodology).
I’ve been looking forward to the post referred to here: “our revised cost-effectiveness estimates (which we will discuss more in a future post)” Is that still in the works? Thanks!
Tim – it is. Sorry for the delay!
Hi Tim et al, Grace Hollister from Evidence Action’s Deworm the World Initiative here.
We just noticed this comment thread. Would have been great to get a heads-up on this as we would have loved to respond in a more timely manner.
There are a number of statements included in your internal chat I’d like to chime in on:
You say: “One difference we do know about is that DtWI and SCI treat different parasitic infections that require different pills. DtWI requires albendazole which costs approximately 3.5 cents per treatment; SCI requires praziquantel which costs approximately 30 cents per treatment. “
In fact, the story is more nuanced than this. GiveWell has focused on Deworm the World’s programs in India, which treat only for soil-transmitted helminths (STH). In Kenya, the national deworming program that is also supported by Deworm the World treats for both STH and schistosomiasis. We therefore use both albendazole and praziquantel in Kenya. Albendazole (and the other drug to treat for STH, mebendazole) is available for donation for school-age children only. Therefore, for programs in India that treat preschool-age children, such as those in Rajasthan and Delhi, the state governments purchase the drugs for the preschool-age population. Both of these state governments have opted to buy albendazole syrup, which is more expensive than tablets but easier to use with small children. Regardless of whether the drugs are donated or purchased, the cost (either direct or imputed) is always included in our program costs.
2. You say: “DtWI works in locations that are not affected by schistosomiasis.”
In fact, in Kenya we support the National School-Based Deworming Programme which treats for schistosomiasis. In Kenya, the Programme with support from the Deworm the World Initiative treated 6,405,645 children for STH of which 890,459 children were also treated for schistosomiasis in districts where schisto is prevalent. Schistosomiasis is not prevalent in India.
3. You say: “DtW’s program really is hugely different since they are just doing technical advocacy.”
We are unsure how you define “technical advocacy.” One contribution that Evidence Action’s Deworm the World Initiative makes is to promote school-based deworming to governments in countries heavily affected by parasitic worms as we believe that school-based deworming is good public policy. We work with governments to provide a policy and financial foundation for deworming programs. But we also provide a full range of technical assistance to our government partners, including:
* conducting worm prevalence surveys to develop treatment strategies and understand impacts ,
* developing training and community awareness materials, and
monitoring and evaluating deworming programs to understand program coverage and quality.
4. You say: “We’re letting DtW not include some central costs.”
Our reported program costs include the full cost of Deworm the World’s technical support. We do not include reserves in our program cost calculations but do include organizational overhead costs, which are essential to our programming.
5. You say: “I might be doing this estimating wrong but, we directed DtW $2.3 million last year, and if we’re saying that for every $237,000 that DtW has spent that they get out 10.8 million treatments, then for 2.3 million they should’ve gotten out >100 million treatments?”
The program cost data that we have provided to Givewell is for specific programs in India, which are only part of our portfolio. Program costs vary across geographies due to differences in program scale, country systems (as your internal discussion pointed out, compensation of government officials such as per diems is quite different in Kenya compared to India) and costs of inputs in the countries that we work in. Likewise, the programs we support are very different across countries depending on how governments structure their specific agencies and approaches to treatment. Some resources we invest are for program development and scoping, or for fixed costs (e.g., new training material development) not only for supporting treatment rounds.
We have shared detailed information with GiveWell about how we are investing resources. We are also happy to share information about our organizational budget with GiveWell and (see below) information about the costs of deworming treatment, but this calculation you have done is not an accurate reflection of your investment.
5. Relatedly, you say: “In 2015 they’re spending 6 million in India (plus maybe .8m centrally? plus .8m on new countries, which we should probably count in some year). So that’s $7.6m in costs. If there are 30 million treatments that’s an extra 25c/treatment which is almost a 2x change.”
We have provided Givewell with our 2015 budget projections for Evidence Action’s Deworm the World Initiative, which is a total anticipated budget of $10.1 million, $6 million of which is for India. However, the number of treatments cited above is incorrect. We will be supporting treatment for more than 150 million children in India in 2015.
We have a comprehensive blog post about how we calculate the cost of deworming a child per year here: http://www.evidenceaction.org/blog-full/how-do-we-calculate-the-cost-of-deworming, that might be helpful to you and your audience. We outline there exactly what we do and don’t include in our deworming-cost-per-child analyses, and why.
Hi Grace – Thanks for the comment and for this additional context. We didn’t include your work in Kenya in our discussion (and we generally exclude it from our analysis of DtWI, discussed here) because that project is funded by CIFF, and we do not believe that GiveWell donors’ funds are used by or affect your work there.
One thing I didn’t realize is that you expect to support programs that deworm 150 million in India in 2015; that’s great!
I’ll email you separately about ways you can hear about public discussions like this sooner. Happy to see the blog post with the detailed description of the theory behind your cost estimates!
Press release is here: http://www.evidenceaction.org/blog-full/largest-deworming-program-in-india-to-start-with-support-from-evidence-action. Target for the firsts national deworming day in Feb is 120 million children.
Comments are closed.