There are a lot of reasons to be interested in charities focused on neglected tropical diseases (NTDs), and particularly on deworming schoolchildren.
- Deworming schoolchildren, as discussed in a recent post, is ranked among the top programs in terms of “cost-effectiveness,” i.e., health benefits (in DALY terms) per dollar.
- Deworm the World has in the past been featured and, seemingly, implicitly endorsed by Poverty Action Lab,* which is one of the top sources of rigorous research on aid.
- Schistosomiasis Control Initiative and Deworm the World are rated as the top charities of Giving What We Can (a group that promotes both more generous and more intelligent giving).
We are positive on deworming as an intervention. But we feel that for individual donors, it is necessary to form confidence in charities, not just interventions. This is where our interest in NTD control hits major obstacles. This post goes through our questions and the difficulty we’ve had answering them, and focuses on charities focusing partly or fully on deworming.
Key questions
As far as we can tell (details below), the major organizations focused on deworming work heavily with developing-world governments. They spend money on both advocacy and subsidies for government control programs. We therefore have the following questions:
- How do the charity’s funds break down between subsidies and advocacy?
- For subsidies:
- How much of the total spending on control programs has been covered by the charity, as opposed to the government?
- How has the government been audited to ensure compliance with terms and conditions? Note that the Stop Tuberculosis Partnership serves in a similar role to NTD charities, subsidizing (through drug grants) government tuberculosis control programs, and it has an extremely thorough auditing process to ensure that drugs are used appropriately, that reported statistics are meaningful, etc. It has in the past discontinued funding for noncompliance. How do NTD control charities compare?
- Does the charity intend to keep subsidies at the same level indefinitely, or does it intend for the government to take over activities? If the latter, what is its track record in accomplishing this very difficult task? Note that while we consider “sustainability” optional for some programs, we believe it is crucial for deworming, for reasons discussed previously.
- For advocacy activities:
- How much has been spent, what programs (and where) have been advocated, and to what results? What does the future advocacy plan and budget look like, and what might be expected from it? Cost-effectiveness of control programs is one question; the cost-effectiveness of advocacy funds could be another question entirely.
- To the extent that advocacy has succeeded, has it resulted in (a) more government funding for NTD control with no offsetting cuts, i.e., increases in total medical budgets; (b) more government funding for NTD control at the expense of other health spending; (c) more funding for NTD control, contingent on more donor subsidies?
- If (a), how has the increase in funding been financed? If (b), where have cuts been made, and to what extent has funding shifted from other worthy health spending? If (c), what are the answers to the key questions about subsidies (above)?
Schistosomiasis Control Initiative (SCI)
A key roadblock to evaluating SCI is that it does not share any information about its budget by program (past or planned). Last year, after failing to find this information on its website, we contacted Prof. Alan Fenwick, and after significant back-and-forth we were told that no budget information could be shared. This makes it impossible to answer key questions about the role of advocacy vs. subsidies, and where future donations are likely to go.
SCI also declined to answer our questions about impact of past work directly, instead suggesting that we search PubMed. We did so, and have also recently reviewed SCI’s updated website, summarizing what we found in this XLS file. Our conclusions:
- There is serious evidence of the impact of past projects, a credit to SCI and something that is rare among charities in general.
- SCI appears to have provided both funding and technical assistance in the past (and from phone conversations we also understand that it has done advocacy work). Without budget information, we do not know how its funds have broken down between these activities or the specifics of what has been paid for by SCI vs. governments.
- SCI appears to have exited at least 3 countries (of 8 that we have information on), and the extent to which its programming has been sustained – both in terms of finances and quality – by the government is unclear.
- We do not have information on how SCI audits government data and practices (though the direct evidence of impact is encouraging), or on the specifics of its advocacy work.
Deworm The World
We have essentially no information about Deworm the World.
- Its website includes no reports on expenses or on specific projects, past or present.
- It is not clear to us where Deworm the World’s focus lies. It states that it can “link” funders to projects, while linking to a page where people can donate directly to it. It has another link implying that donations directly fund projects on the ground. But Poverty Action Lab’s description* makes it sound like the organization focuses significantly on advocacy and public information as well.
- Last year, after failing to find substantive information on the website, we emailed the organization twice and received no response.
We consider Poverty Action Lab itself to be a model of transparency, posting exhaustive information on studies both completed and in progress. But Deworm the World is on the opposite end of the spectrum, providing no substantive public information as far as we can tell.
We have in the past been tempted to recommend Deworm the World simply on the strength of our respect for Poverty Action Lab. But ultimately, conducting research projects is a different enough challenge from working with governments on large-scale programs that we think that doing so would be the wrong move – both in terms of the incentives it would provide and the good it would accomplish.
*At the time this post was drafted, Poverty Action Lab discussed Deworm the World in various places on its site. However, the links have since broken as Poverty Action Lab redid its website. We have not been able to find any mention of Deworm the World on Poverty Action Lab’s new website.
Comments
I have been eager to hear more about these organizations. Thanks for posting.
I agree that there are a lot of unanswered questions here. Do you have an idea how these organizations would compare if you made various estimates for some of the key unknown variables? For me, it would be extremely useful to know, for example, how these organizations would compare under various levels of optimism for estimates. What could we expect if things went about how you’d guess, went poorly, or went badly at various stages?
Nick, to answer you literally: I think a government that is not sufficiently held accountable would probably have wastage between 20% and 80%. I would put an “average” attempt to have the government take over funding of a program long-term around a 90% probability of failure (some reasoning here; I’m also thinking of Bill Easterly’s work on the history of aid to governments), but if done properly such an attempt could have a failure probability as low as 10%. With no information about the organizations as opposed to the programs, I have no way of knowing where in these ranges to put them, though obviously the default should be closer to “average” than “done properly.” I haven’t looked into advocacy and have no idea what it’s like in terms of cost-effectiveness and fungibility.
Bottom line – I think the cost-effectiveness estimates for what these orgs do are already such guesswork that putting in more uncertainty bands, based on my own wild guesses, does not generate numbers that should be used for decision-making.
To be clear, if all of the above questions were answered to our satisfaction and we concluded that these organizations were as strong as Stop TB, we would consider them to be roughly even with, not significantly stronger than, that organization. Averting tuberculosis is highly cost-effective as well, and we believe the estimates are so rough that differences between deworming and tuberculosis $/DALY should not be taken literally.
Sorry, I don’t understand what you mean by ‘taken literally’ of course there needs to be programmatic transparency when you’re deciding whether the programme is worth funding or not but surely we need to look at DALY/$ and have it as the main thing if we’re concerned about wellbeing. If the programme is falty or has hiccups and risks then you take it into account and can roughly knock down the ratio or maybe disregard it if there are negative externalities but seriously, we should regard it!
If the estimates are rough then we should apply more rigorous empirical assessments.
Tom, what I mean is that when two charities work on different “best in class” interventions (for example, tuberculosis and NTDs) one should decide between them based on other factors, e.g., confidence that what they’re doing is working, rather than based on differences in the theoretical cost-effectiveness estimates for the programs they run.
I am not sure whether putting together better cost-effectiveness estimates would be feasible or worthwhile.
Hi,
It’s good to know that you’re looking into SCI and DtW, and taking
GWWC’s recommendation of them as a reason for considering them
further. I’m also glad there’s some disagreement: we both have the
shared aim of doing the most good we can, and having more dialogue
between us can only help that. I’ll reply to the above post here, and
respond to your scepticism about cost-effectiveness estimates (‘inside
the sausage factory’) on that thread.
Some of your greater scepticism about SCI and DtW might be partly due
to methodological reasons. We tend to assume, in the absence of other
information, that charities are average at implementing their
intervention, whereas you seem to assume that charities are bad at
implementing their intervention, until you have been shown concrete
evidence that they are not bad at implementing their intervention.
This, I think, partially explains why we are happier to focus on
causes rather than charities.
Some of your questions are excellent, and I agree it would be great to
find out as much as we can. In particular, issues about how
governments are audited, about the effectiveness of passing programs
wholly over to the home government etc, are really important to find
out more about.
However, I think the scepticism about SCI and DtW is a little
unwarranted. DtW spends about 74% on technical assistance and scaling
up deworming programs within Kenya and India (I don’t have a figure
for SCI, but the impression I get is that it spend a lower proportion
of its money on advocacy work than DtW). Let’s assume (very
implausibly) that all other money (spent on advocacy etc) is wasted,
and assess the charity solely on that 74%. It still would do very
well (taking DCP2: $3.4/DALY * (1/0.74) = $4.6/DALY – slightly better
than their most optimistic estimate for DOTS (for TB), and far better
than their estimates for insecticide treated nets, condom
distribution, etc). So, though finding out more about their advocacy
work is obviously a great thing to do, the advocacy questions don’t
need to be answered in order to make a recommendation: it seems that
DtW and SCI are worth recommending on the basis of their control
programs alone.
Moreover, it seems that these charities are either as good as, or
outperform the DCP2, rather than underperform. In India, for DtW, for
example, the cost per treatment is only $0.18 (economic costs,
including the value of donated drugs) or $0.12 (financial cost, or
cost to the charity – which I think, though I’m not certain about how
the counterfactuals play out, is the more relevant piece of info, from
a donor’s perspective). This suggests, other things being equal, an
improvement in cost-effectiveness by 2.5x over DCP2 (which uses the
benchmark $0.50 per treatment).
Moreover, the above doesn’t take into account the educational
benefits. The fact that deworming is certainly one of the best health
interventions, and has been endorsed as the best education
intervention, is surely very impressive. And it’s something that DOTS
programs – though these are exceptionally impressive themselves –
lack.
Will, I disagree that you assume that charities are “average at implementing their intervention.” As noted elsewhere, cost-effectiveness estimates are generally based on studies of particularly carefully observed programs that are known to have been carried out in a sustained, competent, successful way. Cost-effectiveness can drop to zero or negative if these conditions are no longer met. So by using cost-effectiveness estimates the way you do, you seem to be implicitly assuming that charities are “as effective as the studied projects,” not that they are “average.” We have too little information to know the relationship between these two things, but they are conceptually extremely different.
I have two issues with your statement about Deworm the World (DtW). The first is that I don’t think it’s valid to multiply 74% by the “estimated midpoint competent-case expected cost-effectiveness of the program” to get overall midpoint expected impact, and that in fact 74%*A can be a worse bet than 100%*B even when A is estimated to be 2*B by best available estimates. I elaborated on this view in our other discussion.
The second issue is that you seem to assume that “technical assistance and scaling up deworming programs” can be considered equivalent in cost-effectiveness to “executing deworming programs.” In fact, we don’t know what impact DtW’s technical assistance has on the scale and effectiveness of government deworming programs. Note that if we analyzed VillageReach similarly to the way you analyze DtW, we would have a very different and more favorable cost-effectiveness estimate for them: in digging into what they do and what it costs, we have found it possible that they are spending several times more per unit (life saved / DALY) than the outright DCP estimate of pure vaccination programs. (Details.) I would guess this is the case for many charities focusing on assistance/modification rather than direct delivery, and few provide direct delivery as this is often considered to be the government’s proper role.
I disagree with your point about educational benefits as well. The fact that educational benefits have been observed for deworming and not DOTS seems to me to be more an artifact of what has been measured that a fundamental fact about the two different programs. For example, if DOTS saves the lives of parents, I would guess this has a positive impact on children’s educational experiences; it just hasn’t been examined. Also note that deworming has not been observed to have any effect on performance in school, only attendance; while I think it is a defensible view that deworming is a better bet to improve educational outcomes than education-focused interventions, I don’t think the evidence is any stronger than for DOTS that its benefits truly extend beyond health.
I’ve been reading a lot on this website over the holiday period, and it has drawn be to the conclusion that tackling NTDs is the best use od my charitable donations over the coming year (I would never have imagined this before I started my research!). My question is just which of SCI and DtW is most deserving. Excellent arguments are made above, I just have one point/question.
DtW has a budget of less than $1m. SCI is much larger and has that backing of the Gates Foundation and the US and UK governments. My donation is tiny compared to these major donors. Does this mean that my donation does more good with the smaller organizaton (DtW)? My rationale being that DtW has less money so my small donation will make a bigger difference with DtW than it does with SCI.
And another thought I had: is there a case for funding DtW simply to get some healthy competition into the fight against NTDs? Surely having two organizations on the issue is better than one, even if the second spends my money less effectively than the front-runner?
I actually just made a donation to SCI, but I’m playing devil’s advocate with myself! Your views are much appreciated!
Comments are closed.