Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.
You can view previous open threads here.
Comments
What did you make of the critique of GiveWell in this article? https://www.wired.com/story/deaths-of-effective-altruism/
As I understand you are already intending to improve your representations and communications of uncertainty. I think the deworming criticism was mostly unfounded but would be happy for your thoughts.
Why do you think it was mostly unfounded?
Hi Oscar,
Thanks for your question! We welcome criticism of GiveWell, both on an ongoing basis and via dedicated red-teaming activities such as our Change Our Mind contest.
Our research incorporates potential downsides or unintended consequences, such that our recommendations take into account factors like potential side effects. Most (if not all) of the information in the WIRED article is something that anyone could find on our website. However, we believe some of what is published in the piece is misleading or inaccurate. Here is a lightly edited version of the material that GiveWell sent to WIRED in response to a request for comment on Leif Wenar’s piece.
Regarding the deworming criticism, the conversation referenced in the article was specifically about whether the Deworm the World Initiative improved the quality of programs, but this is not the only or even the primary way that Deworm the World achieves its impact. We emphasize that the impact of Deworm the World may also be achieved via the increase in likelihood of governments implementing deworming at all, for which there’s additional evidence.
For example, India is the only location that Deworm the World works in where there have been substantial government financial resources available to support deworming. In the parts of Kenya, Pakistan, and Nigeria that Deworm the World works, Deworm the World provides funding for the direct costs of delivering deworming. In these locations, we believe it’s likely that deworming would not happen or would happen much less consistently without external funding.
Additionally, GiveWell has reviewed detailed information and results from Deworm the World’s extensive monitoring and evaluation efforts tracking work leading up to deworming provision and measuring coverage after campaigns. Finally, GiveWell has spoken to government officials in Nigeria, Kenya, and India.
I hope this is helpful.
This article was an important reminder. I felt very silly during the SBF thing because I’d been giving some amount of money to the EA fund every month. I might as well have been burning it. After that, I switched it to go to the GW Top Charities fund, but these guys aren’t infallible either. That story about the de-worming pills making kids sick was quite bad, and I sincerely hope GW looks at methods of accountability when making recommendations in the future. The takeaway is that quantitative information and projections are useful, but we shouldn’t pretend that they’re particularly precise, or that we’ve even come CLOSE to figuring out all the “unknown-unknowns” that would make the proper calculation of a standard deviation possible (remember the apocryphal story about the computer in the basement of the Pentagon telling McNamara they’d already won the Vietnam war?). These numbers should be used internally, but not in marketing (“$xxxx can save a life!”), because no-one except some of the people in the organization will properly understand the assumptions they were calculated under. The danger of group think highlighted by the article is also incredibly real. The DoE deliberately cycles out all of its ARPA-E program managers after 5 year appointments. Perhaps similar methods to deliberately shorten institutional memory could be used at GW, especially in new program evaluation. Bringing in a variety of people, some like Mr. Wenar, might help GW realize where they’ve started to take lazy-ish assumptions for granted.
All we can do is be humble and ready to change our minds!
I am interested in your research on the cost-efficacy of different measures to prevent and/or treat tuberculosis.
This post estimates about $6000 per life saved, for a mass screen-and-treat program in India.
https://www.givewell.org/international/technical/programs/mass-screen-and-treatment-programs-tuberculosis
This estimate differs somewhat from the Copenhagen Consensus report, which gave a figure of $2000 per life saved, for a broad package of activities (including improving diagnosis & treatment, giving preventative medication, etc – not just mass case finding). If they are correct, this would be a GiveWell top charity recommendation.
https://copenhagenconsensus.com/publication/halftime-sdgs-tuberculosis
That report itself draws from the Global Plan to End TB, with a modified model that reduces spending and increases the time period under consideration from 2030 to 2050, substantially increasing the cost-efficacy.
I did not look at their methodology in detail but would appreciate it if you could highlight any major points of disagreement.
Hi Tom,
Thanks for your question! We haven’t reviewed the Copenhagen Consensus report in depth, but the estimated difference seems to reflect how GiveWell approaches our cost-effective analyses differently than other organizations. Two main differences we spotted upon quick review from one of our researchers:
1. Our intervention report makes subjective guesses for how the counterfactual treatment and general healthcare improvements over time could mitigate the impact.
2. We also assume some downward effect due to migration. However, the Copenhagen Consensus report assumes very comprehensive and broad program activities, so this is likely less of a concern.
We are continuing to consider tuberculosis programs, and we expect our intervention report on household contact management of tuberculosis to be forthcoming. Hope this helps clarify!
Thanks Chandler, that’s helpful. Look forward to seeing the upcoming reports.
Hi again,
Just wanted to flag this piece of research in case you hadn’t come across it:
https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(23)00324-8/fulltext
– Bhargava et al gave monthly food rations (worth USD 15 per month, including delivery costs) to 2800 TB patients for their 6-month treatment course, plus up to 6 months more if the patient was still underweight at the end of treatment (as 55% of patients were)
– 4% of patients in the program died (compared to a national mortality figure of around 17%).
At face value, this gives impressive cost-efficacy figures of around $1000 per life saved.
Unlike Shreshta et al’s article, this was not a model but rather a real-world intervention (though not an RCT). So perhaps this intervention – providing nutritious food for TB patients – is also worth some investigation from GiveWell.
Thanks so much for sharing, Tom! I passed this along to our Research team.
What is the best way to contact GiveWell staff about specific pieces of research? For example, I’d like to suggest the intervention report on breastfeeding promotion look separately at early initiation of breastfeeding vs sustaining exclusive breastfeeding, since these may be operationally quite different. Early initiation can be a focus of facility-based activities. But I’m not sure who is the best person to write to about something like this.
Hi Katriel,
Thanks for reaching out! Please send your questions/comments to info@givewell.org, and we will route it to the appropriate research team member. We appreciate you engaging with our research and sharing your thoughts!
Hi, it seems like I can only donate indirectly through GiveWell grants and not to the charities directly. Why do you not enable donors to give directly to the organizations you’ve deemed as effective? Why does GiveWell determine the scope and goals of the grants themselves? Do you think it’s more effective for these organizations to use direct donations as they see fit on the ground rather than through specific grants approved by GiveWell? Have you done any research to comparing the effectiveness of direct donations vs Givewell mediated funding streams? Thanks, Jason
Hi Jason,
Thanks for your questions! We do have the option to donate directly to our top charities (see the dropdown menu on our donate page). You can read more about donating directly to our top charities here. We grant funds from the Top Charities Fund on a quarterly basis based on our research team’s assessment of which of our top charities have the highest priority funding needs at that time. We consider factors such as:
– Which funding gaps we expect to be filled and unfilled
– Each charity’s plans for additional funding
– The cost-effectiveness of each funding gap
This helps us ensure that the grant opportunity meets our cost-effectiveness threshold (which is currently 10x cash). Our Top Charity Fund is the best option for supporting the opportunities we think have the highest impact across our top charities at any given time, but we also think donating directly to a top charity is valuable! If you decide to donate directly to a GiveWell top charity, it’s helpful to report it so we can track our impact. Hope this helps!
I was very interested to see your new page on potentially saving the same lives each year (https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/repetitive-saving_. Now that GiveWell is increasingly exploring acute malnutrition, do you think the same broad conclusions apply to CMAM? I can imagine that there would be less year-to-year variation of which children are at the highest risk of starving than of contracting malaria, because they are the ones in very poor families who can’t afford enough food.
Hi Katriel,
Thanks for your question! As part of this project, we considered how this would play out for the community-based treatment of acute malnutrition (CMAM) programs we’ve supported. We didn’t look into this in as much detail as malaria, vaccine-preventable disease, or diarrhea. However, based on a shallow review, we concluded that this doesn’t seem to be a larger issue for CMAM programs than for others. This is because we think there’s a relatively narrow age window for susceptibility to malnutrition.
The CMAM programs we’ve supported reach children from 6-59 months old. We looked at data from Nigeria and Mali, where an NGO had shared detailed age breakdowns of children who were admitted to its program with acute malnutrition. Roughly three-quarters of children with severe or moderate acute malnutrition were under two and ~5% were over three. This implies that most acute malnutrition cases occurred over a one-and-a-half-year period. If children are at much lower risk once they are over two years old, that implies they are less likely to develop severe acute malnutrition (SAM) and moderate acute malnutrition (MAM) repeatedly.
Comments are closed.