Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.
You can view previous open threads here.
Comments
Where are you up to implementing suggestions from the Change Our Minds contest winners or other submissions?
What are you main misgivings about the Happier Lives Institute’s approach to charity evaluations, and/or in the absence of major concerns, are you planning to redo your CEAs with WELLBYs as the comparative unit?
(FYI I am not affiliated with HLI, I just admire their work and think it should be more central to EA grantmaking.)
Hi, Oscar,
Thanks for your questions!
Re: the Change Our Mind Contest, we have concrete plans to address the topics of the two first-place winners, “An Examination of GiveWell’s Water Quality Intervention Cost-Effectiveness Analysis” and “GiveWell’s Uncertainty Problem.” These will likely take the form of some updates to our cost-effectiveness analysis for in-line chlorination and Dispensers for Safe Water, and a write-up on how we plan to approach uncertainty going forward. We don’t yet have a publication date set for these updates.
For the entries we recognized with an honorable mention (see the blog announcement for descriptions), we’ve added the research questions raised by these critiques to our queue. We’re currently considering how to prioritize these alongside other research questions that come up in the course of our grant investigations, so there’s no clear timeline for when those updates will be incorporated.
Re: the Happier Lives Institute’s recommendation of WELLBYs and its corresponding recommendation of StrongMinds (more here for anyone reading this who isn’t familiar with their work), we are working on a report summarizing our review of the evidence behind interpersonal psychotherapy group (IPT-G), the intervention carried out by StrongMinds. That report, which should be published soon, will be the best source of information for our views on this subject. We’ll post another comment here to let you know when that’s published (and feel free to sign up here if you’d like to be notified of future research reports).
I hope that’s helpful!
Best,
Miranda
Hi, Oscar,
Following up to share that we’ve just published our assessment of HLI’s analysis of StrongMinds here.
Thanks for your interest!
Miranda
Hello communications expert,
I browsed the cogent summary of cost-effectiveness of syphilis screening. After browsing the section, I interpreted that this particular syphilis screening might equal top-rated charities. That interpretation reassured my confidence in diligent GiveWell accounting.
But one phrase was:
Only the above phrase distracted from the assurance. The article transparently advises in many cases to disregard the precision of the number 29. The precision of the number 29 distracted a train of thought. The number 28 or 30 is less than 5% different than 29. A misinterpretation would be a margin of error less than a 5%. Carefully reading the section advises the reader to not misinterpret the usage of 29 as meaning greater than 28 and less than 30. A couple questions come to mind:
1. GiveWell published 29 instead of 28 or 30. Has GiveWell also published something like a margin of error or confidence interval around the number 29? Would something like a confidence interval clear up the misinterpretation of confidence in that the number was not 28 or 30?
2. If such a margin or interval is out-of-scope of an analysis, has GiveWell considered future articles to also regard the point-estimate like 29 to also be out-of-scope for an uncertain model compounded by uncertain evidence? In case of uncertainty, rather than the number 29, would a GiveWell reader feel clearer to read a vague summary of optimism?
Ethan
Hi, Ethan,
Thanks for your careful engagement with our work! Currently, we don’t have a margin of error or confidence interval to express our uncertainty about the 29x cash estimate for the grant you mention, or any of our other cost-effectiveness estimates. We also don’t currently have a plan to stop referring to these point estimates of cost-effectiveness in our public write-ups.
We have historically tended to factor uncertainty into our analyses “up front,” by applying adjustments for things like external validity (the applicability of study findings to real-world settings) or replicability (the likelihood of another study finding a result similar to the study results we’re drawing on). We try to think through and adjust for as many such variables as we feasibly can in our public cost-effectiveness analyses, so that the resulting number represents our best guess of cost-effectiveness, inclusive of our uncertainties. (And, as you noted, we heavily caveat the final number when we publish it.) It’s also important to note that we use these cost-effectiveness estimates mostly for comparative purposes, to decide what programs we should prioritize funding—not to serve as an absolute judgment of their impact.
All that said, we have gotten a fair amount of feedback recently suggesting that we adopt a more systematic way of expressing uncertainty in our analyses. In fact, one of the top two winning entries in our Change Our Mind Contest was a critique of our approach to uncertainty. We’re currently exploring how we might update our approach based on this and similar critiques.
I hope this is helpful!
Best,
Miranda
First of all, thank you for all the work that you do in analyzing charities!
In your analyses of Helen Keller International and Malaria Consortium, you analyze the most effective program in each of the charities (vitamin A supplementation and seasonal chemoprevention), and you base your recommendations based on these single programs. On your donations page (https://www.givewell.org/donate/more-information#supportcharities), you recommend earmarking the funds to these specific programs rather than giving unrestricted donations to the organizations.
However, there is a GiveWell blog post (https://blog.givewell.org/2009/12/16/room-for-more-funding-continued-why-donation-restricting-isnt-the-easy-answer) that advises against restricting donations. In particular, the post says this increases administrative burdens on charities and generally does not actually change the allocation, since money can be funged around internally. The post is old, but the logic still seems convincing to me.
This raises two related questions in my mind:
1) Why does GiveWell now recommend restricted donations in contrast to the older recommendation against them?
2) If funging negates the effects of any earmarks, is a donation to HKI (earmarked or not) less effective than one to Against Malaria Foundation, where the charity has a single program and thus funging is not an issue?
Thank you for any insights or recommendations you can provide here!
Hi, Anthony,
Thanks for your support of our work, and for this question! Apologies for the delay in responding.
We think our current top charity recommendations are not very susceptible to this funging risk, because we do a lot of research to ensure they meet the two conditions laid out in Holden’s post: they can productively use more funding, and unrestricted funds are not allocated to them. We conduct extensive room for more funding analyses on these programs, which include their funding needs, anticipated funding from sources outside of GiveWell, and how much unrestricted funding we expect them to receive from parent organizations. For example, in our room for more funding analysis for Helen Keller’s vitamin A supplementation (VAS) program, we estimate that Helen Keller allocates $0 in unrestricted funding to VAS (see here, including the cell note in C22).
In addition, our cost-effectiveness analyses include adjustments for “within-org fungibility,” which is the risk that if GiveWell directs funding to one of these programs, the organization might spend less time fundraising for that program than they would have otherwise. So this risk is to some extent baked into our overall cost-effectiveness estimate for charities we recommend that work on multiple programs. (In the case of Against Malaria Foundation and New Incentives, we put the risk at 0%.)
I hope that’s helpful!
Best,
Miranda
I’ve just learned about a charity called Family Empowerment Media which runs contraception education campaigns in Nigeria. During their initial rollout, Kano state (the area in which FEM’s radio programs were aired) saw a 75% increase in the use of contraceptives. FEM stated that, even if their messaging were only responsible for 15% of that increase (a very conservative estimate), the cost to save a life through reduce maternal mortality was around $2200 USD. I would love to know if GiveWell has evaluated Family Empowerment Media before, and if so, what concerns it has considering the very low cost to save lives.
Hi, Charlie,
Thanks so much for your engagement! We are aware of Family Empowerment Media’s work and have had several conversations with them recently. We plan to continue investigating this program.
Best,
Miranda
Thank you for your great work! While I believe it is far from a primary issue for GiveWell charities, I came to your website looking for a description of how you vet charities regarding internal corruption and general malpractice.
I found some related considerations on this page,
https://www.givewell.org/charities/top-charities/2020/qualitative-assessments, but would be interested to hear if you had a page you would rather direct me toward to describe how GiveWell systematically approaches such issues.
Hi, Henri,
Thanks for your inquiry! Corruption and fraud are always potential risks, and we take them seriously when investigating programs. We don’t have a dedicated page describing how we vet programs for evidence of malpractice. However, I can outline some of the factors that give us confidence that the funds we direct are used for their intended purpose:
I hope that’s helpful!
Best,
Miranda
Dear Miranda,
Thank you for your response and for pointing me to these detailed spreadsheets.
This is surprising to me that Helen Keller allocates no unrestricted funding toward vitamin A supplementation given that you consider it to be their most effective program (and, in fact, one of the most effective programs run by any charity). Do you know why they don’t allocate any unrestricted funding toward VAS? Do they have different estimates of the relative efficacies of their various programs? I know that these are questions about HKI’s internal decision-making processes, which you may or may not be privy to, so I understand if you don’t know the answers to these questions, but if GiveWell and HKI have discussed this I would be interested in hearing their reasoning.
Thank you,
Anthony
Hi, Anthony,
Based on our review of Helen Keller’s financials and our conversations with them over the years, our understanding is that the organization just has relatively little unrestricted funding in general. See this spreadsheet, which is linked in this cell note in our analysis of room for more funding for the VAS program. This data on Helen Keller’s unrestricted and restricted spending is old, but we haven’t learned anything from Helen Keller that leads us to believe the situation has changed.
Best,
Miranda
Miranda,
I interpreted the syphilis screening comparison to cash tranfers pessimistically estimated uncertainty. Pessimism about uncertainty reassured me. I look forward to any of those steps toward normalizing the pessimism.
Ethan
Hi there,
I just wanted to use this article to thank you for your work. I am trying to spread the word about effective altruism and GiveWell in the French-speaking community of health professionals. In particular through my article on humanitarian missions for physiotherapists, which receives about 50 visitors per day 🙂
Keep up the good work!
Best regards,
Nelly
Hi, Nelly,
Thanks so much for your support and for spreading the word! Please let me know if you have any questions we can answer for you.
Best,
Miranda
Have you considered funding the effort against the clot shots? 13-20 million fatalities according to estimates. Very high rate of severe injuries. This cause is extremely large scale and extremely neglected, although I don’t know how tractable it is.
I am a bit disappointed with the latest grant allocations. It seems you are abandoning an evidence based approach. By cutting funding for Against Malaria’s bed nets which are shown to reduce all cause mortality and putting money into seasonal malaria chemoprevention that has no quality data to support a reduction in all cause mortality you are disregarding evidence and going with a failed intervention As you state “The trials were insufficiently powered to detect effects on all-cause mortality. “
Hi, Luke,
Thank you for your comment and your close attention to our work. Apologies for the delay in getting back to you!
We believe that Against Malaria Foundation and Malaria Consortium’s SMC program are both outstanding—both are GiveWell top charities—and we direct substantial funding to both programs. We evaluate the funding gaps that both organizations bring to us and aim to fill all of those that clear our cost-effectiveness bar (currently 10x cash). If we find gaps that are sufficiently cost-effective within both programs, we’ll fund both.
There are differences year to year in how much funding we provide for nets versus SMC, but this doesn’t reflect a shift in our thinking about the relative worth of these programs. In 2022, we directed a lot more funding to SMC than to nets; in 2021 the opposite was true (we granted funding to Malaria Consortium for net distribution campaigns as well as to AMF that year). One recent factor in the timing of our malaria control grantmaking was the 2022 Global Fund replenishment—we wanted to see how much funding the Global Fund would raise so we could better understand the funding outlook for nets before we committed to fill non-time-sensitive requests.
Re: the evidence for SMC, it is true that the randomized controlled trials that form the evidence based weren’t powered to detect effects on all-cause mortality. However, our assumption is that malaria mortality declines proportionally with malaria illness. We’ve explained this assumption in further detail here.
I hope that’s helpful!
Best,
Miranda
Hi Maranda and thanks for your response I appreciate it. The trouble with your rational is that when we find a reduction in malaria cases for which the data does seem robust but find no discernable reductions in all cause mortality we need to consider the adverse effects of the medication more closely which it seems may not have been done in you analysis. As you state “We lightly reviewed two large studies that tracked adverse events in recent SMC distributions in sub-Saharan Africa. We did not deeply vet these studies… We did a back-of-the-envelope calculation” that does not sound reassuring. It seems the CDC no longer recommends sulfadoxine / pyrimethamine for travelers due to the safety profile “The United States Centers for Disease Control and Prevention (CDC) does not recommend using this medication to prevent malaria (prophylaxis while traveling or living in areas where malaria is common) because severe (sometimes fatal) reactions are more common when this medication is used for prophylaxis.” See the link for that info and some more info on the potential adverse reactions to the drug. https://www.medicinenet.com/sulfadoxine_with_pyrimethamine-oral/article.htm
Hi, Luke,
We’ve published a bit more about our research into serious adverse events from SMC in this document, which is linked from our Malaria Consortium review. From our review of the evidence, the rate of such events is low enough that it doesn’t materially affect the benefit from the intervention.
We also feel reassured that serious side effects from SP are not a significant cause for concern, given that the World Health Organization recommends SP or SPAQ for chemoprevention activities, which means that the WHO has found these drugs to be safe and effective.
We do factor a slight downward adjustment into our cost-effectiveness model to account for this risk (and account for other possible downsides of the program with other supplemental adjustments).
Thanks for raising these concerns!
Best,
Miranda
Subject: Request for Grant to Support a Social Entrepreneurship Project in Homabay County, Kenya
Dear Any Well-Wisher,
I hope this letter finds you in good health and high spirits. I am writing to you today with great enthusiasm and a vision to create a positive change in Homabay County, Kenya. My name is Opiyo Ware, and I am reaching out to you as a well-wisher who has shown dedication and support for initiatives aimed at uplifting communities.
I firmly believe that social entrepreneurship has the power to transform societies and improve the lives of individuals who are facing various challenges. With this conviction, I have set out to establish a social entrepreneurship project in Homabay County. The objective of this initiative is to address critical social issues while promoting sustainable development, economic empowerment, and social inclusion.
Homabay County, located in western Kenya, faces several socio-economic challenges, including high unemployment rates, limited access to quality education and healthcare, and inadequate infrastructure. Through this project, I aim to create opportunities for local communities, empower marginalized groups, and foster long-term solutions to these pressing issues.
The proposed social entrepreneurship project will focus on three key areas:
1. Skill Development and Job Creation: We will provide vocational training programs to equip individuals, especially youth and women, with essential skills that align with market demands. By doing so, we will enhance their employability and enable them to become self-reliant, contributing members of society.
2. Accessible Healthcare and Education: Our project aims to establish healthcare and education centers that are easily accessible to underserved communities. Through partnerships with local organizations and professionals, we will ensure that quality healthcare services and education are made available to those who need it the most.
3. Sustainable Agriculture and Environmental Conservation: Agriculture is a vital sector in Homabay County. We plan to implement sustainable farming practices and provide farmers with the necessary resources and knowledge to enhance their productivity and income. Additionally, we will promote environmental conservation efforts to preserve the county’s natural resources for future generations.
To turn this vision into reality, I humbly request your support in the form of a grant. The funds will be utilized to establish the necessary infrastructure, implement training programs, provide equipment and resources, and ensure the smooth operation of the project. Your generous contribution will make a significant impact and empower the local community to uplift themselves and create a brighter future.
I am more than willing to provide detailed project plans, financial projections, and any additional information you may require to evaluate the feasibility and potential impact of this social entrepreneurship initiative. Your trust, guidance, and financial support will go a long way in bringing about positive change and fostering sustainable development in Homabay County.
Thank you for considering my request. I genuinely appreciate your time and attention to this matter. Together, we can make a difference and create a lasting impact on the lives of individuals and the community at large.
Looking forward to your positive response and the possibility of working together to create positive change.
Warm regards,
Opiyo Ware
+254718227127
opiyojack@ymail.com
Hi Miranda and thanks for the response. Unfortunately I have no faith in the WHO and their recommendations have been disastrous in the last few years. Other organizations have found serious adverse events to be far more common with sulfadoxine / pyrimethamine than your sources. Here is some data “Severe cutaneous reactions have occurred in patients who have taken as few as two doses of pyrimethamine-sulfadoxine. The risk of fatal cutaneous reactions in patients receiving pyrimethamine-sulfadoxine has been estimated to range from 1 in 11,000 to 1 in 25,000. In general, the use of sulfonamide antibiotics is associated with large increases in the risk of Stevens-Johnson syndrome or toxic epidermal necrolysis, although these phenomena are rare as a whole.” https://www.drugs.com/sfx/pyrimethamine-sulfadoxine-side-effects.html#refs
Also note that the drug is note recommended for women who are pregnant or plan to become pregnant “There are no adequate and well-controlled studies in pregnant women. However, due to the teratogenic effect shown in animals and because pyrimethamine plus sulfadoxine may interfere with folic acid metabolism, Fansidar therapy should be used during pregnancy only if the potential benefit justifies the potential risk to the fetus. Women of childbearing potential who are traveling to areas where malaria is endemic should be warned against becoming pregnant, and should be advised to practice contraception during prophylaxis with Fansidar and for three months after the last dose.” https://www.drugs.com/pro/fansidar.html
As stated before there is no data to support a reduction in all cause mortality with the use of pyrimethamine / sulfadoxine and it would seem to be hard to recommend the treatment to donors as a life saving charitable donation unlike bed nets for which the data supporting a reduction in all cause mortality is robust. Thanks again.
Hi Miranda as a side note I am not sure if you are aware of just how many people die from prescription drugs world wide. I hope you have the time to read the paper linked to that states “Our prescription drugs are the third leading cause of death after heart disease and cancer in the United States and Europe. Around half of those who die have taken their drugs correctly; the other half die because of errors, such as too high a dose or use of a drug despite contraindications…Major reasons for the many drug deaths are impotent drug regulation, widespread crime that includes corruption of the scientific evidence about drugs and bribery of doctors, and lies in drug marketing,” https://pubmed.ncbi.nlm.nih.gov/25355584/
Comments are closed.