Added October 2022: From 2020 to 2022 we used the name “Maximum Impact Fund” to refer to the fund used to support the highest-priority funding needs among our top charities each quarter. In September 2022, we changed the name of this fund to the “Top Charities Fund” to better describe what opportunities this fund supports; more information here.
Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.
You can view previous open threads here.
Comments
I cannot edit my comment, but feel free to delete it. Hopefully this will be less mangled.
In the March open thread there were a couple of questions about the new malaria vaccine. You said: We’re looking at the results and are in touch with malaria experts to discuss the implications of malaria vaccination for the programs we recommend… and whether there are or might be vaccine-related funding opportunities we should consider supporting. If this work updates our view on what to recommend, we will make our new recommendations public.
Have you found any such opportunities? It would be great to know if funding could bring the first vaccinations forward at all. For example we could fund larger/quicker trials, or fund vaccine production in advance of the phase III trials being complete.
Thanks!
Hi George,
Thanks for your question! (I deleted your previous comment, per your note.)
We are exploring whether there are funding opportunities to help one of the malaria vaccines scale more quickly. However, we do not have funding recommendations at this time, as we are still in the process of investigating.
In light of the recent Rethink Priorities report on charter cities. Can you define clear milestones under which you would consider funding charter cities? Some ideas; 1) endorsement by a head of state or former head of state, 2) charter cities legislation passing in a second country, 3) identification of 10 possible locations for charter cities, 4) estimate of charter city cost effectiveness, including spillover effects, that places them comparable or above cash transfers. I am confident that we can meet whatever milestones you define, but they could help us allocate our resources better in the mean time.
Hi Mark,
Thanks for your question!
We wrote in 2019 that charter cities were among the potential ‘high-leverage’ programs that GiveWell might consider in the future as we expand the scope of our research. High-leverage program research is a newer area of our work, relative to our traditional research to identify top charities.
Within our high-leverage work, we continually prioritize among different areas we could investigate. It’s not feasible for us to define clear milestones about if or when we would consider funding charter cities, as our prioritization depends on the full set of programs we’re considering and how they compare to one another.
Right now, our high-leverage work is focused on regulatory interventions to improve public health. As a result, we don’t have plans to investigate charter cities more deeply in the near term. Given our current focus, we suggest that you do not allocate resources towards seeking funding for charter cities from GiveWell at this time.
The education of children, especially girls, has been linked to multiple desirable outcomes, e.g. increased income and productivity, improvements in child nutrition, decreased child marriage rates, and reduced child mortality. A recent systematic review in The Lancet of 186 studies showed that increased maternal and paternal education had a dose-response relationship of reduced under-5 child mortality, with maternal education being the strongest predictor. For girls, each additional year of schooling was associated with a reduction in under-5 mortality of 3%. (Balaj , M,York HW, Sripada K et,al. Parental education and inequalities in child mortality: a global systematic review and meta-analysis. Lancet 2021 Jun 10;S0140-6736(21)00534-1. doi: 10.1016/S0140-6736(21)00534-1.
https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(21)00534-1/fulltext?rss=yes)
An important limitation of the studies included in the systematic review are the fact that they observational (not RCTs), as are most other studies that link increased education of girls with other outcomes of interest. It would likely be very difficult to perform a RCT of an intervention that improves maternal education with child mortality as the outcome. My understanding though is that GiveWell is interested in considering non-RCT level evidence of program effectiveness and cost-effective analyses. Have you considered looking into programs or organizations that effectively increase schooling for children (especially girls)?
Hi Angelo,
Thanks for sharing the Lancet review! I’ve passed it on to the researchers managing our research pipeline.
Based on our previous work on education programs, we are unlikely to prioritize research on education programs in the near future, though we may revisit them on a longer time horizon. The evidence connecting education to key outcomes appears weaker than that of other programs we’re prioritizing, even as you correctly note that we are assessing a broader range of evidence, including non-randomized controlled trial (RCT) evidence, in our research work today.
Hi Catherine,
In reading some of Givewell’s evaluations, I have come to better appreciate the complexities involved with estimating the effectiveness of charitable donations. There may be a study that provides a baseline estimate. But then there are other considerations that cause Givewell to reduce and/or increase this initial estimate of effectiveness. So it seems that Givewell’s final estimate is a product of several multiplicative factors, all obtained with some level of uncertainty.
In considering this situation, it seems to me that a probabilistic model might provide donors with a better sense of the uncertainty associated with the effectiveness of different organizations. For example, Givewell might express a “best guess” that a particular factor reduces effectiveness by 15 percent. But maybe Givewell would be willing to go further and state that the reduction in effectiveness is likely no more than 25 percent and no less than 10 percent. A statistical distribution, such as the Beta distribution, could be used to approximate this uncertainty, perhaps with a mode near 15 percent and 5th and 95th percentiles near 10 percent and 25 percent. Without going too much into the details here, it seems that one could obtain a random draw from each of these distributions, combine these values in a model, and produce an estimate of effectiveness. With numerous iterations of these random draws, Givewell could produce an uncertainty distribution regarding effectiveness, and perhaps report the 10th and 90th percentiles of this distribution as lower and upper bounds on estimated effectiveness.
The above method would work particularly well in conjunction with a Bayesian analysis of any data from randomized trials that Givewell uses to form estimates.
This is a long comment, and I’m sure that Givewell analysts have excellent reasons for the existing evaluation methods. No doubt, developing distributions would be even harder than the already-difficult task of establishing point estimates. Nonetheless, I can see some solid benefits with this type of analysis. Would you mind providing Givewell’s viewpoint on this type of evaluation?
Hi Kendal,
Thanks for your question! We have discussed why we don’t use confidence intervals, the type of modeling you describe, in this previous comment on our blog:
We have considered but ultimately decided against including confidence intervals with our cost-effectiveness estimates, for a few reasons:
* We’re risk neutral; we’re glad to take on risk when the expected value of a donation is high. It’s not clear how confidence intervals would update us.
* Most of the uncertainty we have about our cost-effectiveness analyses comes from uncertainty about whether we’re modeling things in the right way and including and excluding the right factors. Using confidence intervals wouldn’t reflect our greatest source of uncertainty, which isn’t the individual parameters—it’s what we’re including and excluding in our model.
* Confidence intervals might be useful if we decided to regress our models to the mean—to apply an additional discount to wider confidence intervals, even if the programs have the same expected value. However, we already account for this type of uncertainty in our model by applying a replicability adjustment. This replicability adjustment incorporates our uncertainty about whether the effects seen in the studies we rely on are also found on the ground today.
So, while we think a confidence interval would have some value in communicating our uncertainty, it’s unlikely to substantially change our decisions. We thus haven’t prioritized adding such intervals, given the large time cost, to date. We may do so in future.
We published a 2017 blog post looking at the uncertainty in our cost-effectiveness analysis, which may also be of interest.
Hi Catherine,
Thank you for your reply. The post on cost effectiveness analysis was very informative.
I understand that there are difficulties involved with a Monte Carlo or Bayesian approach to quantifying uncertainty. But for what its worth (maybe I have too much time on my hands), I will make a case for this effort here.
A quick sidebar before I start: It seems there are two types of uncertainty applicable to cost effectiveness: a subjective uncertainty (e.g., appropriate discount rate and relative valuations of income doubling and saving a life) and an objective uncertainty (e.g., the expected relevance of study results from one part of the world to another part of the world). I imagine most investors would trust Givewell to develop reasonable uncertainty distributions for the latter, but they might have their own preference for the former. One donor might feel a life saved is worth 10 years of doubled income, while another might equate it to 1000 years. Would you see value in allowing donors to input their preferences in an app, with personalized recommendations based on those preferences?
Another donor-specific consideration is risk preference. Consider a hypothetical: one can (A) donate $5k to Organization A with a guaranteed outcome of saving one life, or (B) donate $5k to Organization B with a 50% probability of saving no lives and a 50% probability of saving 5 lives. Based on your description of its risk neutrality, Givewell would prefer Organization B. But I believe many donors would prefer A. Individual donor preferences could be revealed and modeled based on answers to simple questions.
Given the risk averseness of some donors, a probability distribution on an organization’s effectiveness could be very helpful (actually, a joint distribution for all organizations, which would account for correlation in estimates of deworming effectiveness, for example). The more risk averse the donor, the more valuable this uncertainty quantification would be. Risk tolerant donors might be willing to allocate all their funding to the single organization with the highest expected value of effectiveness. Risk averse donors, in contrast, would likely donate to multiple organizations, with allocations dependent on the expected value and uncertainty of effectiveness associated with each organization.
I understand the issues with model specification. If we’re not certain about the adequacy of a specific model, we are likely to understate uncertainty when we employ that model. That said, a benefit of a Bayesian or Monte Carlo approach is that you can specify multiple models, if desired, and average them in a way that produces credible intervals that account both for parameter uncertainty and model specification uncertainty. Even if you don’t choose to employ multiple models, the estimated uncertainty will probably still be valid for the model chosen, and that model’s specification is available to anyone who might wish to review it. Ultimately (and unfortunately), both the point estimate and the associated credible interval are going to be based on one or more specific (and probably “wrong” but hopefully adequate) models, and I think donors will understand that.
Based on some basic models (with all assumptions stated), I think it would be great if donors could obtain a personalized recommendation based on their values and risk preference. And that’s certainly something that could be done with modeling and an app. Of course, I can’t speak to Givewell’s available resources or whether there would be sufficient donor interest for such an endeavor. And I know the complexities would be very significant. Just wanted to share some thoughts with you and your team.
Regardless of this specific issue, I’m thrilled with the work Givewell does and the careful analysis that accompanies all Givewell recommendations. I’m very impressed with all of the transparency and donor outreach, and I plan to continue learning more. Thank you!
Hi Kendal,
Thanks for your response! People can indeed input their own values into our model and see how that changes the results. Click “Make editable copy” here. For the specific value-related inputs you reference, you’ll want to go to the “Moral weights and discount rate” tab.
We appreciate you sharing your preference and suggestion for how we can align our model with donors’ risk preferences. We agree that this would be a useful exercise, but we expect the cost of creating and maintaining this extra layer of complexity in our model could be quite large.
Although our model includes many inputs, we aim to minimize the complexity of the tools we use to reduce the likelihood of errors and make our work easier to vet and maintain. In our guide to GiveWell cost-effectiveness analyses (CEAs), we write: “We want CEAs to be accurate, easy to interpret, and easy to maintain. Often these attributes of the CEA are in conflict with one another. Complex changes may improve the precision of our estimates but make the CEA harder to understand and update. Making simplifying assumptions may be worthwhile when a small drop in precision can be exchanged for a substantial reduction in complexity.”
One place you can see some of these tradeoffs is our “Adjustments for excluded effects” tab, in which we discuss some of the items we choose not to include in order to keep our core model tractable.
In 2020, I read research on global health and education charities for at least 200 hours. GiveWell has been the most meticulous source. And so GiveWell was my primary platform for donations.
How would you suggest that a GiveWell donor passively promote GiveWell to acquaintances? Do you have any stats on social techniques or channels that converted GiveWell donors? Or any quasi-experiments on social techniques that raised donations for any charity all. Maybe one of those techniques is applicable to passively recruiting GiveWell donors.
For example, is there a Facebook photo frame for GiveWell? I did not find one by searching for givewell in the Facebook photo frames. If there is no Facebook frame, is there a photo frame image that would be suitable for me to manually merge into my profile photo.
Is using a GiveWell logo for a Facebook, LinkedIn, or GitHub profile photo acceptable? Of course, the employment of the personal profile is clearly listed as a different company and clearly not affiliated with GiveWell staff or a representation on the opinions of GiveWell staff.
And, I would not expect the above techniques to be trackable for conversions.
I look forward to listening. In the meantime, I will continue helping my friends and acquaintances. And if they ask about my hobbies, and there is something relevant about GiveWell that I could mention (such as the recent webinar), I will mention it.
The Lancet meta-analysis that Angelo posted feels like an insightful step toward an education intervention being recognized as most cost-effective.
The spreadsheets in the GiveWell 2018 estimate of education cost-effectiveness demonstrate a technique for comparing interventions. I recognize the GiveWell 2018 consensus was that an extremely poor family needs one more year of medicine more urgently than they need one more year of school.
What would be a rough estimate of the cost of a life saved for a marginal year of schooling?
For example, below is a naive attempt to estimate:
* Between 1.5% to 3% reduction in mortality per marginal year of school enrollment (as cited in the article above).
* 50% multiplier for replicability of a non-RCT. (Just a guess). Suppose, for simplicity, a 1% reduction in child mortality after this bias is accounted for.
* 8% child mortality rate in Sub-Saharan Africa (baseline in a World Bank chart for 2019).
* One child saved per 1000 potential child deaths (rounded for simplicity).
* Roughly 1000 marginal school years might save one child.
* 100 USD to educate one child for one year (Just a rounded guess based on vague recollection of costs mentioned by Pratham, Educate Girls, and a Camfed FAQ).
Naively estimated: 100,000 USD might save one child in the next generation.
Next-generation child-survival is only one of many benefits. Other benefits that were noted in one experiment that GiveWell looked at were also income and family planning for the educated student.
I would learn from listening on how GiveWell roughly estimates the impact from an article like the Lancet article above.
It seems like you’ve removed the link to the “Standout Charities” page from the navigation bar. The page still exists at:
http://www.givewell.org/charities/other-charities
But I can’t find a way to navigate to it through links.
Is GiveWell trying to reduce emphasis on these Standout Charities? I found this page useful for cross-referencing with other charity evaluators.
Hi Ethan,
Thanks so much for your interest in sharing our work—we really appreciate it! Personal referrals have been a very important way for new donors to find us.
You might find the following resource useful: https://www.givewell.org/how-to-help/share. We would guess that sharing resources like this and having conversations are the most effective ways to promote our work. Another potentially effective way to use social media might be to write a post about why you use GiveWell’s research.
If you’re interested in using our logo, please follow up with us directly at info@givewell.org. Thanks again for your interest in promoting our work!
Hi Ethan,
We’re not prioritizing research on education interventions right now, but if we decide to do more work, we would consider incorporating these additional effects beyond income.
If you’d like to learn more about how we approach cost-effectiveness in general, please see here.
Hi Henry,
Thanks for your question! You’re right that the “standout charities” page is no longer linked from our navigation bar; it is linked from our research page. We recently updated our navigation with the goal of highlighting the pages we believe are most useful for donors and potential donors.
As we write on the standout charities page, standout charities support programs that may be extremely cost-effective and are evidence-backed, but we do not feel as confident in the impact of these organizations as we do in our top charities. Directing funding to our top charities, which are thoroughly-vetted, cost-effective, and underfunded, is a higher priority, and our navigation bar reflects that.
We’ve also added a top-level navigation link to the Maximum Impact Fund, which is our number-one recommendation for donors, as we grant donations to the Fund quarterly to the top charity or top charities where we believe they’ll do the most good.
Thanks for letting us know how you use this page! If you’re interested in which programs we’re prioritizing further work on, you might also be interested in our Research on Programs page.
Hi, I have a couple questions.
1. I’d like to make my donation available to support either GiveWell’s incubation grants or top charities or standout charities depending on what GiveWell thinks is the best use of the money. GiveWell’s donation form doesn’t list anything like that as an option. Is donating through EA Funds the best way to make a more flexible donation?
2. Have you looked into moving money to COVAX or any other organization working on Covid vaccination?
Castillo et. al. (https://science.sciencemag.org/content/371/6534/1107) estimate that the value of a four-month speed-up in a course of Covid immunizations is $576 to $989, at a production cost of $6 to $40. That sounds like it has the potential for a great ROI, but I’m not sure whether more funding would really help speed up production, whether supply is the most important factor constraining vaccination rates, or whether the organizations working in this space are doing a good job.
Hi Elizabeth,
Thanks for your questions and for your interest in supporting our recommendations!
(1) Unrestricted support for GiveWell: The most flexible way to support GiveWell is to make an unrestricted donation.
We generally use unrestricted funding for our operating expenses. However, we have an “excess assets” policy, which provides that once we surpass a certain level of unrestricted assets, we will earmark the excess for granting rather than continue to hold it ourselves. You can read more about our excess assets policy here. We cap at 20 percent the proportion of our operating budget that any one individual or organization can contribute. This helps us avoid overly relying on a single source of support.
Maximum Impact Fund: If you’d like to make a flexible donation to GiveWell that will only support our top charities (and not our operations), we suggest giving to the Maximum Impact Fund, which we allocate quarterly to the recommended charity (or charities) where we believe donations will do the most good.
The Maximum Impact Fund is our number-one recommendation for GiveWell donors who want to support our top charities.
Additional information on Incubation Grants: We don’t currently offer an opportunity to fund Incubation Grants directly through GiveWell, though it is possible that unrestricted donations may be used to support such opportunities.
Donations to the EA Global Health and Development Fund, which is managed by GiveWell’s CEO and informed by our research team’s work, may support higher-risk, more exploratory opportunities, such as Incubation Grants, or may be granted to our top charities. As noted on the EA Funds page linked in the previous sentence, donations to this Fund are fungible with other support for GiveWell and our recommendations, as they are likely to displace Open Philanthropy’s funding for top charities. (See the answer to this question in the FAQ: “If the Fund is managed by the head of GiveWell, why doesn’t it just recommend that people give to GiveWell or its top charities?”)
(2) We discussed COVAX and COVID vaccination in response to this blog comment on the March 2021 open thread.
Any plans to make GiveWell a tax-deductable charity in Australia? Please do.
Hi Henry,
Thanks for your request! We don’t currently have plans to register in Australia, but it’s helpful to hear that you would like this. There are a number of options for tax-deductible giving to GiveWell’s recommended charities in Australia listed here.
Hi,
In the light of the newest IPCC report and various organisations including the WHO drawing attention to the effects of the climate crisis on health, life expectancy, and other issues you focus on, will you be supporting/recommending more climate change mitigation/prevention programmes? For example, on your page about clean cookstoves you say that they do not have enough of an impact on health to be very important. But cookstoves also fuel the climate crisis (I read 2% of global emissions but that seems high), so in switching to clean ones people are not only improving their own immediate health, but also that of others, as well as life expectancy, food security and so on – surely that changes the value of such a programme?
Hi Michaela,
Thanks for your question! As you note, we haven’t prioritized work on climate change organizations.
I’m glad to share some additional context. We’re in the process of evolving our research to consider a broader range of evidence. As part of that process, we are thinking about whether environmental and climate change charities could meet our criteria in the future; for example, we mention reducing carbon emissions in our February 2019 blog post on how our research is evolving to look at new areas that might impact the lives of people in poor countries. We have also done some preliminary research into air pollution as part of our investigation into high-leverage areas and public health regulation (see these conversation notes and these more recent conversation notes). That said, it is unlikely we will have an environmental or climate change charity recommendation in the near future (the next few years), as our thinking on these topics is quite preliminary, and we’re not sure that climate change and environmental causes are as promising by our criteria as others we’re prioritizing.
I’m not sure what the right place to ask this question is. I’m from a semi-large city in India and recently I read your report about site visits to Indian charities in Mumbai. It seemed to me that the most enduring and pragmatic way to help kids on the streets is in the form of drop-in centres a-la the Salaam Balak Trust centres. Their model is suited for children whose parents are unable to take care of them for large parts of the day but would not consent to separation and who do not permanently live in the city. However in the absence of such centres, what is the best way of helping these children? I realize this is not about charities per se but I am at a complete loss
Hi Chintu,
Thanks for reaching out about your concerns on behalf of children in your area. GiveWell aims to find charities that are evidence-backed, cost-effective, transparent, and underfunded (see our criteria). Though we consider many charities throughout our research process, we do not aim to rate or review each of them, so unfortunately we generally cannot answer questions about specific charitable models, such as the Salaam Balak Trust centers, that are not on our list of recommended organizations. For lists of the charities that we have considered, see here.
While not strictly related to your question, you may be interested in reading our report on deworming, and specifically Evidence Action’s Deworm the World Initiative, a GiveWell top charity that operates in several countries, including India.
Comments are closed.