Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at email@example.com or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.
You can view our June 2017 open thread here.
In light of the abundance of national disasters recently, I (and I can’t help but think many others) wonder which organizations out there supporting these relief efforts could and would use our money most effectively. I know this isn’t really where you focus because dollar for dollar money spent on global health in extremely poor countries does much more “good”, but it would be nice if there was some guidance for us “effective altruism” followers as to how best to help hurricane, fire, etc. victims that are practically our neighbors.
Thanks for your question. We recognize that there’s a lot of demand for guidance on where to give in the aftermath of a disaster; we wish we could be a more helpful resource, but unfortunately, we can’t generate high-quality recommendations for each disaster.
I’m following up GiveWell’s 2016 money moved metrics :). The last update appears to be at https://www.facebook.com/groups/GiveWellCommunity/permalink/462949240726562/?comment_id=467285776959575 which does not specify a target publication date and the update before that appears to be at https://blog.givewell.org/2017/03/21/march-2017-open-thread/#comment-943448 specifies a publication target of summer (which should be over by now).
Just curious – given the debate around how much an under-5 life matters (staff inputs for “value of an under-5 life” range from 0 to 50 DALYs on the latest version of the GiveWell CEA), why is it used as the unit of comparison on the “Results” tab?
**minor edit: actually staff inputs range from 0.14 to 50 DALYs. I didn’t realize the sheet was rounding.
Thanks for following up; my apologies that we have not yet published the report. We plan to take steps to significantly speed up the timeline by which metrics information is released next year.
This year, we will publish a report by October 1 at the latest, although we may not have as much information ready to publish at that time as we have in previous metrics reports. We do expect to have the most important bottom-line figures on money moved for publication then (i.e., our total money moved, the money moved to each charity, the amounts donated by donors giving different amounts, and updated web traffic statistics).
If we publish a shorter report by October 1st, we’ll still publish a complete report at a later date.
We’ve had some discussion about which metrics would be best to emphasize in the Results tab of our CEA. I don’t believe there’s a single easy-to-interpret metric that’s clearly better than all the other options. We’ve retained the “cost per under-5 life saved equivalent” metric since we’ve used it historically. Admittedly, this metric can be hard to interpret since individuals included in the CEA have a range of different views about the value of preventing deaths of children under 5.
If you prefer another metric, you can select other metrics in an editable copy of our CEA. To do this, click the arrow symbol on the right of cells containing the text “Cost per outcome as good as: averting the death of an individual under 5 — AMF,” to open a drop-down menu. On this menu, you will find other metrics like “Cost per outcome as good as: averting the death of an individual 5 or older — AMF” and “Cost per outcome as good as: short-term health benefits from one year of deworming.”
Some additional context on how this part of our CEA has changed:
We no longer make explicit use of the DALY framework in our CEA. Our use of the DALY framework in our CEA was becoming increasingly unconventional prior to this change. For example, at the end of 2016 many individuals included in our CEA attributed less than 10 averted DALYs to preventing the death of a child under 5 years old. Under the conventional DALY framework, a far greater number of DALYs averted should be attributed to averting an infant’s death.
In 2017, we moved away from explicitly using DALYs in our CEA. Now individuals using our CEA can assign different subjective values to each of the outcomes accounted for in our CEA. There’s no objective meaning to these values. What matters for the bottom line results in the CEA is how the values assigned to each outcome compare relative to one another. With this new structure, an individual can still elect to use the DALY framework by assigning a DALY weight to each outcome on the Personal values tab. While some individuals included in our published CEA take this approach, not everyone does. As an example of an alternative approach, one could choose to define a value of 1 unit as representing “doing something as good as doubling a poor individual’s consumption for one year.” That definition could then be used as an anchor for assigning values to other outcomes considered in the CEA.
We felt that using the DALY framework in an unconventional way could lead to confusion, while strictly adhering to the conventional version of the framework would prevent some individuals from adequately accounting for their views within our CEA. For example, the standard DALY framework–using age-weights and discounting conventionally–cannot reflect some individuals’ view that it would be far better to avert the death of a young adult than the death of a newborn infant. The flexibility of our new framework allows for accommodating views that don’t align with the conventional DALY framework.
How does GiveWell determine the size of the incubation grants it makes? Do charities submit grant proposals for specific amounts? (Apologies if I missed this explanation elsewhere on the site!)
We determine the size of the Incubation Grant in consultation with the charity. We ask the charity to submit a budget proposal and provide information about what could be achieved with different levels of funding (for example, scale-up of a program to a particular size). Then, we make a judgment call based on our assessment of the evidence base for the program and the charity’s track record about what kind of information we’d need to see in order to move the charity forward in consideration as a potential future top charity, and make a grant recommendation of that amount.
Giving from Europe
AMF seems to be pro-actively trying to gain tax-deductible status for European countries. As can be seen on their feed, Ireland and Denmark have both granted AMF tax-deductible status this year, and I know they have applied for the same in Belgium too.
As a general comment, I would love to see some articles on donating to GiveWell’s Top Charities as a European. It is a complex process; for example TGE, or Transnational Giving Europe, liaises for some charities but not all, and they take 5% off each donation. Tax rebates can be significant in European countries, but despite discussions with my local EA group, I am still not 100% clear how it all works.
The Netherlands has an online calculator, where you enter your Bruto year salary and how much you wish to give; it then spits the amount you get back, and tells you how much more you could give, should you wish to give as much as possible, after tax rebates.
Hi GiveWell team,
Thanks for taking questions.
I like following GiveWell’s suggestions about donating to the Against Malaria Foundation, and encourage my friends/ families to do the same. However I have always had a couple of niggling doubts about it.
Specifically, I know that the Bill and Melinda Gates Foundation & Good Ventures do a lot of work in the malaria area, and they have a lot of money. So how come money is still being sought for this issue? Is it that these groups are lacking money? Is it that they have some wisdom in not spending their money all at once? Is it that the Gates Foundation specifically are more interested in long term things (like malaria eradication, vaccines, genetically modified mosquitoes etc) and if so is GiveWell’s recommendation of AMF too short-term focused?
Ultimately, should GiveWell’s community of donor’s be mimicking the Gates Foundations and/ or Good Ventures behaviours?
Apologies if my questions are naive. I’m having a lot of trouble finding information about this.
Thank you for the suggestion! We’ll keep in mind the idea of writing more about European donation logistics in the future. In case it’s helpful, we’ve published some information on tax-deductibility here: https://www.givewell.org/how-we-work/research-faq#ArethetopcharitiestaxdeductiblewhereIlive and you can also reach out to firstname.lastname@example.org with specific questions.
Hello Give Well,
Thanks for all your work over the years!
Can you point me to an explanation of what the “cost per life saved equivalent” exactly is and the philosophical justification for it? I’ve looked through your website several times but not found anything. This, for example – https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness – tells me that you use it, not what it is or why you’ve picked it. I work in moral philosophy and as it stands I can’t evaluate whether either (a) GW is making the most cost-effective recommendations by it’s own moral theory or (b) which moral theories will concludes GW’s top charities are the best picks because I dont know what sort of assumptions you’re using (i.e. theory of value, view of the badness of death, stance on population ethics).
All the best,
p.s. as a full disclosure, I’ve critiqued GW’s evaluation of AMF before and this was your reply to me https://blog.givewell.org/2016/12/12/amf-population-ethics/)
Thanks for the thoughtful question!
For context, last year, we estimated the total funding gap for LLINs over the next 3 years at over $400 million: https://www.givewell.org/charities/against-malaria-foundation#Expectedmaximum. So, while the Bill & Melinda Gates Foundation or Good Ventures would be able to fill this gap if they chose to, it’s worth keeping in mind that the size of the gap would require even a funder that large to substantially change its strategy. (Note: LLINs are just one malaria prevention intervention; we have also looked into the funding gap for for seasonal malaria chemoprevention: https://www.givewell.org/charities/malaria-consortium#GlobalNeed.)
Both Good Ventures and the Bill & Melinda Gates Foundation give a significant amount to directly reduce malaria deaths in the short term, Good Ventures via donations to GiveWell’s recommended charities and the Bill & Melinda Gates Foundation primarily via donations to the Global Fund, among other organizations. So, these two funders give a substantial amount to direct efforts to help the global poor, but also aim to create impact in other ways. Good Ventures in particular also gives to riskier, more-difficult-to-evaluate areas as part of its giving strategy. For additional insight into how Good Ventures and the Open Philanthropy Project think about their strategy, see these posts: https://www.openphilanthropy.org/blog/good-ventures-and-giving-now-vs-later-2016-updateand https://www.openphilanthropy.org/blog/worldview-diversification.
We do not recommend that most individual donors attempt to mimic how these major funders behave. For more discussion of this topic, see this blog post: https://blog.givewell.org/2017/06/21/are-givewells-top-charities-the-best-option-for-every-donor/. One key consideration is that institutional funders – unlike most individual donors – are able to invest large amounts of time in their research, which enables them to have the context needed to identify good bets on riskier giving opportunities. We continue to believe that GiveWell’s top charities are the best option for most donors.
Question: Wouldn’t it be very high-impact to have certain essential parts of the GiveWell website available in other major languages?
I am currently writing an article in French for a magazine with a circulation of 11’000 copies. In October I’ll give a workshop in German. In both cases I would absolutlely *love* to recommend GiveWell and I know of few good alternatives to recommending GiveWell.
However, while a *certain* sizeable segment of the population here in non-english-speaking Europe is perfectly fine with navigating an English website, it’s tricky to use an English-only website as a recommendation for a broad audience. My guess is also that many English-speaking people in educated, cosmopolitan circles overestimate how many people in non-English-speaking countries are actually comfortable with English (particularly among the elder generation — including highly educated elder people).
Basically: The fact that the website is in English only is quite a barrier for spreading it *widely* in non-English-speaking countries. While translation of (key parts of) the website costs money, my guess is that this might be a highly valuable investment.
Apologies if I’ve missed previous discussions or other relevant bits of information.
Thanks for the suggestion! We have considered this in the past; our primary concern centered on finding a translation we were comfortable with, due to our interest in using highly precise language on the site to convey our recommendations and views on research and the fact that we update our website (including key pages) regularly. If we could overcome those two obstacles, we’d be excited to see GiveWell.org translated into other languages.
Thanks for making the deadline, Catherine!
Thanks for your thoughtful question – good hearing from you again!
First, I’d like to provide some quick context on the terminology in our cost-effectiveness analysis model (CEA, for short). Then, I’ll address your points (a) and (b).
The “cost per life saved equivalent” was a metric we used prior to 2017 to compare interventions that achieve very different results (e.g. averting deaths versus increasing incomes). To calculate the “cost per life saved equivalent,” individuals included in GiveWell’s CEA first assigned a number of disability-adjusted life years (DALYs) to each outcome considered in our CEA. One of these outcomes was averting the death of a child younger than 5 through AMF’s net distributions. The “cost per life saved equivalent” captures how much money it would take (according to our model) to achieve an outcome (or a set of outcomes) “worth” as many DALYs as averting the death of a child under 5 in AMF’s program. We calculated this metric because we wanted a common bottom-line figure that could be used to compare cost-effectiveness across charities.
In recent versions of our cost-effectiveness model, we switched to using “cost per outcome as good as [X],” where “X” can be selected by the user to represent the benefits of any of our top charities, such as “averting the death of an individual under 5 – AMF” or “doubling consumption for one person for one year” (see row 10 of the “Results” tab for options). These figures are calculated based on the inputs people enter into the “Personal Values” tab that indicate how they weight different outcomes against one another. The “cost per outcome as good as [X]” is formed by calculating how it much money would be needed to to achieve an outcome (or set of outcomes) “worth” as much as [Outcome X].
Note that we realized in response to your comment that we still list “cost per life saved equivalent” on our CEA overview page, and are planning to update this to reflect the fact that we no longer use that term in our model. Thank you for calling this to our attention.
(a) Is GiveWell using its own moral theory and (b) which moral theories will conclude GiveWell’s top charities are the best picks?
There is no single, consistent moral framework or theory GiveWell relies upon to generate its cost-effectiveness estimates. As indicated above, there are several sets of subjective values in our cost-effectiveness model. Most of these values are contributed by individuals who work on GiveWell’s research team.
How each person comes up with their values varies. For example, members of the team have drawn their values from the DALY framework, arguments made by moral philosophers, and personal views that have little relation to formal arguments made by moral philosophers. Individual values are not always set according to a consistent moral framework in all cases; aggregate values—GiveWell’s bottom-line figures—are not set in this way.
We find it difficult to make trade-offs between charities that accomplish different types of good. We try to be flexible in our current approach so that any number of different values systems can be accommodated by our cost-effectiveness model, which is used by members of GiveWell’s research team as well as available to anyone who comes to GiveWell’s website and is interested in inputting their own values. We may wish to modify this approach in the future as we continue engaging with this question. We hope to improve our communications around our approach (how individuals contributing to GiveWell’s published CEA articulate the reasoning for their values) as well as our approach itself going forward, and have plans to write about this in the future.
Two quick questions/comments about the draft Zinc/ORS CEA file (https://docs.google.com/spreadsheets/d/18IU465NO0O1Kop1cERpNnhDGOggbCZcCeo2d1Z9sczI/edit#gid=1020139846)
1) In the “Results” sheet, I think cell B7 is missing its formula
2) Can you set the linked workbooks in the importrange formulas (e.g., sheet “Value Assignments” cell A2) to be viewable by anyone with the link, or are those meant to be internal only?
Thanks for flagging both of these!
(1) We have updated the formula in cell B7.
(2) We made a mistake by including a link to that spreadsheet in our Zinc/ORS CEA—the “Value assignments and interim CEA results” sheet is intended to be internal to GiveWell. This document is a work in progress, and people who work on GiveWell’s cost-effectiveness model will sometimes input values they later intend to change; it also includes values for intervention reports we have not yet published, and where views might change as we work to finalize those reports. The published cost-effectiveness model includes values for each of the interventions we recommend: https://docs.google.com/spreadsheets/d/1ulAmsDjWKgV07DiUuvVCRRJOVLw-PCjXMe-Eq2J3DtY/edit#gid=1362437801. We want our staff to be able to modify and change their values internally before a ‘final’ version is published.
We’ve now removed the link from the Zinc/ORS CEA file. Apologies for any confusion this caused.
Have you reviewed malaria case rate data for areas where AMF has done a net distribution? (Or other outcome data comparing the incidence of malaria before & after an AMF distribution?)
It seems important to review actual outcome data for the intervention, especially given that AMF is a perennial GiveWell recommendation.
Thanks for the question. It’s not something we’ve considered recently, but given (a) the amount of money we expect to move to AMF in the future and (b) our partnership with IDinsight (who could potentially carry out this work), it’s an idea worth exploring.
First, some background. We wrote about AMF’s experience trying to get case rate data in Malawi here. In brief, we and AMF learned that it wasn’t possible to rely on the existing health infrastructure and administrative data for reliable data on malaria rates.
Second, because AMF monitors net ownership and use, my guess is that the most important question to answer is whether people having and owning nets leads to a reduction in malaria cases. My impression is that there’s a decent amount of research out there on this question, such as from researchers focusing on the possible impacts of insecticide resistance or places like the Malaria Atlas Project, which shares information about the impact of malaria and disease control efforts.
We have recently reviewed the literature related to insecticide resistance and hope to publish more information about that shortly (in brief, the data didn’t cause us to be more concerned than we were already about the impact of resistance); we haven’t looked at MAP data.
Finally, we’d guess that it would be very costly to set up a system to track case rates directly because it would likely require funding additional surveys after AMF net distributions. Because a number of factors (such as rainfall) influence malaria rates in a given year, we would need to do this in many cases to gather enough data to meaningfully update our view about AMF’s cost-effectiveness.
We put together a back of the envelope model to estimate how much this research would cost and how valuable it could be. My guess is that it’s very unlikely that we’d update based on additional surveys we conducted, but we’re planning to revisit this conclusion after we review the MAP data mentioned above.
Thanks for the thoughtful reply!
– Will the MAP data review be completed in time to influence the 2017 recommendations?
– The BOTEC model presents two scenarios – is one of them the best guess? (seems important because they imply different courses of action)
– Does GiveWell plan to stop recommending AMF 15 years from now? The BOTEC model implies that by only counting benefits 5 to 15 years in the future.
If not, this would be double-discounting the value of information – it seems more accurate to model far into the future with a compounding discount rate. I did that in this version of your model, which implies a much higher value of information from the research: https://docs.google.com/spreadsheets/d/1YDZ35Y-Ca9xjapjfcEdQErwzkxr96Fozcngy3X32t1M/edit?usp=sharing
Have you recently looked into organizations that focus on the issue of indoor air pollution in the developing world? Bjorn Lomborg has identified it as a major environmental killer (https://www.forbes.com/sites/bjornlomborg/2014/05/12/the-worlds-biggest-environmental-killer/#440442825a0f) but I’ve seen very little activism or campaigning around the issue.
It’s been several years since GiveWell last looked into the issue – I’m wondering if research around indoor air pollution has progressed since then, or if you’ve otherwise revisited this topic as an area of inquiry.
Did you give an incubation grant to the Centre for Pesticide Suicide Prevention? Do you think you might recommend them as a charity at some point?
I would find it helpful if you had a page listing all the discretionary regrants GiveWell has made, that is kept up to date. Currently I see
$4.9 million documented at https://blog.givewell.org/2017/04/03/allocation-of-discretionary-funds/
$2.25 million documented at https://blog.givewell.org/2017/08/30/why-were-allocating-discretionary-funds-to-the-deworm-the-world-initiative/
$0.7 million documented at https://blog.givewell.org/2017/11/27/our-top-charities-for-giving-season-2017/
However, I don’t know if this is a comprehensive list, and even if it is right now, I don’t know how to check for new regrants in the future.
This becomes more important now that donating funds for discretionary regranting is now your preferred suggestion for donors, as described in https://blog.givewell.org/2017/11/27/our-top-charities-for-giving-season-2017/
We did make an Incubation Grant to the Centre for Pesticide Suicide Prevention (CPSP) this year; a write-up is forthcoming.
A goal of our Incubation Grants program is to grow the pipeline of potential future top charities, and so CPSP may be considered as a potential top charity in the future.
Do you have a list of all publicly disclosed participation grants given out by GiveWell? https://www.givewell.org/charity-application-process says that charities are eligible for participation grants up to $100,000 if they cooperate with you for the first phase of being investigated for top charity status. The Open Philanthropy Project is listed as having awarded GiveWell $500,000 each for awarding participation grants in 2016 and 2017 respectively, see https://www.openphilanthropy.org/focus/global-health-and-development/miscellaneous/givewell-top-charity-participation-grants-2015 and https://www.openphilanthropy.org/focus/global-health-and-development/miscellaneous/givewell-top-charity-participation-grants-2016
Thanks for the idea! We’re planning to create such pages; we will link here once completed.
Don’t scientists already know what pesticides have the highest fatality rates? Ie. Class 1 and 2 pesticides. Isn’t it therefore more a matter of the public applying pressure to government to ban the HHPs? It worked for endosulfan in Kerala. I suppose the fatality rates of the commercial formulations is not usually known, but at least for most active ingredients it is generally known how toxic they are. If a country won’t ban, for example, all organophosphates, which are known to highly lethal, I don’t see why they would care about the fatality rates from commercial formulations. For motivated Ministries of Agriculture like in Nepal, it seems even more futile. Can’t CPSP just give them a list of HHPs to ban? There’s already a number of those.
Would GiveWell consider funding an advocacy organization to pressure governments to ban pesticides that are already known to be too dangerous to be on the market (especially in India which has most of the world’s pesticide deaths)? What about technical assistance to national pesticide registrars? Some poor countries are interested in banning HHPs but simply don’t know how to go about it, or may believe the myth that lockboxes prevent self-poisoning. Both seem to be more urgent than CPSP’s model.
CPSP’s research-oriented program looks great, don’t get me wrong, I just don’t see how it’s a higher priority than the above. Pesticide Action Network does anti-HHP advocacy, of course, but I don’t know how good they are at it. They seem to make HHP bans seem positively aversive to governments by pushing chemical-free agriculture as the only solution. Has GiveWell or OpenPhil considered actually starting their own charities for the highest impact interventions if no one seems to be interested in starting it? I mean, you could do the interventions as a program within your existing charities or create a team to start independent ones.
Thanks for your deep engagement with this grant.
“Don’t scientists already know what pesticides have the highest fatality rates? Ie. Class 1 and 2 pesticides. Isn’t it therefore more a matter of the public applying pressure to government to ban the HHPs?”
I agree there is existing evidence about the case fatality of different pesticides. See for example Dawson et al. 2010, a prospective cohort study in Sri Lanka (http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1000357). Deciding which pesticide regulations would have the greatest impact on reducing suicides is a function of (i) case fatality of that pesticide (ii) the number of self-harm attempts with that pesticide. Our understanding is the latter is not well known in India or Nepal, where CPSP are working, and may differ substantially from other countries.
“Would GiveWell consider funding an advocacy organization to pressure governments to ban pesticides that are already known to be too dangerous to be on the market (especially in India which has most of the world’s pesticide deaths)? What about technical assistance to national pesticide registrars?”
CPSP is providing technical assistance to national pesticide registrars. Their explicit goal is to influence pesticide regulation (rather than just carry out research). They believe the most promising way of doing that is to assist regulators with data collection and drafting of legislation. This is similar to the approach which was successful in Sri Lanka (see Pearson et al. 2015 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4287191/).
“Has GiveWell or OpenPhil considered actually starting their own charities for the highest impact interventions if no one seems to be interested in starting it?”
We (GiveWell) have not considered this in the past as we believe it falls outside our core competencies. We have provided early stage funding to a number of organizations as part of GiveWell Incubation Grants (including CPSP, and Charity Science: Health). You can see all our incubation grants here.
Thanks for the question! We have not researched indoor air pollution recently. However, we plan to look into ambient air pollution reduction as part of our work exploring policy advocacy on various issues as potential priority programs.
Thank you for the additional questions and sorry for the slow follow up. Elie's initial model was a way to think about the question but it was rough and, in some ways, inaccurate. I tried to put together a more thorough model that better reflects my thinking on this.
I added a separate sheet to Elie's model with my best guess and looked at our potential impact through 65 years as you did. I also changed the units of the bottom line value of this research to reflect the expected impact in terms of life-equivalents saved since the previous "Benefit / cost" ratio was vague.
On reflection, my instinct is that the right discount rate to use under these modeling assumptions would be closer to 9% as opposed to 3%. The 9% discount rate incorporates a roughly 5% annual chance that GiveWell stops recommending bednets and a rough 4% general discount rate (see staff inputs and explanations for general discount rates on the "Parameters" sheet of our latest CEA).
The model is sensitive to a number of inputs such as the relative cost-effectiveness of AMF vs. alternative giving opportunities in the case that this research leads to a negative update. If this research led us to believe that AMF was actually 4x less cost-effective than the alternative use of funds then funding this research would be competitive with current top charities.
We didn't complete an analysis of the MAP data before we released our updated recommendations last week, but my impression is that the MAP data generally shows positive results. (See this Giving What We Can blog post from December 2015 summarizing MAP data; we previously updated our report on this topic in 2013.)
For context, it would be a very large project to dig into the MAP data sufficiently to conclude that we don't agree with the headline results.
So, what's next?
– We're going to chat with IDinsight about the costs of this sort of study to see whether we can get a more accurate cost estimate.
– Within the next couple of weeks, we'll publish some work we did to review a couple of new insecticide resistance studies (mentioned in previous comment).
– We'll consider prioritizing a deeper look at the MAP data next year or in the future.
– The 9% discount rate strikes me as steep, though there is a big range of opinion about what discount rate is intuitive.
– Very interested to see the outcome of the IDinsight conversation; seems plausible that 80% of the benefit here could be had from studying ~4 distributions (which would have a big impact on the cost-effect).
I am absolutely thrilled that CPSP exists and that GiveWell has decided to fund it. I just wonder what else can be done to mitigate pesticide suicides. I am obsessed with suicide prevention, as you know, and dealing with Michael and Leah got me nowhere.
From talking to Gamini, doing outreach to national registrars to provide authoritative information and tech support appears to be the most important activity right now, so that’s my main concern. But I also think that for a major world issue like pesticide suicide prevention, it should be something that everyone should have the opportunity to engage in if they want to and right now there is no guidance for the public on how, either for direct involvement, advocacy, or financial support.
Comments are closed.