The GiveWell Blog

September 2017 open thread

Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

You can view our June 2017 open thread here.

Comments

  • In light of the abundance of national disasters recently, I (and I can’t help but think many others) wonder which organizations out there supporting these relief efforts could and would use our money most effectively. I know this isn’t really where you focus because dollar for dollar money spent on global health in extremely poor countries does much more “good”, but it would be nice if there was some guidance for us “effective altruism” followers as to how best to help hurricane, fire, etc. victims that are practically our neighbors.

  • Catherine (GiveWell) on September 12, 2017 at 2:08 pm said:

    Hi Nathan,

    Thanks for your question. We recognize that there’s a lot of demand for guidance on where to give in the aftermath of a disaster; we wish we could be a more helpful resource, but unfortunately, we can’t generate high-quality recommendations for each disaster.

  • Vipul Naik on September 12, 2017 at 4:37 pm said:

    I’m following up GiveWell’s 2016 money moved metrics :). The last update appears to be at https://www.facebook.com/groups/GiveWellCommunity/permalink/462949240726562/?comment_id=467285776959575 which does not specify a target publication date and the update before that appears to be at https://blog.givewell.org/2017/03/21/march-2017-open-thread/#comment-943448 specifies a publication target of summer (which should be over by now).

  • Just curious – given the debate around how much an under-5 life matters (staff inputs for “value of an under-5 life” range from 0 to 50 DALYs on the latest version of the GiveWell CEA), why is it used as the unit of comparison on the “Results” tab?

  • **minor edit: actually staff inputs range from 0.14 to 50 DALYs. I didn’t realize the sheet was rounding.

  • Catherine (GiveWell) on September 13, 2017 at 5:58 pm said:

    Hi Vipul,

    Thanks for following up; my apologies that we have not yet published the report. We plan to take steps to significantly speed up the timeline by which metrics information is released next year.

    This year, we will publish a report by October 1 at the latest, although we may not have as much information ready to publish at that time as we have in previous metrics reports. We do expect to have the most important bottom-line figures on money moved for publication then (i.e., our total money moved, the money moved to each charity, the amounts donated by donors giving different amounts, and updated web traffic statistics).

    If we publish a shorter report by October 1st, we’ll still publish a complete report at a later date.

  • Chris (GiveWell) on September 13, 2017 at 9:02 pm said:

    Hi Milan,

    We’ve had some discussion about which metrics would be best to emphasize in the Results tab of our CEA. I don’t believe there’s a single easy-to-interpret metric that’s clearly better than all the other options. We’ve retained the “cost per under-5 life saved equivalent” metric since we’ve used it historically. Admittedly, this metric can be hard to interpret since individuals included in the CEA have a range of different views about the value of preventing deaths of children under 5.

    If you prefer another metric, you can select other metrics in an editable copy of our CEA. To do this, click the arrow symbol on the right of cells containing the text “Cost per outcome as good as: averting the death of an individual under 5 — AMF,” to open a drop-down menu. On this menu, you will find other metrics like “Cost per outcome as good as: averting the death of an individual 5 or older — AMF” and “Cost per outcome as good as: short-term health benefits from one year of deworming.”

    Some additional context on how this part of our CEA has changed:

    We no longer make explicit use of the DALY framework in our CEA. Our use of the DALY framework in our CEA was becoming increasingly unconventional prior to this change. For example, at the end of 2016 many individuals included in our CEA attributed less than 10 averted DALYs to preventing the death of a child under 5 years old. Under the conventional DALY framework, a far greater number of DALYs averted should be attributed to averting an infant’s death.

    In 2017, we moved away from explicitly using DALYs in our CEA. Now individuals using our CEA can assign different subjective values to each of the outcomes accounted for in our CEA. There’s no objective meaning to these values. What matters for the bottom line results in the CEA is how the values assigned to each outcome compare relative to one another. With this new structure, an individual can still elect to use the DALY framework by assigning a DALY weight to each outcome on the Personal values tab. While some individuals included in our published CEA take this approach, not everyone does. As an example of an alternative approach, one could choose to define a value of 1 unit as representing “doing something as good as doubling a poor individual’s consumption for one year.” That definition could then be used as an anchor for assigning values to other outcomes considered in the CEA.

    We felt that using the DALY framework in an unconventional way could lead to confusion, while strictly adhering to the conventional version of the framework would prevent some individuals from adequately accounting for their views within our CEA. For example, the standard DALY framework–using age-weights and discounting conventionally–cannot reflect some individuals’ view that it would be far better to avert the death of a young adult than the death of a newborn infant. The flexibility of our new framework allows for accommodating views that don’t align with the conventional DALY framework.

  • How does GiveWell determine the size of the incubation grants it makes? Do charities submit grant proposals for specific amounts? (Apologies if I missed this explanation elsewhere on the site!)

  • Catherine (GiveWell) on September 15, 2017 at 12:30 pm said:

    Hi Jeremy,

    We determine the size of the Incubation Grant in consultation with the charity. We ask the charity to submit a budget proposal and provide information about what could be achieved with different levels of funding (for example, scale-up of a program to a particular size). Then, we make a judgment call based on our assessment of the evidence base for the program and the charity’s track record about what kind of information we’d need to see in order to move the charity forward in consideration as a potential future top charity, and make a grant recommendation of that amount.

  • Timo Carlier on September 16, 2017 at 3:31 am said:

    Giving from Europe
    AMF seems to be pro-actively trying to gain tax-deductible status for European countries. As can be seen on their feed, Ireland and Denmark have both granted AMF tax-deductible status this year, and I know they have applied for the same in Belgium too.

    As a general comment, I would love to see some articles on donating to GiveWell’s Top Charities as a European. It is a complex process; for example TGE, or Transnational Giving Europe, liaises for some charities but not all, and they take 5% off each donation. Tax rebates can be significant in European countries, but despite discussions with my local EA group, I am still not 100% clear how it all works.

    The Netherlands has an online calculator, where you enter your Bruto year salary and how much you wish to give; it then spits the amount you get back, and tells you how much more you could give, should you wish to give as much as possible, after tax rebates.

  • Mohammad on September 19, 2017 at 7:46 am said:

    Hi GiveWell team,
    Thanks for taking questions.
    I like following GiveWell’s suggestions about donating to the Against Malaria Foundation, and encourage my friends/ families to do the same. However I have always had a couple of niggling doubts about it.

    Specifically, I know that the Bill and Melinda Gates Foundation & Good Ventures do a lot of work in the malaria area, and they have a lot of money. So how come money is still being sought for this issue? Is it that these groups are lacking money? Is it that they have some wisdom in not spending their money all at once? Is it that the Gates Foundation specifically are more interested in long term things (like malaria eradication, vaccines, genetically modified mosquitoes etc) and if so is GiveWell’s recommendation of AMF too short-term focused?

    Ultimately, should GiveWell’s community of donor’s be mimicking the Gates Foundations and/ or Good Ventures behaviours?

    Apologies if my questions are naive. I’m having a lot of trouble finding information about this.

  • Isabel (GiveWell) on September 21, 2017 at 4:59 pm said:

    Hi Timo,
    Thank you for the suggestion! We’ll keep in mind the idea of writing more about European donation logistics in the future. In case it’s helpful, we’ve published some information on tax-deductibility here: https://www.givewell.org/how-we-work/research-faq#ArethetopcharitiestaxdeductiblewhereIlive and you can also reach out to donations@givewell.org with specific questions.

  • Michael Plant on September 26, 2017 at 1:13 pm said:

    Hello Give Well,

    Thanks for all your work over the years!

    Can you point me to an explanation of what the “cost per life saved equivalent” exactly is and the philosophical justification for it? I’ve looked through your website several times but not found anything. This, for example – https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness – tells me that you use it, not what it is or why you’ve picked it. I work in moral philosophy and as it stands I can’t evaluate whether either (a) GW is making the most cost-effective recommendations by it’s own moral theory or (b) which moral theories will concludes GW’s top charities are the best picks because I dont know what sort of assumptions you’re using (i.e. theory of value, view of the badness of death, stance on population ethics).

    All the best,
    Michael Plant

    p.s. as a full disclosure, I’ve critiqued GW’s evaluation of AMF before and this was your reply to me https://blog.givewell.org/2016/12/12/amf-population-ethics/)

  • Isabel (GiveWell) on September 26, 2017 at 3:33 pm said:

    Hi Mohammad,
    Thanks for the thoughtful question!
    For context, last year, we estimated the total funding gap for LLINs over the next 3 years at over $400 million: https://www.givewell.org/charities/against-malaria-foundation#Expectedmaximum. So, while the Bill & Melinda Gates Foundation or Good Ventures would be able to fill this gap if they chose to, it’s worth keeping in mind that the size of the gap would require even a funder that large to substantially change its strategy. (Note: LLINs are just one malaria prevention intervention; we have also looked into the funding gap for for seasonal malaria chemoprevention: https://www.givewell.org/charities/malaria-consortium#GlobalNeed.)
    Both Good Ventures and the Bill & Melinda Gates Foundation give a significant amount to directly reduce malaria deaths in the short term, Good Ventures via donations to GiveWell’s recommended charities and the Bill & Melinda Gates Foundation primarily via donations to the Global Fund, among other organizations. So, these two funders give a substantial amount to direct efforts to help the global poor, but also aim to create impact in other ways. Good Ventures in particular also gives to riskier, more-difficult-to-evaluate areas as part of its giving strategy. For additional insight into how Good Ventures and the Open Philanthropy Project think about their strategy, see these posts: https://www.openphilanthropy.org/blog/good-ventures-and-giving-now-vs-later-2016-updateand https://www.openphilanthropy.org/blog/worldview-diversification.
    We do not recommend that most individual donors attempt to mimic how these major funders behave. For more discussion of this topic, see this blog post: https://blog.givewell.org/2017/06/21/are-givewells-top-charities-the-best-option-for-every-donor/. One key consideration is that institutional funders – unlike most individual donors – are able to invest large amounts of time in their research, which enables them to have the context needed to identify good bets on riskier giving opportunities. We continue to believe that GiveWell’s top charities are the best option for most donors.

  • Question: Wouldn’t it be very high-impact to have certain essential parts of the GiveWell website available in other major languages?

    I am currently writing an article in French for a magazine with a circulation of 11’000 copies. In October I’ll give a workshop in German. In both cases I would absolutlely *love* to recommend GiveWell and I know of few good alternatives to recommending GiveWell.

    However, while a *certain* sizeable segment of the population here in non-english-speaking Europe is perfectly fine with navigating an English website, it’s tricky to use an English-only website as a recommendation for a broad audience. My guess is also that many English-speaking people in educated, cosmopolitan circles overestimate how many people in non-English-speaking countries are actually comfortable with English (particularly among the elder generation — including highly educated elder people).

    Basically: The fact that the website is in English only is quite a barrier for spreading it *widely* in non-English-speaking countries. While translation of (key parts of) the website costs money, my guess is that this might be a highly valuable investment.

    Apologies if I’ve missed previous discussions or other relevant bits of information.

  • Catherine (GiveWell) on September 28, 2017 at 11:30 am said:

    Hi Dominic,

    Thanks for the suggestion! We have considered this in the past; our primary concern centered on finding a translation we were comfortable with, due to our interest in using highly precise language on the site to convey our recommendations and views on research and the fact that we update our website (including key pages) regularly. If we could overcome those two obstacles, we’d be excited to see GiveWell.org translated into other languages.

  • Vipul Naik on September 29, 2017 at 8:52 pm said:

    Thanks for making the deadline, Catherine!

  • Catherine (GiveWell) on October 2, 2017 at 2:23 pm said:

    Hi Michael,

    Thanks for your thoughtful question – good hearing from you again!

    First, I’d like to provide some quick context on the terminology in our cost-effectiveness analysis model (CEA, for short). Then, I’ll address your points (a) and (b).

    Terminology

    The “cost per life saved equivalent” was a metric we used prior to 2017 to compare interventions that achieve very different results (e.g. averting deaths versus increasing incomes). To calculate the “cost per life saved equivalent,” individuals included in GiveWell’s CEA first assigned a number of disability-adjusted life years (DALYs) to each outcome considered in our CEA. One of these outcomes was averting the death of a child younger than 5 through AMF’s net distributions. The “cost per life saved equivalent” captures how much money it would take (according to our model) to achieve an outcome (or a set of outcomes) “worth” as many DALYs as averting the death of a child under 5 in AMF’s program. We calculated this metric because we wanted a common bottom-line figure that could be used to compare cost-effectiveness across charities.

    In recent versions of our cost-effectiveness model, we switched to using “cost per outcome as good as [X],” where “X” can be selected by the user to represent the benefits of any of our top charities, such as “averting the death of an individual under 5 – AMF” or “doubling consumption for one person for one year” (see row 10 of the “Results” tab for options). These figures are calculated based on the inputs people enter into the “Personal Values” tab that indicate how they weight different outcomes against one another. The “cost per outcome as good as [X]” is formed by calculating how it much money would be needed to to achieve an outcome (or set of outcomes) “worth” as much as [Outcome X].

    Note that we realized in response to your comment that we still list “cost per life saved equivalent” on our CEA overview page, and are planning to update this to reflect the fact that we no longer use that term in our model. Thank you for calling this to our attention.

    (a) Is GiveWell using its own moral theory and (b) which moral theories will conclude GiveWell’s top charities are the best picks?

    There is no single, consistent moral framework or theory GiveWell relies upon to generate its cost-effectiveness estimates. As indicated above, there are several sets of subjective values in our cost-effectiveness model. Most of these values are contributed by individuals who work on GiveWell’s research team.

    How each person comes up with their values varies. For example, members of the team have drawn their values from the DALY framework, arguments made by moral philosophers, and personal views that have little relation to formal arguments made by moral philosophers. Individual values are not always set according to a consistent moral framework in all cases; aggregate values—GiveWell’s bottom-line figures—are not set in this way.

    We find it difficult to make trade-offs between charities that accomplish different types of good. We try to be flexible in our current approach so that any number of different values systems can be accommodated by our cost-effectiveness model, which is used by members of GiveWell’s research team as well as available to anyone who comes to GiveWell’s website and is interested in inputting their own values. We may wish to modify this approach in the future as we continue engaging with this question. We hope to improve our communications around our approach (how individuals contributing to GiveWell’s published CEA articulate the reasoning for their values) as well as our approach itself going forward, and have plans to write about this in the future.

  • Two quick questions/comments about the draft Zinc/ORS CEA file (https://docs.google.com/spreadsheets/d/18IU465NO0O1Kop1cERpNnhDGOggbCZcCeo2d1Z9sczI/edit#gid=1020139846)
    1) In the “Results” sheet, I think cell B7 is missing its formula
    2) Can you set the linked workbooks in the importrange formulas (e.g., sheet “Value Assignments” cell A2) to be viewable by anyone with the link, or are those meant to be internal only?
    Thanks!

  • Catherine (GiveWell) on October 17, 2017 at 4:10 pm said:

    Hi Jeremy,


    Thanks for flagging both of these!


    (1) We have updated the formula in cell B7.


    (2) We made a mistake by including a link to that spreadsheet in our Zinc/ORS CEA—the “Value assignments and interim CEA results” sheet is intended to be internal to GiveWell. This document is a work in progress, and people who work on GiveWell’s cost-effectiveness model will sometimes input values they later intend to change; it also includes values for intervention reports we have not yet published, and where views might change as we work to finalize those reports. The published cost-effectiveness model includes values for each of the interventions we recommend: https://docs.google.com/spreadsheets/d/1ulAmsDjWKgV07DiUuvVCRRJOVLw-PCjXMe-Eq2J3DtY/edit#gid=1362437801. We want our staff to be able to modify and change their values internally before a ‘final’ version is published.

    We’ve now removed the link from the Zinc/ORS CEA file. Apologies for any confusion this caused.

  • Have you reviewed malaria case rate data for areas where AMF has done a net distribution? (Or other outcome data comparing the incidence of malaria before & after an AMF distribution?)

    It seems important to review actual outcome data for the intervention, especially given that AMF is a perennial GiveWell recommendation.

  • Elie (GiveWell) on November 18, 2017 at 8:45 pm said:

    Hi Milan,

    Thanks for the question. It’s not something we’ve considered recently, but given (a) the amount of money we expect to move to AMF in the future and (b) our partnership with IDinsight (who could potentially carry out this work), it’s an idea worth exploring.

    First, some background. We wrote about AMF’s experience trying to get case rate data in Malawi here. In brief, we and AMF learned that it wasn’t possible to rely on the existing health infrastructure and administrative data for reliable data on malaria rates.

    Second, because AMF monitors net ownership and use, my guess is that the most important question to answer is whether people having and owning nets leads to a reduction in malaria cases. My impression is that there’s a decent amount of research out there on this question, such as from researchers focusing on the possible impacts of insecticide resistance or places like the Malaria Atlas Project, which shares information about the impact of malaria and disease control efforts.

    We have recently reviewed the literature related to insecticide resistance and hope to publish more information about that shortly (in brief, the data didn’t cause us to be more concerned than we were already about the impact of resistance); we haven’t looked at MAP data.

    Finally, we’d guess that it would be very costly to set up a system to track case rates directly because it would likely require funding additional surveys after AMF net distributions. Because a number of factors (such as rainfall) influence malaria rates in a given year, we would need to do this in many cases to gather enough data to meaningfully update our view about AMF’s cost-effectiveness.

    We put together a back of the envelope model to estimate how much this research would cost and how valuable it could be. My guess is that it’s very unlikely that we’d update based on additional surveys we conducted, but we’re planning to revisit this conclusion after we review the MAP data mentioned above.

New Comment

Your email address will not be published. Required fields are marked *

*