The GiveWell Blog

Investigating neglected goals in scientific research

A major goal of the Open Philanthropy Project is to explore the topic of scientific research funding, starting with life sciences. This post discusses the process we’ve used so far, including some of the challenges we’ve faced and changes we’ve made in our investigation methods:

  • We first discuss some of the general challenges of finding good giving opportunities in this space.
  • We then introduce the concept of scientific research “gaps” – areas that the existing system doesn’t put enough investment into, leaving potential philanthropic opportunities. One type of gap, which we call a “neglected goal,” has been the focus of many of our efforts so far.
  • We discuss our process so far for investigating neglected goals, and our plans for the future. Future posts will discuss other types of potential gaps that we think could be very important, but would find more difficult to investigate: gaps in high-risk early-stage research and gaps in “translational” research that sits between academic and industry work.

Some challenges of investigating scientific research funding
There are several reasons that we’re finding this space particularly challenging.

One is the sheer level of expertise required. As we’ve written previously, we often don’t feel positioned even to understand the meaning – much less the plausibility – of many key claims. We’ve sought generalist scientific advisors to help us with this issue. My early intuition is that even with strong scientific advisors, it would take far more time to do a shallow investigation for a cause in this space than for a cause in another space, which may mean we have to take more shortcuts in order to arrive at priorities.

Another challenge – discussed more in later sections of this post – is that it seems more difficult to get engagement from relevant experts. When investigating U.S. policy and global catastrophic risks, we’ve found that many of the people we want to talk to see educating the public and/or influencing funders as an essential part of their role. This is often much less true of scientists.

Another challenge is that there is already a lot of funding going toward science (particularly life sciences). Tens of billions of dollars per year come from U.S. government and industry sources (see data we’ve pulled from a government survey on R&D funding at universities (details of the query are in the spreadsheet) (XLS)). In addition, philanthropic sources are significant: Howard Hughes Medical Institute alone spends in the range of $700 million per year. It’s reasonable to ask how much value a new funder – even a relatively large one – can add in this context.

Gaps and neglected goals
In our early conversations, we’ve therefore tried to focus on what we call “gaps” (or sometimes “broad market inefficiencies“) in the current system of scientists, funders and supporting institutions. We’ve sought out institutional constraints, suboptimal incentives, and other factors that might make the current system “miss” particularly strong opportunities to do good. Our hope is that a new funder might be able to do an outsized amount of good by supporting what the existing system can’t or won’t.

One type of gap is what I’ll call a “neglected goal”: a case where the existing system doesn’t put enough investment into solving a particular social problem or developing a particular kind of technology. For example, improving malaria control and elimination efforts is arguably a neglected goal. New drugs, vaccines, and other methods for controlling malaria could result in helping an enormous number of people, mostly low-income people in the developing world. Because relatively few wealthy people are affected by malaria, both government and industry may not put enough funding into the scientific research and development necessary to develop such things. Similar logic could be extended to tuberculosis, HIV/AIDS, and other diseases/conditions that primarily affect the global poor. The Gates Foundation has often publicly expressed this line of reasoning.

More broadly, I’m familiar with arguments for many other neglected goals, such as developing in vitro meat (to reduce animal suffering); improving treatment of chronic pain; treating and slowing symptoms of aging; developing methods of food production that are primarily useful in worst-case scenarios; developing biological interventions that enhance people’s abilities rather than simply counteract diseases; and more.

Neglected goals are not the only kinds of gaps we’ve considered. However, they are probably the easiest for non-scientists to understand and engage with. They stem from insufficient societal attention to a particular social problem, rather than issues with the details of how science is pursued. They are the focus of this post; future posts will discuss other types of potential gaps.

Our progress so far on evaluating neglected goals
When exploring an unfamiliar area, we often take the approach of:

  • Starting with a question that seems both (a) relatively tangible and tractable, and (b) reasonably representative of other, future investigations we might do in the area.
  • Being ready to take many wrong turns and rethink our process multiple times, until we come to a satisfying answer to the question.
  • Try to repeat our process for other similar, questions, making it more efficient and systematic over time.

In this case, we decided – about a year ago – to tackle the question: “How much good could we accomplish by funding research and development targeting the needs of the global poor?” This is probably the most straightforward example I know of a potential neglected goal. We recruited a small group of scientific advisors and set up periodic calls and meetings to work on this investigation, though it was a low priority for the year (our main goal was progress on U.S. policy and global catastrophic risks).

This question, while more contained than “What sort of scientific research should we fund?”, is extremely broad. Early in our investigation, we determined that it included the following sub-questions:

  1. Would it be most productive to focus on malaria, tuberculosis, HIV/AIDS, water/sanitation-related applications, or something else?
  2. What sort of research is likely to be most valuable? Presumably, there is more work being done on diseases like cancer than on diseases like malaria – but where is the gap greatest and most important? For example, would it be best to focus on earlier-stage research or later-stage research?
  3. How should we view the work of the Gates Foundation, which has shown a strong interest in the idea that this sort of research is underfunded? Should we believe that the Gates Foundation is taking the most effective possible approach (in which case the best research in this area would be research the Gates Foundation is funding) or that there are important opportunities it is missing?
  4. How valuable is the most valuable work in this area, compared to other possible uses of money?

After an initial period of better defining these questions, we started scheduling conversations with people who could bring a broad perspective to these questions, help us cut through many possibilities and identify the most promising types of research to support. We sought to speak with people who work on developed-world diseases such as cancer, in order to get perspective on #2 above. We sought to speak with people who have broad, cross-cutting roles working on developing-world diseases, in order to get perspective on #1 and #3.

Unfortunately, we ultimately weren’t able to make the sort of progress we had hoped for using this method. The people we hoped to speak with were not always interested in talking, and when they were it was often off the record. In most cases, we didn’t get much feedback on which types of research were most promising.

Trying a different approach, I spoke with two of our junior scientific advisors about the fields they knew best. I spoke about HIV/AIDS, tuberculosis, and other areas with Anna Bershteyn of the Institute for Disease Modeling (conversation notes here), and I spoke about malaria (primarily focused on drug development and risks of drug resistance) with Micah Manary, a graduate student specializing in malaria (conversation notes here). For simplicity, I focus here on the second conversation, though similar dynamics applied to both.

Rather than focusing on broad questions about the most promising paths, I aimed to gain a basic understanding of how malaria works, what the shortcomings are in current treatment/control methods, and how new drugs/diagnostics/vaccines/other technologies might help. I learned that one of the most important goals of drug development is simply to stay ahead in the race against drug resistance, which evolves rapidly and presents a major problem. I learned that due to the complexity of the parasite, there’s fairly little basis for predicting what sort of compound might be effective in killing it, so much of drug development comes down to essentially testing random compounds against malaria samples in a lab. Learning these things gave me a basic framework for thinking about how to quantify the humanitarian benefits of malaria-focused research and development: for any given research path (from funding drug development directly to funding higher-risk research aimed at making drug development more efficient), one could estimate the effect on the number of compounds tested per year, and from there estimate the impact of testing additional compounds on the threat of drug resistance among other things. Having this framework in mind made it easier to see how a variety of different research paths could be relevant, and what their benefits might look like.

Details are available via our public notes on the conversation.

We’re still far from having a view on the best research paths for malaria drug development, and we’re still far from an estimate of how much good (per dollar) funding such things would accomplish. However, the process for getting there seems more clearly defined and tractable than it did a year ago. I now see such investigations as ideally consisting of:

  • A scientist who can familiarize himself/herself with how a disease works, what methods we have for controlling it, what potential future technologies might be useful, and what kind of science might help speed these along – while also being available for intensive conversations with GiveWell staff.
  • A GiveWell staffer who can interview the scientist about potential research paths, think about the likely “good accomplished per dollar” of such paths, and work with the scientist to learn more about the most promising ones.

We’re currently working on two such investigations, each involving a different scientific advisor. Due to the complexity of the subjects and the limited time our advisors have available, we expect these investigations to be challenging and to take a few months. As we go, we’ll be thinking about how to make the process more systematic and efficient, how to increase our scientific advisory capacity, and how else we might narrow the field of possible neglected goals to focus on.

Comments

  • Mitul Sarkar on March 27, 2015 at 8:09 pm said:

    Mr. Holden, a hat-tip to you for your GiveWell vision, discussion and the tons of typing (hopefully, dictating).

    I’d suggest one GiveWell staffer integrating the knowledge gleaned from not one but 2-3 or more scientists. These scientists can include “junior” scientists, like PhD students and postdocs, too (increasing numbers among them are looking to transfer their research skills to outside academia, so there’s their motivation!).

    You might also look at the challenges regarding “gaps” as posed by Hult Prize over the past few years, and consider how “crowdsourcing” the research and solutions might fit with your plans for GiveWell’s future.

    Also, I’d vote for this- “developing biological interventions that enhance people’s abilities rather than simply counteract diseases” (your words).

  • M. Jahi Chappell on April 10, 2015 at 11:45 am said:

    Dear Holden,

    I have followed GiveWell’s work with interest for some years now. Looking at this post, it leads me to once again wonder about the lack of focus on social change–which is so much harder to measure, but arguably the most impactful changes possible. Analyses make clear that many of the most fundamental challenges facing us–hunger/food insecurity, non-communicable diseases, insufficient treatment for treatable diseases for the poor–are those that we already have “the solutions” too (in terms of technical ability). There is more than enough food produced, most countries have food energy sufficient for their population, it takes five times more food to “trickle down” to the poorest in Latin America than it would to simply supply sufficient food at even distribution, we waste 20-40% of food, etc. (References available on request.) We even have very compelling evidence that smaller farms are more productive per unit area, yet this is marginalized as a result because it does not fit prevailing economic theory.

    All of this is to say that, arguably, the biggest changes producing mass improvements in quality of life and liberty have resulted from social movements: the end of Apartheid and slavery; increased rights for women, including reproductive rights and legal rights to fight domestic violence; public schooling; the end of US apartheid (Jim Crow); the vast improvements in food security and decrease in poverty severity in Brazil under Fome Zero — and so on. (Again, references on request.) Brazil is arguably seeing the biggest modern decrease in poverty severity, ever.

    All of these things are hard to quantify and measure and subdivide, certainly hard to calculate dollar gained per dollar invested. But I humbly ask those working this area (human development & well-being): if changes like improved labor laws and workplace safety and improved rights for marginalized population (by gender, race, etc.) do substantially boost well-being–and I think the evidence is fairly clear that they do–and the things that bring such changes about are not simply random (which I think they are not), doesn’t it behoove us to fill the research gap of “how does effective [redistributive?] change happen?”

    Funding basic treatment for treatable diseases, or schemes to increase access to food (which conditional cash transfers have had some success in–I remember your research on cash transfers and wonder if you haven’t tried looking at it with regards to food security) are essentially redistributive. Alleviating poverty is usually, at some level, redistributive (even if we look at the “rising tide” ideas around economic growth, (a) they tend to help the poor only if actions to spread gains are taken, being effectively redistributive vs. the “non-intervention” distribution of gains; (b) the metaphor of a rising tide is likely apt–a rising tide lifts all boats because and only because water spreads perfectly evenly; (c) economic growth is almost certainly, from basic thermodynamics, not indefinitely possible.

    In short: I think social movements for social redistribution (be it of political power, gains or growth) have arguably done the most good for human welfare, they almost certainly have done *large* amounts of good for human welfare by any measure. The fact that the dynamics are immensely difficult to suss out means it is precisely the kind of under-supported scientific endeavor you mention; and unless one thinks social change is simply “irreducibly complex”, the fact that it is hard to measure means we need to improve our investments in understanding it, rather than leave it aside because it is “too hard.”

  • Michael Schwartz on April 10, 2015 at 2:23 pm said:

    As an additional complication, the best results might not come from the ‘traditional’ route of developing drugs/vaccines. It is possible that other paths are more effective/efficient, ranging from the low-tech and cheap distribution of mosquito netting, to the high-tech such as developing and releasing a population of mosquitoes genetically engineered to be resistant to the parasite (with all the questions involved in such uses of genetic engineering). An important gap might be in funding the discovery of such new ‘outside-the-box’ solutions to problems.

  • Holden on April 20, 2015 at 1:57 pm said:

    Thanks for the comments, both.

    M. Jahi Chappell: I think a great deal of good has been done by all three of (a) social movements (b) scientific innovation (c) direct aid, and we have done some work (though of varying stages of development) to investigate all three. I think the work we’ve done that is most relevant to your comments is our work on U.S. policy.

    Michael Schwartz: I agree that there is an important distinction between traditional approaches and higher-risk, higher-reward approaches. With limited investigative resources, I find it most helpful to find the areas that seem least crowded relative to their importance, then examine the potential of both traditional and nontraditional approaches in those areas.

  • Tom Stocker on May 27, 2015 at 5:37 pm said:

    I’m sure you’ve thought about crowdsourcing neglected areas in science by putting out a high profile prize or essay competition for scientists and grad students that can identify work most worth funding or something similar? Cochrane often have such initiatives to identify how to improve systematic reviews / evidence base. What did you conclude?

Comments are closed.