The GiveWell Blog

Open Philanthropy Project: Progress in 2014 and plans for 2015

(updated on: ) | by Holden

This is the fourth post (of six) we’re planning to make focused on our self-evaluation and future plans.

We’re in the midst of finalizing detailed updates on Open Philanthropy Project progress and plans. This post gives a high-level summary, comparing our progress and stage of development with what we hoped for as of a year ago. In brief:

  • We made substantial progress on our main priorities: U.S. policy and global catastrophic risks. The precise nature of our goal (“commitments” to causes) shifted, but we have completed a substantial number of high-level cause investigations and decided on our working cause priorities. We are now shifting our focus from cause investigations to aiming for major grants and/or hires. Within the next six months, we hope to make at least one major grant and/or specialized hire in each of (a) U.S. policy and (b) global catastrophic risks; we consider this a “stretch goal” (substantial probability we will fail to hit it), and we plan another high-level check-in around 6 months from now.
  • We made less progress than hoped on other cause categories: scientific research funding and global health and development. We put substantial time into scientific research funding, but have found it to be probably the most complex and challenging of the broad categories. Our main goal for 2015 is to form clear priorities within scientific research funding, comparable to where we currently stand on U.S. policy and global catastrophic risks. This is a stretch goal. We don’t plan to work on global health and development this year for the Open Philanthropy Project.
  • We made some progress in separating the Open Philanthropy Project brand from the GiveWell brand, including a renaming (the Open Philanthropy Project was known as GiveWell Labs until last August). In the coming year, we plan to launch a more substantial website for the Open Philanthropy Project, continuing the process of separation.

U.S. policy and global catastrophic risks
We covered recent progress in these areas, and compared it to goals set last year, in previous posts (U.S. policy, global catastrophic risks). This section gives a summary of that progress; those who have read the previous two posts may wish to skip it.

Last year, we wrote:

…our top goal for 2014 is a stretch goal (substantial probability we will fail to hit it): making substantial commitments to causes within these two categories. We aren’t sure yet how many causes this will involve; it will depend partly on our ability to find suitable hires. We also haven’t fully formalized the notion of a “substantial commitment to cause X,” but it will likely involve having at least one staff member spending a substantial part of their time on cause X, planning to do so for multiple years, and being ready to commit $5-30 million per year in funding. Given this level of commitment, it is likely that we will not be able to commit to more than 1-3 causes for each broad category (“global catastrophic risks” and “US policy issues” are instances of “broad categories”) in the coming year. Sub-goals of this goal are:

* Completing enough shallow- and medium-depth investigations to feel that we’ve looked near-comprehensively at potential focus causes in these two categories, and writing up our reasons for narrowing the field to a smaller set of “contender causes.”
* Deeply investigating “contender causes” – possibly including some amount of preliminary grantmaking – and prioritizing these “contender causes” relative to each other (and discussing our reasons for such prioritization).
* Recruiting people to focus primarily or exclusively on finding giving opportunities within the causes we select.

We see this as an extremely challenging goal for the coming year, given our current status in these areas. There is no precision to estimating that one year is roughly sufficient, and the project of prioritizing causes in these categories could easily stretch into 2015. With that said, this prioritization is our top priority for 2014, and we think we have a chance to accomplish it. If we do so, we believe that GiveWell Labs [now Open Philanthropy Project] will become a much easier product to understand, discuss and critique, and we will reach the sort of crucial juncture for GiveWell Labs [now Open Philanthropy Project] that we reached for our traditional work around the end of 2009: having concrete recommendations that we can promote and defend, leading to much better engagement with and appeal to donors.

We have not delivered on this goal as initially envisioned. This is mostly because our thinking on how, and how much, to “commit” to causes has evolved. Rather than commit major time and funding up front to a small number of causes, we are going with a longer list of prioritized causes, and we’re looking for a good combination of “high-priority cause” with “strong specific giving and/or hiring opportunity.” The evolution of our thinking on this front is documented in several previous posts: Expert philanthropy vs. broad philanthropy, Thoughts on the Sandler Foundation, and our most recent update on our priorities for U.S. policy causes (similar reasoning applies to our work on global catastrophic risks).

With that said, we’ve done a large number of shallow- and medium-depth cause investigations, and as of the end of January 2015 (a little less than a month behind the schedule implied above), we were transferring the bulk of our energy from these sorts of investigations to seeking out hires and grants in the causes we’ve prioritized.

We haven’t yet made specialized hires, and we feel that the progress we had hoped for on making the Open Philanthropy Project “more concrete” has been partially but not fully realized.

Within the next six months, we hope to make at least one major grant and/or specialized hire in each of (a) U.S. policy and (b) global catastrophic risks; we consider this a “stretch goal” (substantial probability we will fail to hit it). We plan another high-level check-in around 6 months from now.

Scientific research funding and global health and development
Last year, we wrote:

We feel that we are at an earlier stage with two other broad categories of philanthropic causes: scientific research funding and foreign aid. In the case of scientific research funding, we have determined that scientific advisors are crucial, and we have recently recruited several such advisors and started working with them on a trial basis. In the case of foreign aid, despite our history of recommending charities that aid the developing world, we have not developed a strong understanding of how to evaluate a broad cause such as “malaria control” or infrastructure in sub-Saharan Africa from the perspective of flexible, large-scale philanthropy (as opposed to focusing in specifically on delivery of evidence-backed interventions).We hope that at the beginning of 2015, we will be able to say about these two areas what we currently say about global catastrophic risks and US policy: that we have a general sense of the landscape of causes and of how to investigate and evaluate causes, and can aim to make serious commitments to causes in these categories within a year. This is also an ambitious goal, especially in light of its being a secondary priority to the above goal.

We did not deliver on the above goal.

I put substantial time into working with a team of six junior scientific advisors on a couple of fronts:

  • Having general conversations about philanthropic opportunities around scientific research.
  • Investigating the hypothesis that diseases such as malaria and tuberculosis (which disproportionately affect the global poor) are underfunded, and therefore present strong philanthropic opportunities.

I found this work highly challenging. In many cases, I found that the scientists we most wanted to talk to were reluctant to speak at all and/or reluctant to be candid about strong funding opportunities. This contrasts with our experience investigating U.S. policy areas, where many of the people who are best positioned to answer our questions see educating the public as part of their job. In addition, we struggled to find the right way to approach the second investigation listed above: we weren’t able to find people who could give broad, cross-cutting summaries of what philanthropic opportunities might look like, and struggled to think of how to divide the question into smaller questions for optimal efficiency.

Looking at where we stand today, it seems to me that:

  • We are still struggling to identify the right division of labor between full-time employees and scientific advisors when investigating a question of interest. We recently transitioned from working with a team of six advisors on fairly broad questions to working more intensively with individual advisors on investigating opportunities around specific diseases. We expect our thinking on this point to evolve further.
  • I am well behind where I’d like to be in terms of having basic background knowledge about scientific research. Obviously, I will never be an expert in any particular scientific field, so I need to be thoughtful about what a realistic level of background knowledge is, but I would like to improve significantly more before stepping up our efforts to investigate specific causes within scientific research.
  • Between the above two points, we still aren’t yet at the point where we can be working on large numbers of shallow- and medium-depth investigations of scientific research causes.
  • All of the above pertains only to biomedical sciences. We haven’t even begun exploring other scientific fields, including social sciences, energy, and computer science and machine learning.

In light of this situation, we are planning to free up some capacity by postponing Open Philanthropy Project work on global health and development until we’ve made significantly more progress on exploring scientific research. We feel that GiveWell’s top charities represent outstanding giving opportunities for people whose main concern is global health and development – in fact, we are not highly confident that we will find better ones for this particular goal through Open Philanthropy Project. We see improving our understanding of scientific research as the more daunting and pressing goal.

Our main goal for 2015 is to form clear priorities within scientific research funding, comparable to where we currently stand on U.S. policy and global catastrophic risks. This is a stretch goal.

We’ll be writing more about the specifics of our findings and views on biomedical sciences in the coming weeks.

Other progress and plans for Open Philanthropy Project

  • We have recently been prioritizing investigation over public writeups, and our public content is running well behind our private investigations. We are experimenting with different processes for writing up completed investigations – in particular, trying to assign more of the work to more junior staff. If we could do this, it would make a major difference to our capacity, since senior staff already have a substantial challenge keeping up with all of our priority causes. By the end of 2015, we hope that our public content will be no further behind our private investigations than it is at the moment.
  • In 2014, we created the “Open Philanthropy Project” name – replacing GiveWell Labs – and created a preliminary website. In the coming year, we plan to launch a more substantial website for the Open Philanthropy Project, and to look further into the idea of creating a separate organization (which we see as highly likely to happen in the long run).
  • We continued work on our History of Philanthropy project; we put out the first public case study, completed two more that are not yet public, and funded a conference on History of Philanthropy at the Rockefeller Archive Center. We plan to continue work on case studies with our existing consultants, and take new opportunities that come up to make grants and/or bring on new consultants. However, we have no specific goals for the year on this front.
  • We are continuing to network with people at major foundations and people who may become major funders in the future. We do not have specific goals on this front for 2015.
  • We are in the process of considering two grants to major U.S. think tanks trying to build their presence in India and/or China. We feel that we will eventually find it important to be able to work in other countries, particularly countries as important to the future global economy as India and China. However, we expect doing so to be extremely challenging, due largely to the difficulty of forming good relationships and having background context. We see these grants as a small step that will better position us to learn, over time, about the challenges of working overseas. This is not a major priority for 2015.

Open Philanthropy Project update: Global catastrophic risks

(updated on: ) | by Holden

This post lays out our progress, since last year, on identifying potential focus areas for our work on global catastrophic risks.

Summary
Note: this section is similar to the introduction of our previous post on U.S. policy. The overall approach of our work has evolved similarly in the two areas.

Last year, we set a “stretch goal” for the Open Philanthropy Project:

There are two types of causes – global catastrophic risks and US policy issues – that we now feel generally familiar with (particularly with the methods of investigation). We also believe it is important for us to pick some causes for serious commitments (multiple years, substantial funding) as soon as feasible, so that we can start to get experienced with the process of building cause-specific capacity and finding substantial numbers of giving opportunities. As such, our top goal for 2014 is a stretch goal (substantial probability we will fail to hit it): making substantial commitments to causes within these two categories. We aren’t sure yet how many causes this will involve; it will depend partly on our ability to find suitable hires. We also haven’t fully formalized the notion of a “substantial commitment to cause X,” but it will likely involve having at least one staff member spending a substantial part of their time on cause X, planning to do so for multiple years, and being ready to commit $5-30 million per year in funding.

Since then:

  • Our thinking on how, and how much, to “commit” to causes has evolved. Rather than commit major time and funding up front to a small number of causes, we are going with a longer list of prioritized causes, and we’re looking for a good combination of “high-priority cause” with “strong specific giving and/or hiring opportunity.”
  • With that said, we feel that we’ve fulfilled the spirit of the above goal, about a month behind the date we had set. We’ve done a large number of shallow- and medium-depth cause investigations, and we’re now transferring the bulk of our energy from these sorts of investigations to seeking out hires and grants in the causes we’ve prioritized.
  • Our new goal is to be in the late stages of making at least one “big bet” – a major grant ($5+ million) or full-time hire – in the next six months. We think there is a moderate likelihood that we will hit this goal; if we do not, we will narrow our focus to a smaller number of causes in order to raise our odds.
  • Our highest priority is to make a full-time hire for working on biosecurity. As a second priority, we are spending significant time on various aspects of geoengineering, geomagnetic storms, risks from artificial intelligence, and some issues that cut across different global catastrophic risks. A more extensive summary of our priorities and reasoning is available as a Google sheet.
  • We have recently been prioritizing investigation over public writeups, and there are many shallow- and medium-depth investigations we have completed but not written up. We are experimenting with different processes for writing up completed investigations – in particular, trying to assign more of the work to more junior staff – so our public writeups could remain behind our private investigations for much of the next few months.

Below, we go into more detail on:

Progress since our June update
Biosecurity. We put significantly more work into understanding the fairly large and complex biosecurity space, which includes efforts to prevent or mitigate the harm from natural pandemics, accidental release of dangerous natural pathogens, currently existing biological weapons, and accidental or purposeful release of synthetic pathogens in the future. We believe there are significant philanthropic opportunities here. We are currently strongly considering one grant and may consider others, though we believe this space is complex enough that the best way to approach it would be with specialized staff.

Artificial intelligence. We began an investigation of risks from potential unintended consequences of advances in artificial intelligence. We hoped to hear the perspectives of mainstream computer scientists, artificial intelligence experts, and machine learning experts regarding arguments like those advanced in the recent book Superintelligence. We temporarily paused this investigation on learning that the Future of Life Institute was planning a conference on this topic; Howie Lempel and Jacob Steinhardt attended the conference on our behalf. We see the conference as a major update:

  • An open letter following the conference makes it fairly clear, to us, that a wide variety of people with relevant expertise see artificial intelligence as a technology whose potential for great benefits may come along with real risks on which meaningful preparatory research can and should be done.
  • This was followed by a $10 million commitment from Elon Musk to fund such research.
  • We see this cause as highly important and worthy of investment. It remains unclear to us how to think about its “crowdedness,” and we plan to coordinate closely with the Future of Life Institute to follow what gets funded and what gaps remain.

Geoengineering. We continued to investigate the cause of governance of and research into geoengineering, and are currently strongly considering a grant in this space.

Geomagnetic storms. We began an investigation (by consultant David Roodman, who previously investigated labor mobility and the mortality-fertility connection) into the conflicting claims we’ve seen about the threat posed by geomagnetic storms. This investigation is still in progress. Depending on its outcome, we may become interested in funding research into electrical grid robustness.

Other risks. We looked further into philanthropic possibilities for reducing risks from nuclear weapons, completed a shallow investigation on risks from atomically precise manufacturing, and did a small amount of investigation on general food security (a cross-cutting issue, since several different global catastrophic risks could disrupt global agriculture).

We have not yet made the results of any of the above investigations public, though we plan to. As mentioned early in this post, we have been prioritizing investigation over public writeups, and we are experimenting with different processes for writing up completed investigations – in particular, trying to assign more of the work to more junior staff.

As with U.S. policy, we have noted significant variation in the extent to which different issues are suitable for specialized staff. We feel that biosecurity would be best handled by specialized staff. The other areas we’re considering – with the possible exception of geoengineering – seem better suited to a “broad” model in which we scan multiple areas at once, looking for the most outstanding grant opportunities.

Plans
While there are more cause investigations we could do, at this point we think it’s appropriate to shift our priorities in the direction of granting out significant funds in the causes we’ve already identified as promising. At the same time, we’re trying to give ourselves the flexibility to look across multiple possible causes, and only make a “big bet” (a full-time hire or major grant) where we feel the opportunity is outstanding. As such, we’ve created a relatively long prioritized list of causes, with goals for each, and our six-month goal is to be in the late stages of making a “big bet” in at least one area. We may continue to make smaller grants, with relatively light investigation, when we see reasonably strong opportunities, but this is not our main goal.

We’ve ranked biosecurity as our top priority, for the following reasons.

Suitability for a full-time hire. Biosecurity stands out along several dimensions that make it an appealing but also particularly complex target for philanthropy:

  • Governments spend a large amount on biosecurity preparations but many opportunities to improve preparedness remain and there is little philanthropic spending in the field. This suggests an opportunity for philanthropy to leverage public money but also increases the complexity of the cause.
  • Some interventions may increase our preparedness for both near-term risks from natural pandemics and larger, longer-term risks related to the misuse or abuse of emerging synthetic biology technology. Efforts to reduce long-run risks may be more sustainable if they simultaneously produce verifiably short-run benefits but also risk losing sight of their long-run mission. Comparing the expected impact of interventions focused on different time horizons also presents a challenge and is one reason that hiring a specialist may be particularly valuable.
  • Biosecurity presents opportunities to intervene in many venues. Preparations include global, regional, national, and local components and a biosecurity strategy may target or fund governments, intergovernmental organizations, NGOs, for-profits, or other entities.

Overall, we feel biosecurity is the best-suited (of the causes we have ranked relatively highly) to a specialized hire, and hiring is a top priority of ours.

Importance, tractability, crowdedness. We see this area as the most threatening risk on the list, with the possible exception of artificial intelligence, in terms of probability of a massive global disruption to civilization, and we are fairly convinced that there are real opportunities to improve preparedness.

We find it difficult to predict whether the additional attention brought to the cause by the Ebola outbreak in West Africa will lead to major changes in available funding. We plan to monitor this situation and expect the most important effects to be on the relative crowdedness of different interventions within the cause. Our current view is that it would be a surprise if most of the promising opportunities to increase preparation were funded by other actors in the near future.

Our next few priorities are a set of risks that we see as (a) posing substantial threats of massive global disruptions to civilization in the next century; (b) presenting a strong possibility of useful, not-already-funded preparatory work in the near future; (c) not being a good fit for extremely intensive or full-time investigation at this time, either because we have some key open questions remaining or because we aren’t aware of a large enough space of giving opportunities. Specifically:

  • We believe geoengineering research and governance is a promising philanthropic space. Because it is a relatively thin space (not many researchers or organizations currently devoted to it), a specialized hire in this area may need to very actively field-build and generate interest from potential grantees who are not currently seeking (additional) funding to work on geoengineering; we remain uncertain of how wise or efficient such a strategy would be at this time. We might make a specialized hire if we found an outstanding fit, but might also simply continue to monitor the space and capitalize on giving opportunities that arise.
  • We believe that research on risks of unintended consequences from the development of artificial intelligence is a promising philanthropic space. Here again, the field is relatively thin; in addition, we are unsure what sorts of giving opportunities will remain in the wake of Elon Musk’s $10 million commitment. We are monitoring this space and communicating closely with the Future of Life Institute.
  • We plan to finish our investigation of risks from geomagnetic storms, after which point we might pursue the idea of funding research on electrical grid robustness. We don’t think we would fund other work in this area before learning more about the amount of damage that could be done by a severe storm.
  • Now that we have formed a broad view of the most threatening global catastrophic risks, we are interested in giving opportunities that could “cut across risks,” addressing multiple risks at once – for example, improving food security (which we have looked into a bit; we have preliminarily have found a lack of consensus on promising projects), forecasting future risks, researching ways to increase society’s general resilience to shocks, or improving general mechanisms for governance of emerging technologies. We are currently assessing some such opportunities and will continue to be open to more.

Below these priorities, we list risks where (a) a massive global disruption to civilization is highly unlikely to occur in the next century; or (b) we have found less useful preparatory work that is not already being done.

An additional goal for the next several months is to write up the more recent work we’ve done, most of which is not yet public.

Public summary of our global catastrophic risk priorities

Open Philanthropy Project update: U.S. policy

(updated on: ) | by Holden

Last year, we set a “stretch goal” for the Open Philanthropy Project:

There are two types of causes – global catastrophic risks and US policy issues – that we now feel generally familiar with (particularly with the methods of investigation). We also believe it is important for us to pick some causes for serious commitments (multiple years, substantial funding) as soon as feasible, so that we can start to get experienced with the process of building cause-specific capacity and finding substantial numbers of giving opportunities. As such, our top goal for 2014 is a stretch goal (substantial probability we will fail to hit it): making substantial commitments to causes within these two categories. We aren’t sure yet how many causes this will involve; it will depend partly on our ability to find suitable hires. We also haven’t fully formalized the notion of a “substantial commitment to cause X,” but it will likely involve having at least one staff member spending a substantial part of their time on cause X, planning to do so for multiple years, and being ready to commit $5-30 million per year in funding.

This post is an update on our plans for U.S. policy; a future post will discuss global catastrophic risks.

In brief:

  • Our thinking on how, and how much, to “commit” to causes has evolved. Rather than commit major time and funding up front to a small number of causes, we are going with a longer list of prioritized causes, and we’re looking for a good combination of “high-priority cause” with “strong specific giving and/or hiring opportunity.”
  • With that said, we feel that we’ve fulfilled the spirit of the above goal, about a month behind the date we had set. We’ve done a large number of shallow- and medium-depth cause investigations, and we’re now transferring the bulk of our energy from these sorts of investigations to seeking out hires and grants in the causes we’ve prioritized.
  • Our new goal is to be in the late stages of making at least one “big bet” – a major grant ($5+ million) or full-time hire – in the next six months. We think there is a moderate likelihood that we will hit this goal; if we do not, we will narrow our focus to a smaller number of causes in order to raise our odds.
  • Our highest priority is to make a full-time hire on criminal justice reform, factory farming (pending a last bit of cause investigation, focused on the prospects for research on meat alternatives), or macroeconomic policy. Our second-highest priority is to further explore international labor mobility and land use reform, with an eye to either finding more giving opportunities (hopefully including at least one major one) or to developing a full-time job description. A more extensive summary of our priorities is available as a Google sheet.
  • We have recently been prioritizing investigation over public writeups, and there are many shallow- and medium-depth investigations we have completed but not written up. We are experimenting with different processes for writing up completed investigations – in particular, trying to assign more of the work to more junior staff – so our public writeups could remain behind our private investigations for much of the next few months.

Below, we go into more detail on:

  • Our progress since our May update. More
  • How our thinking and approach have evolved. More
  • Our plans from here. More

Progress since our May update
We’ve done internal shallow- and medium-level investigations – most of them not yet written up – on causes including:

  • Land use reform (e.g., zoning regulations), which we perceive as a cause with moderate importance and extremely little public-interest advocacy infrastructure (a green field).
  • Alcohol policy, which we have a similar assessment of, though our assessment of importance is still ongoing.
  • Aspects of intellectual property reform that go beyond software patent reform. We believe there are multiple interesting issues here, but nothing outstanding enough to be at the top of our priority list.
  • A broad set of issues around enhancing welfare by enhancing income security for relatively low-income people in the U.S. These include regulations around minimum wages, overtime, family and medical leave, the “minimum basic income” concept, etc. We initially expected this set of “traditional” issues to be too crowded and politically polarized to interest us, but having investigated more, we now believe there are a fair number of opportunities for funders to have impact at the state and local level, and that there isn’t enough funding to take advantage of all such opportunities.
  • The health care policy space. Our initial assessment is that this area seems very important but also quite crowded. We’re hoping to eventually investigate some sub-spaces that may be less crowded.

We’ve also put significantly more time into exploring causes that were already on our radar:

  • Criminal justice reform. We conducted a several-month, in-depth investigation of opportunities in criminal justice reform, a complex space and one that we see as high-priority due to the window of opportunity. The main findings from this investigation are available within this document (DOCX).
  • Macroeconomic policy. We came across what we considered a fairly outstanding giving opportunity in this area, the CPD Fed Up campaign, and invested a good deal of time in background research for this grant. (We put particular energy into understanding the complex debates behind the appropriate course of monetary policy.) We have provided $850,000.00 in support to this campaign to date.
  • Factory farming. Eliza Scheffler has conducted a medium-depth investigation of this cause; a writeup is forthcoming. We see this as a promising cause, and it is on our list to consider hiring in, pending a last bit of cause investigation, focused on the prospects for research on meat alternatives
  • Labor mobility. We have continued to work with Michael Clemens on finding and following giving opportunities in this space, and have been particularly following our grant to support IOM Haiti’s work facilitating seasonal migration.
  • Marijuana policy. We had been helping Good Ventures to navigate this cause as general support, but recently we’ve come to see an aspect of it – emphasizing optimal regulation rather than simply legalization – as a potential fit for the Open Philanthropy Project. Importance seems fairly low compared to most causes we are considering, but there is a very strong window of opportunity, and essentially no other funding focused on optimal regulation. We are still working on our estimate of the potential importance of this work, and expect to decide soon how to prioritize this cause.

Finally, we’ve done some cross-cutting work including:

How our thinking and approach have evolved
Through this work, our overall thinking on U.S. policy has shifted in several ways. Most importantly, we see a great deal of variation in how much different causes demand specialized, full-time, hires.

  • When looking at the criminal justice reform space, we see a very wide array of organizations that we could potentially support. In addition, there is room to have substantial impact in a broad variety of venues, from state-level policy changes (examples) to local-level experiments (example). We think that the “expert philanthropy model” would have major advantages over a lower-intensity approach in this space. We feel similarly about macroeconomic policy and factory farming, though in the case of macroeconomic policy the argument for “expert philanthropy” stems more from the complexity of the subject matter than the profusion of venues or potential grantees.
  • By contrast, causes like international labor mobility and land use reform present a different kind of challenge: there are very few people and organizations aligned with our priorities in these areas. Because of this, we can’t currently envision a full-time employee worth of high-value work to do in these areas. We also expect that the limited size of the fields would make it unusually difficult to find someone appropriate to work on these issues full time. Our current plan is to devote relatively little staff time to these causes beyond the level required to stay reasonably well-networked, while being explicitly open to potential grant opportunities and being prepared to devote significantly more staff time if a major grant opportunity arises.
  • Many causes occupy a middle ground. In many cases, it would not be unreasonable to take an “expert” approach to a cause, but we also feel we might maximize efficiency by understanding the cause at a high level and focusing on a small number of potentially outstanding giving opportunities. Examples in this range include foreign aid policy and tax policy. The decision to hold off on hiring experts in these areas is currently driven by a need to prioritize the use of our existing staff resources, rather than a view that it would be fundamentally mistaken to hire specialized staff in these areas.

Another general observation from working on U.S. policy is that we believe that the mix of policy priorities that is coming into focus for us does not map neatly onto progressivism, libertarianism, conservatism, or any other platform currently common in the U.S. policy world. This topic was a prominent theme in our recent day-long convening noted above.

This observation has a couple of implications. In the short run, we expect to have more difficulty than most funders with finding organizations that share our positions across a broad range of policy areas; we expect that we will find organizations/partners that share some but not all of our perspectives. For example, the organizations that share our views on land use policy may have a different take on income security. In the long run, we would like to strengthen and support the network of people who broadly share our policy priorities. Both of these are reasons to pursue a relatively broad philanthropic approach, keeping up on a large number of causes rather than specializing in a small number.

Plans
While there are many more cause investigations we could do, at this point we think it’s appropriate to shift our priorities in the direction of granting out significant funds in the causes we’ve already identified as promising. At the same time, we’re trying to give ourselves the flexibility to look across multiple possible causes, and only make a “big bet” (a full-time hire or major grant) where we feel the specific opportunity is outstanding. As such, we’ve created a relatively long prioritized list of causes, with goals for each, and our six-month goal is to be in the late stages of making a “big bet” in at least one area. We may continue to make smaller grants, with relatively light investigation, when we see reasonably strong opportunities, but this is not our main goal.

In order to determine which causes are most outstanding, we’re still putting substantial weight on the framework we outlined last May: looking for causes that (a) stand out on one of [importance, tractability or uncrowdedness], while (b) being competitive with other causes on the other two dimensions.

In addition, we’ve started separating issue areas by “need for a specialist,” as discussed above. On issues that demand a specialist, our main goal is to make a full-time hire; on others, our main goal is to wait and see what kinds of opportunities might be available (and potentially have existing generalist staff explore and make a grant if we find a credible opportunity). A third important category of cause is “shovel-ready” causes: causes where we have an unusually concrete sense of what (and in some cases whom) we could fund and the next step is largely deciding whether to do so.

A final note on our ranking of causes is that we’ve started more explicitly thinking of “high-venue” issues (issues where policy at the state and local level, not just the federal level, is relevant) as being both more complex and presenting more possible paths to impact that we can choose between. For this reason, we now list the “venues” for a cause separately in our spreadsheet.

Our highest priority is to hire and increase capacity in areas where we feel prepared to, and accordingly our current policy priorities list puts the causes that we’d be interested in hiring in at the top; they are followed by the strongest issues where we don’t have a good sense of what kinds of additional opportunities we could support, and those in turn are followed by our strongest “shovel-ready” issues. Our highest priority is to make a full-time hire for criminal justice reform, factory farming (pending a last bit of cause investigation, focused on the prospects for research on meat alternatives), or macroeconomic policy. Our second-highest priority is to further explore international labor mobility and land use reform, areas that we find conceptually very promising but in which we aren’t currently aware of (multiple promising-seeming) potential grant opportunities, and accordingly aren’t ready to make full-time hires in. These priorities are followed by several issues on which we have a relatively specific idea of what we could fund, and the next steps would be to investigate in much greater depth to decide whether the specific potential grants were worth making.

An additional goal for the next several months is to write up the more recent work we’ve done, most of which is not yet public. We are experimenting with different processes for writing up completed investigations – in particular, trying to assign more of the work to more junior staff.

Public summary of our U.S. policy priorities

2015 plan for GiveWell’s traditional (“top charities”) work

(updated on: ) | by Josh Rosenberg

This is the third post (of six) we’re planning to make focused on our self-evaluation and future plans. The goal of this post is to update GiveWell’s followers on our plans for our traditional work in 2015 and to establish a general set of goals by which we can evaluate ourselves at the beginning of 2016.

As discussed in our previous post, in 2014, GiveWell’s traditional (“top charities”) work conducted a large amount of research while maintaining research quality and building substantially more capacity to conduct research in the future. The amount of money moved to our recommended charities continued to grow; we moved about $28 million to recommended charities in 2014.

This year, our primary goals are to:

  • Build management and research capacity for GiveWell’s traditional work while further reducing senior staff time (note 1) spent on this work, primarily by reallocating Elie Hassenfeld’s management responsibilities related to GiveWell’s traditional work.
  • Maintain our core research product by completing updates on all eight 2014 recommended charities and determining which of them should be recommended as top charities for the 2015 giving season.

Our secondary goals for 2015 are to:

  • Continue to seek outstanding giving opportunities by reviewing 2-4 new charities and publishing 2-4 new intervention reports.
  • Improve the cost-effectiveness analyses and room for more funding analyses in charity reviews.
  • Finish and launch a redesigned GiveWell website.
  • Make further progress on experimental work to “seed” potential recommended charities.

We expect our total output on “top charities” work to be roughly comparable to last year’s, despite a growing staff, because (a) a major focus of the coming year is training, and we expect to trade some short-term efficiency for long-run output; (b) we may be reallocating some capacity from our “top charities” work to the Open Philanthropy Project this year.

More details on some of these goals are below.

Building capacity

In 2015, we hope to build substantially more management and research capacity for GiveWell’s traditional work in order to move toward our goal of having a sustainable organization that is not dependent on past senior staff. As we have discussed before, building capacity is challenging and generally leads to reductions in capacity in the short term. This year, we plan to build capacity by:

  • Training relatively senior staff to take on management roles by reallocating Elie Hassenfeld’s management responsibilities to them as much as possible. For example, Senior Research Analyst Natalie Crispin is currently performing all of Elie’s management responsibilities with respect to GiveWell’s 2015 charity reviews and charity updates. Elie is overseeing Natalie during this transition. Since Natalie is managing others on this work, she does not have as much time to directly do research work herself.
  • Training relatively junior staff to do most charity updates, intervention reports, and new charity reviews. Most junior staff members are relatively new to this type of work.
  • Continuing to hire and train new Research Analysts, Outreach Associates, and Conversation Notes Writers.

We expect that these efforts to build capacity will enable us to do more research – for both GiveWell and the Open Philanthropy Project – in the long run but will reduce the efficiency of our work in the short run, requiring more person-hours per unit of output than in 2014.

Building capacity to do intervention reports

As we wrote in our 2014 self-evaluation post, completing new intervention reports in 2014 was much more difficult and time-consuming than we had anticipated. This year, we are trying to build more capacity for completing these reports by training more staff to do intervention-related research and by improving our process for doing this research. Our ultimate goal is to have a process for completing a reasonable number of intervention reports that does not require substantial involvement from Elie.

We consider building capacity to do more intervention reports to be a high priority because we must be able to complete these reports in order to best prioritize new charities for investigation.

Charity updates

We plan to publish updates on all eight of our recommended charities in 2015. We are generally aiming to have conversations with each charity in February, May, and September that will each result in conversation notes and/or an update report (example). This update schedule may vary somewhat by charity. We are following this charity update schedule so that a) we learn about any major updates that might cause changes in a charity’s recommendation status as soon as possible and b) we spread out the work of refreshing our charity reviews over the course of the year.

As part of our February update calls with recommended charities (especially top charities), we will be gathering more information about charities’ room for more funding situations. As we mentioned in December 2014, we may update our recommended allocation to top charities to reflect any major changes in charities’ funding needs. We tentatively plan to publish such an update in April.

In our charity updates, in addition to our standard questions to follow up on each charity’s activities, we will be focusing in particular on whether any new information from our “standout” charities might lead them to be recommended as “top” charities by the end of the year.

Intervention reports

This year, we hope to complete 2-4 new intervention reports. The programs and program areas that we have preliminarily prioritized for investigation include:

  • Nutrition programs (e.g., folic acid fortification and iron fortification)
  • Immunization programs (e.g., immunization against measles and meningitis)
  • Neglected tropical diseases programs (e.g., trachoma and onchocerciasis)
  • Programs for which we believe a charity would apply if we determined the intervention to be a priority program (e.g., “Targeting the Ultra-Poor” (or “Ultra-Poor Graduation”) programs and voluntary medical male circumcision for the prevention of HIV)

We also plan to publish two intervention reports that are near completion: maternal and neonatal tetanus elimination and mass drug administration to eliminate lymphatic filariasis.

A major factor in prioritizing among intervention reports is determining which interventions seem to be most broadly similar to our other priority programs. We believe that such interventions are most likely to succeed in our current process. Factors that seem to be common among our priority programs include:

  • The program has strong evidence of effectiveness (preferably from multiple high-quality studies)
  • The program is very low-cost per person reached
  • Studies of the program’s effects seem not to be overly dependent on the particular context in which the program was implemented (e.g., studies of health programs often seem more likely to be externally valid than studies of education programs because the mechanisms by which health programs have their effects are often more consistent across populations)
  • The program is highly replicable for an implementing organization (i.e., a charity would face a low burden of proof to show that they were carrying out the same intervention that was studied and shown to be effective)
  • The program has informative proximate outcomes (e.g., deworming pills taken, bed nets delivered and used, etc.) that can be fairly easily measured and monitored

New charity reviews

We plan to actively pursue evaluations of 2-4 new potential top charities this year. Our tentative plans for which charities we may evaluate are below. However, note that there are many reasons that new charity review prioritization could change during the year, such as learning new information from potential recommended charities and completing new intervention reports that change our views on which interventions are promising (several of the new charities that we may evaluate implement interventions for which we have not yet published intervention reports). We also plan to maintain our “open-door policy” for allowing any charity to apply for a recommendation.

As with previous years, we chose to mainly prioritize charities based on our best guess about whether they will become top-rated organizations. We also gave additional weight to organizations that we guessed have some chance of being substantially more cost-effective than our current top charities. The charities that we may evaluate include:

We have also reached out again to Nothing But Nets about applying for a recommendation because it distributes long-lasting insecticide-treated bed nets, which we consider to be one of the most cost-effective priority programs.

Experimental work

In 2015, we (in collaboration with Good Ventures) plan to continue the experimental work to “seed” additional top charities that we began in 2014, though we still do not consider this work to be a high priority. A few activities that we are considering in this area include:

  • Providing funding to promising young charities, such as New Incentives, that could eventually become recommended charities.
  • Funding additional research on and support for scale up for programs that a) could be priority programs if they were supported by additional studies and b) could be scaled up with additional support. The main way in which we are currently doing this is our funding of Evidence Action (early conversation, recent conversation, grant page). We have also investigated other potential partners and may take one or more on in the future.
  • Funding additional independent monitoring that could increase our confidence in the success of recommended charities’ programs (e.g., deworming, salt iodization) or increase our confidence in other organizations’ ability to carry out priority programs (e.g., if we learned that standard government-led bed net distributions were high-quality, then we might recommend additional funding to large non-profits that fund government bed net distributions). We are initially planning to work with IDinsight on this project.

We plan to explore options and publish updates on our progress on this work throughout 2015.

Improving quality of charity reviews

In general, we feel that our charity reviews are high-quality. However, we believe that there are some ways in which they could be improved. In particular, in 2015, we hope to improve the cost-effectiveness analyses (CEs) (example) and “room for more funding” analyses (example) in our charity reviews, if we have the capacity to do so.

We feel that the quality of our CEs has been acceptable in the past, but we have identified tangible ways to improve them and feel that it is worth using some of our additional capacity to do this because such analyses are relatively important to our charity recommendations. In particular, we aim to:

  • Generally improve the transparency and clarity of our CEs.
  • Think more carefully about the major inputs that cause a substantial amount of variation in our CEs and ensure that we know as much as we can about those inputs. For example, the proportion of deworming pills that were given to children as part of Schistosomiasis Control Initiative (SCI)‘s campaigns is a relatively important parameter in our CE for SCI, but we did not have as much confidence in our understanding of this parameter as we could have at the end of last year.
  • Ensure that we properly account for “leverage” considerations when appropriate (e.g., in our CEs for organizations such as Iodine Global Network and Deworm the World Initiative).
  • Ensure that we are applying rules for including and excluding costs and benefits consistently across all CEs (e.g., making sure that we have captured all in-kind donations to all of our charities’ programs in our CEs).

Similarly, we would like to improve the “room for more funding” sections of our charity reviews. To achieve this, we plan to:

  • Discuss charities’ funding needs with them earlier in the year so that we can gain as much clarity as possible about their funding situations.
  • Standardize the questions that we ask charities about their room for more funding so that we can be more confident that we are making similar comparisons across organizations.

How large is the pipeline of potential top charities and priority programs?
We are building a charity review organization with a substantial amount of capacity for conducting research, but it is unclear how many charities that we have not yet reviewed may be competitive with our current top charities.

One factor that affects our estimate of the size of the pipeline is that, over time, we have broadened our criteria and research process so that we are able to evaluate more types of potentially high-impact giving opportunities. For example, in the last two years we have begun to evaluate organizations that play an advocacy and advisory role to governments, such as Deworm the World Initiative and Iodine Global Network. We are also now open to evaluating components of “mega-charities” that are working to scale up potential priority programs, such as UNICEF Maternal and Neonatal Tetanus Elimination Initiative.

We have also seen that charities have been more willing to engage in our review process over time. This may be due to our growing money moved, growing influence, and generally improved incentives for charities to apply (discussed in our 2014 self-evaluation post).

These changes have had the effect of increasing the pipeline of potential charities and programs to evaluate and potentially allowing us to find more cost-effective giving opportunities than we had been able to find previously.

We are also trying to increase the pipeline of outstanding giving opportunities through our experimental work mentioned above (e.g., by funding New Incentives and replications of studies).

Currently, we do not have a strong sense of the overall number of potential top charities and potential priority programs that we have not yet evaluated, but we feel that there are substantially more promising charities and programs than we will be able to evaluate this year and believe that it is possible that the pipeline will grow over time.

Note 1: In this post, senior staff refers to Elie, Holden, and Alexander. Many staff took on additional responsibilities throughout 2014, so this refers to senior staff as of January 2014, not as of today.

2014 progress on GiveWell’s traditional (“top charities”) work

(updated on: ) | by Josh Rosenberg

This is the second post (of six) we’re planning to make focused on our self-evaluation and future plans.

This post reviews and evaluates last year’s progress on our traditional work of finding and recommending evidence-based, thoroughly vetted charities that serve the global poor. It has two parts. First, we look back at the plans we laid out in early 2014 and compare our progress against them, providing details on some of the most significant accomplishments and shortcomings of the year. Then, we reflect on the quality of our traditional work and critically evaluate some of our major strategic decisions. In our next post in this series, we will cover our plans for GiveWell’s traditional work in 2015.

Summary

Overall, we feel that 2014 was an excellent year for GiveWell’s traditional work.

At the beginning of 2014, we laid out our most ambitious research goals yet, including publishing updates on all recommended charities, reviewing several new charities, and completing new intervention reports. We expected to be able to complete a higher volume of work than ever before while also reducing senior staff time (note 1) devoted to GiveWell’s traditional work by continuing to hire, train, and develop non-senior staff. We feel that we broadly met those goals while maintaining the overall quality of our research.

The impact of GiveWell’s traditional work continues to steadily increase, as we moved about $28 million to recommended charities in 2014 (more details forthcoming in the final post of this series, which focuses on our metrics).

We believe that our 2014 recommended charities list was high quality. A notable development was that we included four new “standout” organizations on our recommended charities list. We believe that some of these organizations may become top charities in the future.

A few areas in which we fell short in 2014 were:

  • Publishing new intervention reports. Completing intervention reports was more difficult than we had expected at the beginning of 2014; we hope to improve on our process for doing these reports in 2015.
  • We finalized the details of our top charity recommendations in late November 2014, after we had already made a recommendation to Good Ventures about how to allocate its giving among the recommended charities. If we had completed our analysis earlier, we may have recommended a different allocation to Good Ventures. We see the harm here as minimal since we ultimately adjusted our public targets to account for grants from Good Ventures; however, in the future we should try to avoid a recurrence of this issue, perhaps by announcing our recommendations to Good Ventures and the public at the same time.
  • We did not follow the ideal process for ensuring that our cost-effectiveness analyses were robust, accurate, and easily understandable, which led us to finalize these analyses very late in the year.

Our progress in 2014 relative to our plans
Our “2014 plan” blog post laid out several main goals for the year:

  • Continue to build capacity for conducting “top charities”-related research work. Reduce senior staff time devoted to this work by training other staff to take over senior staff’s responsibilities
  • Publish updates on previously recommended charities
  • Conduct reviews for several new potential recommended charities
  • Maintain our “open-door policy” for allowing charities to apply for a recommendation
  • Publish four intervention reports that were near completion (maternal and neonatal tetanus elimination, salt iodization, Vitamin A supplementation, and polio)
  • Publish 5-10 new intervention reports on nutrition programs, behavior change programs, and other programs
  • Fund experimental work that may lead to more recommended charities in the future (e.g., providing early funding to promising charities such as New Incentives or funding replications for promising interventions)
  • Conduct other miscellaneous research (e.g., produce a cost-effectiveness estimate for Dispensers for Safe Water (DSW), review the midline of Development Media International (DMI)‘s randomized controlled trial (RCT), consider evaluating a mega-charity, etc.)

We feel that we broadly achieved these goals in 2014. In summary, we:

One area in which we fell short of our expectations was publishing new intervention reports, largely because completing these reports was more difficult than we had anticipated.

More details on some of our major achievements and shortcomings are below.

Capacity building

GiveWell’s traditional work produced more total research ‘output’ in 2014 than in previous years while also using less senior staff capacity than in previous years. A rough measure of total research output is the number of charity updates, charity reviews, intervention reports, and other major research work that we did during the year. In 2014, we completed four charity updates, four new charity reviews, two intervention reports, and some work on “seeding” new top charities. For comparison, in 2013, we completed three charity updates (two of which (AMF and SCI) required a significant amount of senior staff time because they were substantial updates), one new charity review (Deworm the World Initiative), and one intervention report (water quality). (More details on the work we did in 2013 are in our 2013 self-evaluation.) We consider the increase in research output in 2014 to be a major achievement.

Non-senior staff continued to be trained to take on additional responsibilities, and our staff continued to steadily expand. Examples of greater responsibilities shared by non-senior staff and reductions in senior staff time spent on GiveWell’s traditional work include:

  • All four new charity reviews (DMI, IGN, GAIN, and Living Goods) were led by non-senior staff. Our first new charity review led by non-senior staff was the DtWI review in 2013. We would not have had the capacity to do four new charity reviews in a year if not for our expanded non-senior staff capacity.
  • Holden and Alexander spent very little time on traditional work. In particular, Holden substantially reduced the amount of time that he spent writing blog posts for GiveWell’s traditional work. Elie continued to spend most of his time on traditional work but also passed off some of his responsibilities to other staff.
  • Non-senior staff took on increased management responsibility. For example, Natalie Crispin managed other staff on our updated review of SCI, our new review of Living Goods, and other work. Other staff supported with management of new Research Analysts, Summer Research Analysts, and Conversation Notes Writers.
  • All three site visits to recommended charities were conducted without senior staff.
  • All intervention report work was primarily conducted by Jake Marcus.

We see reducing senior staff time spent on GiveWell’s traditional work as a major success because a) making the organization less dependent on a few individuals improves the sustainability of the organization and b) we have historically primarily been constrained by senior staff capacity, so freeing up senior staff capacity should enable us to make progress on goals such as the Open Philanthropy Project.

We have also substantially improved our capacity by hiring and training Conversation Notes Writers. GiveWell has published about 150 conversation notes per year for the last two years (see them on our conversations page). In 2013 and early 2014, Research Analysts spent a substantial amount of their time writing conversation notes. In 2014, we hired Conversation Notes Writers to handle this responsibility. We now have 8 Conversation Notes Writers, and Research Analysts generally spend very little of their time on conversation notes.

Finally, although increased capacity has already allowed us to accomplish more than we had previously, we believe that many of the largest benefits will come in the future. Staff has consistently contributed significantly more as their tenure at GiveWell has grown. We currently have only five staff members who have been at GiveWell for more than two years.

New charity reviews

During 2014, we completed new reviews for four charities (DMI, IGN, GAIN, and Living Goods) that we ultimately recommended as “standout” organizations. (note 2)

We feel that this was a major success of our research work for the year. We believe that adding these “standout” charities to our list of recommended charities was valuable because (roughly in order of importance):

  1. These organizations seem to be very promising giving opportunities; some of them may become top charities in the future.
  2. If our money moved continues to grow, it will be important to have as much “room for more money moved” as possible. Even if current standout charities never become as strong (in isolation) as our current top charities, they may become the best options available when room for more funding is taken into account.
  3. The “standout” charities represent the organizations that we felt, on preliminary review, had the best chance of being significantly stronger giving opportunities than our current top charities. This time around, further review concluded that they were not as strong, but we feel it is important to continue engaging in these sorts of investigations and evaluating the best possible challenges to our current list.
  4. The “standout” designation and associated changes to our review process improves the incentives for potentially promising charities to apply for a GiveWell recommendation, which makes us more likely to be able to find the best giving opportunities. In particular, in 2014 we provided participation grants to promising charities that allowed us to review them publicly, directed some funding to the “standout” organizations by adding them to our list of recommended charities, and conferred some status on these organizations by giving them a GiveWell recommendation. These factors improve the cost-benefit analysis for a charity considering applying for a GiveWell recommendation, which we hope leads to more promising charities applying over time. Consistent with this, we saw increased interest from charities in engaging in our process in 2014 and expect this to continue as our money moved and influence grows.
  5. Adding more charities to our recommended list provides donors with more options. If donors have different values from us or different fundamental beliefs about which types of organizations are likely to be most effective, then we could be providing a valuable service by doing research on a wider set of donation options.


Intervention reports

We published fewer intervention reports than we had hoped to at the beginning of 2014. We completed intervention reports for salt iodization and vitamin A supplementation, but we have not yet published the other two reports that we had said were near completion at the beginning of 2014 (polio and maternal and neonatal tetanus elimination) and did not publish any new reports, though we said last year that we had hoped to publish 5-10 new reports. That said, our goal of publishing 5-10 new intervention reports was arbitrary and, upon further reflection, unrealistic given the amount of time that it has typically taken us to complete intervention reports in the past.

We did not accomplish as much as we expected on this front primarily because completing these reports was much more difficult and time-consuming than we had anticipated. As of the beginning of the year, we had only completed 3 intervention reports that match our current standards of thoroughness, and senior staff had led the completion of each such report. This year, we tried to complete intervention reports with far less involvement from senior staff, and this proved challenging. There are an essentially unlimited number of questions we could ask about a given intervention, and making the right decisions about which to focus on (and at what level of thoroughness) is key; with less involvement from senior staff, it was more difficult to ensure that time spent investigating and writing up questions was allocated to the right questions at the right level of detail. In particular, we had cases in which an intervention report appeared close to completion, but late-stage reviews and peer feedback added many more questions.

Improving our process for doing intervention reports is one of our primary goals for 2015 (more on our goals in a forthcoming blog post). Additionally, the main staff member who worked on intervention reports (Jake Marcus) also worked on other evidence reviews, such as reviewing a new, promising study on deworming, an early, unpublished draft of the Living Goods study, and the midline of the DMI study. He also spent some of his time investigating donating to the Ebola response as a giving opportunity.


Other shortcomings

Late completion of top charity recommendations

We finalized the details of our top charity recommendations later in the year than would have been ideal. In late November, we were still clarifying facts and debating some key issues related to our recommendations, such as SCI’s room for more funding and estimated cost-effectiveness and AMF’s room for more funding.

This is problematic because we made our recommendation to Good Ventures about how to allocate its giving in mid-November. We had agreed with Good Ventures that it should aim to announce its giving plans at the same time that we released our recommendations to the public in order to avoid potential fungibility concerns. To meet this deadline, we sought to finalize our recommendation to Good Ventures a couple of weeks before our public recommendations were released.

If we had fully completed our analysis before making a recommendation to Good Ventures, we likely would have recommended relatively more to AMF and relatively less to GiveDirectly. (For more details on how Good Ventures allocated its giving and our recommended allocation to donors, see our 2014 recommendations announcement post.)

In the end, we adjusted the public targets we announced based on the grants Good Ventures had committed to, so we don’t see a major issue here. However, in the future we should try to avoid a recurrence of this issue.

In the past, we have tried more than once to finalize our recommendations well in advance of giving season. At this point, we’re not sure that goal is realistic: we want our giving-season recommendations to take advantage of the most recent possible information and ideas, and it’s unlikely that we’d be comfortable with finalizing our recommendations before the date that we have to do so. An alternative way to avoid the issue described above might be to announce our recommendations to Good Ventures and the public at the same time.

Issues with cost-effectiveness analysis

We did not follow the ideal process for reviewing and internally critiquing our cost-effectiveness analyses, which led us to finalize them later in the year than would have been ideal. In particular:

  • There was little senior-level review of the details of some of our key cost-effectiveness analyses (e.g., the cost-effectiveness analyses for SCI and DMI) until late in our research process.
  • We did not ensure that multiple staff members understood the most important parameters and assumptions in all cost-effectiveness analyses until late in the research process. For example, the proportion of deworming pills that were given to children as part of SCI‘s campaigns was a relatively important parameter in our cost-effectiveness analysis for SCI, but we did not have as much confidence in our understanding of this parameter as we could have at the end of the year.
  • The cost-effectiveness analyses were often complicated and somewhat opaque, which made it difficult for staff members to use the analyses as an input to their thinking about what GiveWell’s recommendation should be.

After putting in additional work on the cost-effectiveness analyses late in the research process, we ultimately felt that they were acceptable, but we plan to improve these analyses in the future (more details in the next post in this series).

Quality of our traditional work
 

Quality of recommended charities list

The quality of our top charities list (measured roughly in terms of expected impact) improved in 2014 relative to 2013 because AMF had room for more funding, a new study increased our estimate of the impact of deworming programs, and GiveDirectly had a stronger track record after another year of successfully distributing unconditional cash transfers at scale.

Additionally, we added four “standout” organizations to our recommended charities list, which we felt improved the quality of our recommendations for the reasons mentioned above.

Research quality

We feel that we maintained the high quality of our research in 2014. Though evaluating the quality of our research is difficult and involves many subjective judgments, we feel we have maintained our research quality because:

  • Our major research reports (charity reviews, intervention reports, etc.) lay out all reasoning explicitly and back up all evidence-backed claims with footnotes that show what evidence is being used to support their claims. These standards force all researchers to produce reports that can be easily vetted by other staff and the public. All reports receive many levels of critical review before they are published. For example, each charity review and intervention report is reviewed by at least one staff member who did not write the report and by the staff member’s manager. For intervention reports, we generally solicit feedback on the quality of the reports from experts in the appropriate fields (see, e.g., our water quality report).
  • We feel that we have a very strong understanding of our recommended charities’ activities. In general, we feel that the quality of our “What does [the charity] do?” and “Does it work?” sections of charity reviews are as high as or higher than they have ever been. For example, our understanding of (top charity) SCI’s activities is much stronger now than it had been in the past due to greater capacity for deepening our investigation.

However, we believe that there is still room to improve the quality of our research. In particular, we think that the “What do you get for your dollar?” (cost-effectiveness) sections of our charity reviews could be substantially improved and that the “Room for more funds?” sections could be improved. More details on this in the next blog post in this series.

Other self-evaluation questions
 

Does our impact justify the size of our staff?

In 2014, we moved about $28 million to our recommended charities. Excluding Good Ventures’ giving, we moved approximately $12.7 million to our recommended charities. (More details on our 2014 money moved will be in our forthcoming 2014 metrics blog post.) We currently project total GiveWell/Open Philanthropy Project expenses of about $2.3 million for 2015 (more). We estimate that about half of those expenses are attributable to GiveWell’s traditional work. We previously wrote that we believe that expenses that are 15% of money moved are well within the range of normal, so we feel comfortable with the relative size of our operating expenses at this point.

How much larger should GiveWell’s staff become?

As noted above, we have substantially increased our capacity for GiveWell’s traditional work after many years of struggling to do this. However, we feel that it is worth critically evaluating how much value is being added by our additional capacity and how much further we should expand our staff, if at all.

An important factor in our thinking about the ideal size of GiveWell staff is that we now see more potential than we had previously for some staff to transition to working for the Open Philanthropy Project.

To analyze the costs and benefits of different staff sizes, we can imagine three scenarios for future GiveWell staff:

  1. Expansion: increasing the size of GiveWell’s staff would allow us to: review as many or more new charities each year in the future, eventually enable us to allocate more staff to the Open Philanthropy Project, potentially improve our work of “seeding” potential future top charities, and potentially improve our future outreach efforts.
  2. Status quo: if we kept the size of GiveWell staff the same as it is now, we would likely dedicate most staff to maintaining our current level of research. Under this scenario, we would likely halt the transition of staff to the Open Philanthropy Project, not do substantial work to improve future outreach efforts, and do relatively little to seed potential future top charities.
  3. Contraction: in this scenario, we would reduce the size of GiveWell staff to the minimum amount of staff needed to maintain our recommendations. A smaller staff would likely be able to publish updates on our past top charities while conducting about one new charity review per year. Under this scenario, we would be relatively unlikely to find promising new giving opportunities, so we would be making a bet that we had already largely found the best giving opportunities.

The main arguments we see in favor of expansion are:

  • If our money moved continues to grow, we will likely need more “room for more money moved.” To increase “room for more money moved” and ensure that we are recommending high-quality giving opportunities, we will likely need to do research on new charities and do more work to seed potential future top charities.
  • The Open Philanthropy Project is early in its process of finding promising new giving opportunities and is severely capacity-constrained. Increasing the size of GiveWell’s staff will likely lead to more capacity for the Open Philanthropy Project.
  • GiveWell would need more staff in order to do more work on seeding potential future top charities and to do more outreach while maintaining its current level of research. These activities could be highly valuable.
  • Hiring operates on a long time scale; there are long lags between a) advertising a position, b) hiring and c) the new staff member reaching their full potential. Highly experienced hires are very versatile and valuable; the benefits of making such hires are robust across many potential future paths for GiveWell and the Open Philanthropy Project.
  • The worst case scenario for overexpansion is that some amount of money is used inefficiently on staff and that GiveWell must contract later, while the worst case scenario for underexpansion is that GiveWell and the Open Philanthropy Project are unable to capitalize on a vastly larger future opportunity for impact.

The main arguments we see in favor of maintaining the status quo or contracting are:

  • GiveWell’s “impact per dollar” would likely be higher in the short term in the status quo or contraction scenarios because we could maintain our current top charities list while spending less on our operations. GiveWell has not found many new top charities in the recent past, so we may not be sacrificing much impact by contracting. However, the legitimacy of GiveWell’s top charities list may degrade over time if the set of plausible candidates for top charities grows relative to the set of charities we have considered.
  • To some extent, there are diminishing returns to additional hiring because a growing staff requires more overhead- and human resources-related work.

Ultimately, we feel that the arguments in favor of expansion are significantly stronger than those for maintaining the status quo or contracting. However, we are still unsure of how much larger GiveWell’s staff should become in the longer term. The ideal future size depends on many factors, such as whether our research process has been identifying new top charities, the size of the “pipeline” of potential new top charities and priority programs (which we plan to discuss in the next post in this series), how many existing GiveWell staff ultimately work for the Open Philanthropy Project, and the size and success of our outreach operation. We plan to continue revisiting this question periodically.

Allocation of resources to research vs. outreach

As with previous years, we did not set a goal to do more outreach in 2014; we maintained our outreach at similar levels to what we had done in the past. Our approach to outreach has been to prioritize the highest return-on-investment activities while not making outreach a major priority. That said, the resources that we devote to outreach are not insignificant. For example, Co-Executive Director Elie Hassenfeld spent more than 10% of his time on outreach in 2014. More details on how we think about prioritizing outreach are available in this blog post.

Note 1: In this post, senior staff refers to Elie, Holden, and Alexander. Many staff took on additional responsibilities throughout 2014, so this refers to senior staff as of January 2014, not as of today.

Note 2: These were not necessarily the charities that we had expected to review at the beginning of 2014. At that time, we believed that we might complete reviews for ICCIDD (now named IGN), Centre for Neglected Tropical Diseases (CNTD), Nothing But Nets, UNICEF Maternal and Neonatal Tetanus Elimination Initiative (MNT), Measles and Rubella Initiative, and Menafrivac. Of those charities, we completed a review for IGN and made substantial progress on forthcoming reviews for CNTD and UNICEF MNT. Nothing But Nets declined to participate in our process. We ultimately prioritized different charity reviews because we learned new information–for example, Living Goods contacted us to share early results from its RCT and DMI found promising midline results from its RCT.

GiveWell’s progress in 2014 and plans for 2015: Summary

(updated on: ) | by Holden

This is the first post (of six) we’re planning to make focused on our self-evaluation and future plans.

As in past years, we’re going to be posting our annual self-evaluation and plan as a series of blog posts. This post summarizes what changed for GiveWell in 2014 and what it means for the future. Future posts will elaborate.

Money moved to our top charities was ~$28 million, compared to ~$17 million in 2013. Excluding Good Ventures, money moved to top charities went from ~$8.1 million in 2013 to ~$12.7 million in 2014.

We made major progress on building capacity, and plan to continue expanding.

  • At the beginning of 2014, we had 11 full-time staff and 1 Conversation Notes Writer; as of today, we have 18 full-time staff and 8 Conversation Notes Writers.
  • Non-senior staff (note 1) have been taking on significantly more responsibility, as senior staff have focused more on the Open Philanthropy Project and management. In particular, all four new charity reviews (DMI, IGN, GAIN, and Living Goods) as well as all three site visits were led by non-senior staff.
  • Of our current full-time staff, five work primarily on the Open Philanthropy Project, while the other thirteen do a mix of top charities work and cross-cutting work (including managing Conversation Notes Writers, vetting content from both projects, and administrative work). Currently, our payroll expenses are roughly evenly allocated between the two projects.
  • We are hoping to add 4-8 additional Research Analysts over the next 12 months. There are three future Research Analysts (two of whom were previously Summer Research Analysts) who have accepted offers and are starting mid-year. We are hoping to involve more Research Analysts in the Open Philanthropy Project, particularly for helping with writeups of cause investigations and grants, as well as build still more capacity for evaluating potential top charities. In addition, we are starting to seek cause-specific hires for the Open Philanthropy Project, and we have started to advertise for an Outreach Associate position to help us continue to maintain relationships with a growing number of people who give significantly to our top charities.

Our work on top charities produced much more output than in past years.

In the coming year, we hope for a similar level of output, while further improving the quality of our research, particularly when it comes to the transparency of our cost-effectiveness analysis and the reliability of our room for more funding analysis. We hope to do this while further reducing the role of senior staff, and shifting some capacity to the Open Philanthropy Project.

We feel that our top charities generally improved as giving opportunities. There were no new additions to the list, though some of this year’s “standout charities” may become top charities in the future. Against Malaria Foundation returned to our list for reasons related to room for more funding. A combination of new evidence and successful scaling up improved our confidence in all four organizations.

The Open Philanthropy Project progressed and evolved substantially, though it came short of our stretch goals.

  • We made substantial progress on our main priorities: U.S. policy and global catastrophic risks. The precise nature of our goal (commitments to causes) shifted, but we have completed a substantial number of high-level cause investigations and decided on our working cause priorities. We are now shifting our focus from cause investigations to aiming for major grants and/or hires.
  • We made less progress than hoped on other cause categories: scientific research funding and global health and development. For 2015, our main goal (a stretch goal) is to form clear priorities within scientific research funding, comparable to where we currently stand on U.S. policy and global catastrophic risks.
  • We have recently been prioritizing investigation over public writeups, and our public content is running well behind our private investigations. We are experimenting with different processes for writing up completed investigations – in particular, trying to assign more of the work to more junior staff.

We are planning to launch new websites for both GiveWell and the Open Philanthropy Project this year. Creating separate websites for GiveWell and the Open Philanthropy Project is a step in the direction of creating clear separation between the two. We are also hoping to begin conversations about what it would look like to form two separate organizations.

Fundraising remains a priority. We are currently fundraising for unrestricted support, supporting a team that is allocated flexibly between Open Philanthropy Project and our more traditional work.

Note 1: In this post, senior staff refers to Elie, Holden, and Alexander. Many staff took on additional responsibilities throughout 2014, so this refers to senior staff as of January 2014, not as of today.