The GiveWell Blog

Narrowing down U.S. policy areas

Throughout the post, “we” refers to GiveWell and Good Ventures, who work as partners on GiveWell Labs. [Added August 27, 2014: GiveWell Labs is now known as the Open Philanthropy Project.]

As part of our work on GiveWell Labs, we’ve been exploring the possibility of getting involved in policy-oriented philanthropy (see our previous posts on this subject). At this point, we feel that:

  • We’ve done at least some degree of investigating the causes that seem most promising to us, and we’ve gained an initial level of familiarity with how to think about what a promising cause is.
  • We see major gains to choosing longer-term focus areas – causes that we can commit substantial person-hours, and substantial funding, to over the next several years.

Because of this, we are now laying out the causes we tentatively feel most likely to commit to, and doing substantial investigation (including some grantmaking) in these areas. We aren’t yet committing to these causes, but we think that laying out our current thinking and reasoning will help surface important questions and intensify the period of reflection leading up to a decision.

We previously wrote about the importance of committing to causes.

Why commit to causes in U.S. policy?
We believe that policy-oriented philanthropy is an extremely important type of philanthropy to be familiar and experienced with. The potential leverage of influencing governments (whose budgets and other powers generally dwarf what philanthropists can provide) means that policy-oriented philanthropy on a broad range of causes could potentially be competitive (in terms of “good accomplished per dollar”) with even the most effective direct aid programs. For the moment, we are focused on the U.S., because:

  • GiveWell is located in the U.S. We have a far greater level of background familiarity with the U.S. policy landscape than with policy in other countries.
  • We find it far easier to identify, form relationships, and learn from people with expertise on U.S. policy than people with expertise on other countries’ policy (since the former tend to live in the U.S. and to share language and background knowledge with us).
  • The U.S. is the world’s largest economy and most powerful country, so the potential leverage of affecting U.S. policy is enormous. We don’t perceive that there is an obviously superior country to focus on.

We believe that making commitments to causes (intending to allocate a substantial number of person-hours to them for the next several years, accompanied by substantial potential budgets) would dramatically improve our ability to learn about these causes, and to learn more generally about how to engage in policy-oriented philanthropy. It would also make it easier for us to make plans around hiring and developing policy-focused staff.

With the level of investigation we’ve done so far, we feel we’re hitting diminishing returns on our ability to distinguish between different causes; however, gaining more in-depth experience with the ones that currently seem most promising – including identifying more giving opportunities, following grants, developing more relationships and putting in more time and thought – could improve our ability to think intelligently about what constitutes a promising cause.

By the end of this calendar year, we hope to make substantial commitments to several (probably 1-3) causes in the category of policy-oriented philanthropy.

What we’ve done to investigate policy-oriented philanthropy
We wrote previously about the work we’ve done to gain basic context in policy-oriented philanthropy. Since then, we have done the following.

Conversations with “generalists” who can speak to a variety of different political causes. We previously mentioned speaking with Dylan Matthews, Frank Baumgartner, Steven Teles, Mark Schmitt, Gara LaMarche, and the heads of the Center for Global Development and Brookings Institution. Since then, we have:

  • Spoken further with many of the people mentioned above, especially Steve Teles, whom we have retained as a consultant. Prof. Teles is the only person we’ve come across who has extensively studied the historical role of philanthropy in politics, and seems to have a broad view of the different ways in which philanthropy can influence policy.
  • Posted notes from Matt Stoller, Dean Baker, Keith Humphreys, Philip Heymann, and Robert Greenstein, and had several more general policy conversations that we don’t have notes available for (in some cases the notes are forthcoming).
  • Spoken extensively with people at the Pew Charitable Trusts, a large charity engaged in a great deal of work on a variety of U.S. policy issues, with a high degree of attention to political tractability and a tendency to set concrete goals for policy change. (More extensive notes from Pew are forthcoming.)
  • Tried to deepen our understanding of causes that are highly relevant to global poverty via conversations with two people referred to us by Beth Schwanke, Senior Policy Counsel at Center for Global Development. Unfortunately, we have not been able to publish notes from these conversations.

Shallow investigations.

Medium-depth investigations. We have done deeper investigations on criminal justice reform and macroeconomic policy, and we have explored giving opportunities in labor mobility. We have also gained relatively deep familiarity with drug policy (this is a Good Ventures interest, with GiveWell providing support consistent with our policy on general support) and with organ transplant supply policy (via Alexander Berger’s personal interests and networking on this topic).

Finally, we have made a general informal effort to be more attentive to news and debates relating to U.S. policy, including regularly reading Wonkblog and now Vox (which we have found particularly helpful).

Our investigations have been far from comprehensive; we’ve prioritized causes we’ve had some reason to think were particularly promising, often because we suspected a relative lack of interest from other philanthropists relative to the causes’ humanitarian importance or because we encountered a specific idea from someone in our network. With that said, at this point we have put a great deal of work into discussing and investigating different possible ways of engaging with U.S. policy, and have put at least some consideration (not always including a formal investigation) into every potential cause we can identify. We’ve also tried to expand the horizons of the causes we’re considering via activities like scanning the publication lists of major think tanks, scanning summaries of the U.S. budget and scanning the list of federal agencies.

General patterns in what causes we find promising
At any given time, we have both a working theory of what our criteria should be (what makes a cause promising, and hence what we should focus our information-gathering efforts on for a given cause) and of what the most promising causes are (considered holistically, without necessarily relying on our existing criteria). Reflecting on the latter (what causes seem most promising to us) often causes us to modify the former (what our list of criteria looks like), while collecting information using the rubric provided by our criteria often causes us to update our views of what the most promising causes are.

So far, it seems to us that the most important broad qualities that make a political area seem promising are:

  • Importance: how much humanitarian benefit would a small, medium, or large “victory” – in the sense of impacting a change in policy (or defending the status quo when a change would have been negative) – bring about?
  • Tractability: what do the prospects seem to be for achieving a victory over the short or long term? Is the status quo too politically entrenched to overcome?
  • Crowdedness (analogous to room for more funding): how much of the existing advocacy infrastructure is pushing for goals similar to ours? Are there gaps in this infrastructure that we might fill?

Speaking generally (more details in the next post), we’ve been able to assess these aspects of a cause only at fairly low resolution, and we haven’t fully explored their interrelations.

  • Re: importance. Importance can’t be assessed fully in isolation from tractability-related concepts, since we need a sense of what a small, medium and large “victory” would look like, and that in turn requires a sense of what might be possible. We’ve done back-of-the-envelope estimates for a variety of causes and generally believe that we can tell the difference between an enormously important policy area (one in which changes in policy could dramatically affect large numbers of people), a reasonably important area, and a relatively unimportant area, but that we can’t say much with confidence beyond that (and even our confidence in assigning an issue to one of those three categories is quite limited). We believe that exploring causes more deeply will improve our ability to think about what a small, medium and large victory would look like, and thus to assess importance.
  • Re: tractability. As discussed previously, it can be very difficult to predict whether and when a policy area might become tractable, and there is an argument against putting too much weight on the apparent tractability of a cause that one seeks to work on for the long run. We have been hesitant to dismiss any issue as fully intractable where we see room for improvement (from a humanitarian perspective) in policy. In addition, tractability can’t be assessed fully in isolation from crowdedness: the more gaps there are in existing efforts, the more reasons one might have to hope that entering a space will change the dynamics. We generally believe that we can distinguish between (a) an “unusually tractable” cause – one in which dynamics are shifting and a “window of opportunity” for change seems to be present or imminent; (b) an “unusually intractable” cause – a highly crowded cause which seems to be at a stalemate; and (c) causes that fit neither category.
  • Re: crowdedness. Crowdedness is also a complex thing to assess, since it can encompass several different questions. Does a policy area get (a) a lot of attention? (b) a lot of funding from relevant interest groups? (c) a lot of funding from philanthropy specifically (which may have structural strengths and weaknesses relative to other interest groups, and therefore may have things to offer that they don’t)? A policy space may be highly crowded in some respects and uncrowded in others – for example, there may be a great deal of academic research but little think tank work (more on the different types of infrastructure that can work toward policy change). We believe that we will improve our understanding of how to assess the crowdedness of a cause as we get deeper into areas and see whether ideas that look neglected from the outside turn out to be truly neglected. With that said, we have done a good deal of work on assessing the crowdedness of different causes, and often have a sense for how much work – and of what type – goes into a particular policy space.

Given the generally low level of resolution at which we understand all three of these factors, we have become most interested in causes that seem to clearly stand out on at least one dimension while performing relatively well (compared to other standout causes on the same dimension) on other dimensions. In other words, we are interested in causes that seem to have enormous importance, while being at least as tractable and uncrowded as similarly important causes; causes that seem to have unusual “windows of opportunity,” while being at least as important and uncrowded as similarly tractable causes; and causes that seem to be extremely uncrowded/neglected, while being at least as important and tractable as similarly uncrowded causes. We could imagine that any of these three profiles could turn out to be optimal for a philanthropist, since we could imagine that any one of these three criteria turns out to be more robustly detectable than the others.

In the next post, we will discuss specifics of what causes we feel stand out on each dimension, and which causes we believe we are most likely to commit to (and are accordingly investigating deeply at the moment).

Nothing But Nets

In 2014, we prioritized Nothing But Nets (http://nothingbutnets.net) as a potential GiveWell top charity because it funds insecticide-treated nets to prevent malaria, one of our priority programs.

In early 2014, we contacted Nothing But Nets to explain our process and to invite it to apply for a recommendation. Nothing But Nets provided brief responses to questions we had previously sent them (.docx), but has decided not to fully participate in our review process at this time.

In our 2013 annual review, we wrote that we were seeing more interest from charities in participating in our process and expected fewer organizations to decline to participate. We’ve written this post to share our impression about Nothing But Nets’ decision.

Our understanding is that Nothing But Nets has a small team (~2 staff members) focused directly on fundraising, and that it would be challenging for its existing staff to engage with GiveWell in our process. Our review process would require a significant time investment, including at the very least, (a) several lengthy (~2-hour) phone calls to understand Nothing But Nets’ role and value added; (b) submission of documents we request and email responses to questions we ask; c) a multi-day site visit with GiveWell staff, and (d) review of any write-up we produce about Nothing But Nets.

Although GiveWell’s recent money moved is high relative to Nothing But Nets’ funding — in 2013, GiveWell directed more than $17 million to the 4 organizations recommended throughout the year, with more than $2 million going to each organization, and Nothing But Nets has raised approximately $50 million since its founding in 2007 — it would not surprise us if the amount of time needed to meaningfully engage with us would be a major cost for an organization Nothing But Nets’ current size.

With that said, based on the information Nothing But Nets has shared with us (its brief responses to questions we had previously sent, linked above) – as well as our view that we have been clear about the requirements and the likely benefits of becoming recommended – our impression is that that Nothing But Nets’ decision not to apply has not caused us to miss out on a likely top charity.

Notwithstanding this impression, we remain interested in Nothing But Nets and hope that they will engage with us in the future.

Update on GiveWell’s web traffic / money moved: Q1 2014

In addition to evaluations of other charities, GiveWell publishes substantial evaluation of itself, from the quality of its research to its impact on donations. We publish quarterly updates regarding two key metrics: (a) donations to top charities and (b) web traffic.

The table and chart below present basic information about our growth in money moved and web traffic in the first quarter of 2014 (note 1).

Money moved: first quarter

Growth in money moved, as measured by donations from donors giving less than $5,000 per year, was strong in the first quarter of 2014 (money moved was 96% higher than in the first quarter of 2013), and was substantially stronger than growth in the first quarter of 2013. The total amount of money we move is driven by a relatively small number of large donors. These donors tend to give in December, and we don’t think we have accurate ways of predicting future large gifts (note 2). We therefore show growth among small donors, the portion of our money moved about which we think we have meaningful information at this point in the year.

Growth in number of donors was also strong, and similar to growth in this metric in the first quarter of 2013.

Web traffic: first quarter

In the past, we have relied on data from the web analytics company Clicky for our metrics updates. We also track web traffic through Google Analytics, and between January 2012 and February 2014, Google Analytics consistently tracked lower overall traffic than Clicky. We do not know what the cause of the discrepancy between the two sources is, and do not have a view on which data source is more likely to be correct. For that reason, we present data from both sources here. Full data set available at this spreadsheet. (Note on how we count unique visitors.)

Traffic from AdWords decreased in the first quarter because in early 2014 we removed ads on searches that we determined were not driving high quality traffic to our site (i.e. searches with very high bounce rates and very low pages per visit).

Data in the chart below is an average of Clicky and Google Analytics data, except for those months for which we only have data (or reliable data) from one source (see full data spreadsheet for details).


Note 1: Since our 2012 annual metrics report we have shifted to a reporting year that starts on February 1, rather than January 1, in order to better capture year-on-year growth in the peak giving months of December and January. Therefore metrics for the “first quarter” reported here are for February through April.

Note 2: In total, GiveWell donors have directed $1.45 million to our top charities this year, compared with $0.70 million at this point in 2013. For the reason described above, we don’t find this number to be particularly meaningful at this time of year.

Note 3: We count unique visitors over a period as the sum of monthly unique visitors. In other words, if the same person visits the site multiple times in a calendar month, they are counted once. If they visit in multiple months, they are counted once per month.

Note 4: Google Analytics provides ‘unique visitors by traffic source’ while Clicky provides only ‘visitors by traffic source.’ For that reason, we primarily use Google Analytics data in the calculations of ‘unique visitors ex-AdWords’ for both the Clicky and Google Analytics rows of the table. See the full data spreadsheet, sheets Data and Summary, for details.
 

The importance of committing to causes

Updated Sept. 12, 2014 to change “GiveWell Labs” to “Open Philanthropy Project,” in line with our August 2014 announcement.Throughout the post, “we” refers to GiveWell and Good Ventures, who work as partners on the Open Philanthropy Project.

In our work on the Open Philanthropy Project, we’ve consistently found that the level of interest we show in a cause – including our perceived willingness to provide funding within it – is a major driver of what sorts of giving opportunities we’re able to find.

This dynamic has been one of the major factors in the grants we’ve made so far, and it’s also a major reason that we’re eager to “commit” to causes, as mentioned earlier this year. We believe that there’s a limited amount we can learn about a cause when presenting ourselves as “potentially interested in providing moderate amounts of funding” rather than “strongly interested in providing major funding.”

We’ve come to believe in the importance of committing to causes in order to investigate them, and in the importance of “giving to learn” for the Open Philanthropy Project, via the following process:

  • We initially envisioned a process that first gathers information on giving opportunities and then identifies which grants should be made.
  • However, from observing the behavior of potential grantees and other funders, we came to believe that a funder must be highly prepared (and likely) to make grants in an area in order to find giving opportunities in that area. Many people will only make the relevant referrals, propose relevant ideas, etc. once they are convinced of a philanthropist’s serious interest in providing funding.
  • As such, we have in many cases tried offering funding in an area – or at least expressed strong interest in the area – before knowing what giving opportunities would turn out to be available. This approach has led to multiple cases in which much of the learning value of a grant (from our perspective) comes from the process leading up to the grant.
  • “Giving to learn” can mean multiple things. It can mean (a) funding research in order to gain specific knowledge; it can also mean (b) funding a project in order to learn from following the project’s progress. The dynamic laid out in the above bullet points represents perhaps the most counterintuitive meaning: “giving to learn” can mean (c) offering funding in order to learn from the process of finding grantees.

This post lays out:

  • An overview of the paths we’ve taken in the last few years to find giving opportunities for the Open Philanthropy Project. More
  • Examples of where showing strong interest in a cause, and particularly in making grants within it, led to information that we couldn’t have gathered in any other way. More
  • The implications of this dynamic – including why we do not believe (contrary to what some have suggested) that funding research is necessarily the best way to gather more information relevant to our work (more) and why we think it is important to commit to specific causes fairly soon (more).

Approaches we’ve taken to finding giving opportunities for the Open Philanthropy Project
Early on in our work on the Open Philanthropy Project, we spoke to a broad array of people and organizations – including academics, funders and nonprofits – and asked them for their opinions on the best giving opportunities, broadly speaking. (We also asked for referrals to other people who might be able to help us with this question.) As noted in a previous post, this approach generally didn’t yield much in the way of actionable ideas (and was often met with responses like “That question is too broad” or “First I need to know what causes you’re passionate about”).

We gravitated toward assessing “causes”, which helped us to ask more focused questions (rather than “What is the best giving opportunity you know about?” we could instead ask something like “What would you do if you were a funder seeking to make progress in solving problem X?”) But as we conducted shallow (and even medium) investigations, we still encountered relatively few “shovel-ready” giving opportunities. We encountered organizations seeking more funding generally, but we didn’t see many cases where we had a clear sense that more funding would play a crucial role in allowing particular promising work to go forward.

At the same time, we were speaking with major foundations and trying to understand how they go about finding giving opportunities. We explored the possibility of co-funding projects with them, and we encountered ideas – such as a project combating malaria drug resistance in Myanmar (with the Gates Foundation) and the Service Delivery Indicators (with the Hewlett Foundation) – that seemed like interesting and relatively tangible (in the sense of understanding what activities were made possible by the funders’ support) giving opportunities.

In addition to these co-funding conversations, we devoted substantial time to exploring the cause of meta-research and observing how funders in that space were finding giving opportunities. We noticed that:

  • The “meta-research” ideas we saw, such as a registry for randomized controlled trials, were brought straight to the funders who already were known for supporting relevant research, and it seemed to us that the projects might get sufficient support this way without needing a broader search for funds. These funders seemed to us to be well-connected to the people best positioned to come up with ideas for working toward “meta-research” related goals.
  • A similar dynamic seemed like it might apply more broadly. We knew that as a grantee ourselves, we had been connected to the Hewlett Foundation by someone who knew of their interest in improving philanthropy; if not for that connection, it wouldn’t have occurred to us to approach them. Generally, it seemed to us (both from our reading of philanthropic success stories and from our conversations regarding “co-funding”) that funders encounter many opportunities via being approached, such that their reputation for being interested in one cause or another directly affects what opportunities they come across.
  • While investigating meta-research broadly, we sourced the proposal for a meta-research-oriented center at Stanford that we wrote about previously. This was a clear-cut case in which actively expressing an interest in funding certain kinds of projects led to discovery of a giving opportunity we couldn’t have encountered otherwise.

We started to feel that we might need to “invert” our investigative process: rather than (a) first “exploring” a cause, finding potential giving opportunities, and then deciding whether we were interested in providing funding, we should perhaps (b) pick a few causes and definitively express an interest in providing funding, before knowing of any particular giving opportunities. Our initial thinking that led to this idea was outlined in a 2013 post, Challenges of Passive Funding.

This shift led to a noticeable improvement in our ability to source tangible giving opportunities.

Examples of the “giving to learn” dynamic
The first cause we chose for a relatively deep investigation – including some grants – was criminal justice reform. Of the causes we were interested in, it seemed to offer the best odds of quickly finding “shovel-ready” giving opportunities, based on the comments of Steven Teles. We told Prof. Teles that we were interested in making some initial grants in this cause, and he quickly connected us to Mark Kleiman and Angela Hawken, each of whom sought funding. We also informed the relevant team at Pew Charitable Trusts that we were actively looking for giving opportunities, and discovered that this team was seeking funding (something that hadn’t come up in the first conversation we conducted with this team). We have since begun a thorough investigation of this organization’s track record in this space, with their help – something that wouldn’t have been as feasible if we hadn’t seriously been considering providing the requested funding.

Over time, we’ve seen more giving opportunities come up. We’ve been approached by multiple groups with confidential proposals to work toward reducing incarceration. In addition, Prof. Hawken contacted us when her organization, BetaGov, came across a seemingly unique and temporary opportunity to study the impact of changing marijuana policy in the state of Washington (more details forthcoming). These are giving opportunities we’re convinced we couldn’t have come across without expressing strong, credible interest in funding work on criminal justice reform (and in some cases, particularly Prof. Hawken’s, actually providing such funding).

In the meantime, Prof. Teles has continued to think actively about the topic of criminal justice reform, and has come up with multiple new ideas for things a funder might do. We are currently seeking to pause our work in this space, as we try to investigate other causes to a similar level of resolution; however, we’ve now gotten multiple people and organizations to see us as a potential source of funding and to start thinking about more work that would align with the aspects of the space we’re interested in.

Similar dynamics have applied to the other causes we’ve explored:

  • Malaria control and elimination. We commissioned Dr. Steve Phillips to explore this space for us and identify giving opportunities. We had substantial discussion with Dr. Phillips around setting expectations appropriately – in particular, what to say about the likelihood of funding – which he found important in order to have conversations about possible projects. In this particular case, we did not commit any funds to malaria control and elimination projects, but we wouldn’t have been able to conduct this project if we hadn’t seen a legitimate possibility of doing so, and we’d be hesitant to dig further on these proposals (or to do a similar project in another area of global health) without having a relatively strong expectation of following up with funding.
  • Labor mobility. We did a shallow investigation of the cause of labor mobility that included a conversation with Michael Clemens, a researcher on the global economics of migration who has been a leading voice on the humanitarian benefits of labor mobility.. However, it wasn’t until we communicated an intent to fund labor-mobility-related work that Dr. Clemens approached us with giving opportunities, including support of his own work as well as another project that we will be writing about in the future. We find the project both promising and unlikely to get funded without our involvement, and believe that we wouldn’t have been able to find out about it without specifically communicating an intent to provide funding. In addition, Dr. Clemens offered to introduce us to a person who might consider leaving their current post in order to pursue work in this area, but wouldn’t want this information widely disseminated.
  • New top charities. We have also been interested in making grants to increase the supply of evidence-backed charities serving the global poor. Here too, we have started conversations with research organizations that we wouldn’t have been able to have without a strong interest in (and high likelihood of) providing funding.

Why expressing stronger interest can lead to better giving opportunities
Without pretending to know exactly how the dynamics work, it seems to us that:

  • People are often hesitant to ask for funding, or even mention that they’re seeking it, until and unless they perceive a strong specific interest on the part of the person they’re talking to. (Just knowing that the person they’re talking to is a funder and “open to many possibilities” is often not enough.) Part of this may be a fear of being perceived as “unsuccessful” if they are public about having an idea that they can’t find funding for.
  • People are often hesitant to put time into fleshing out an idea until they see a potential path to getting it funded. This seems rational, especially since different funders will often have different preferences in terms of what information they find most important, what sorts of proposals they want to see, and what aspects of the work are most important to prioritize from their perspective.
  • Many people seem to seek funding primarily by going through their networks, and seeking out people who are clearly interested in what they’re doing (rather than by publicly disseminating their ideas).

In theory, it seems possible to have a world in which funding ideas are written up and posted publicly for anyone to browse. In reality, funding ideas are often not even internally fleshed out (much less written up) until specific interest is perceived. Because of this, asking someone for giving opportunities often means asking for substantial amounts of their time and energy, and it can be inappropriate to do so except when one has a high probability of following through with funding.

The approach we’re taking
Our basic heuristics for deciding what and how much to fund has been:

  • Be very thoughtful and careful about how we spend our time, and realize that making grants will almost always have implications for how we spend our time (by signaling our interests, by causing more people doing similar work to approach us, by forming relationships with grantees, etc.)
  • When we’re interested in an area, be willing to express strong interest in providing funding and to follow through with high probability.
  • Fund projects we come across that seem reasonable, that we’re willing to spend some time following up on, and that fit within our interests in terms of what areas we want to learn more about and see more proposals from.

Our priority at the moment is asking what sorts of giving opportunities might exist in different causes (and, along with this, learning about meta-issues such as the “giving to learn” dynamic described in this post). We think of this as an learning/information-gathering agenda, supported by “giving to learn” grants whose informational value comes from (a) following through on initial expressions of interest; (b) signaling our further interests; and (c) giving us opportunities to follow up over time and learn about the relevant people and organizations and their progress.

We’ve seen a few comments that we don’t seem to place much value on “value of information,” since we’re primarily funding direct work of various kinds rather than research projects aiming to identify the best causes. I disagree with comments along these lines. Our work is a research project aiming to identify the best causes, and funding projects in causes of interest is an important tool for carrying this project out. This sort of “giving to learn” provides certain kinds of information (e.g., “what are the giving opportunities in cause X like?”) relatively quickly and efficiently; more broadly, it is a form of “learning by doing” that has already yielded insights about grantmaking (such as how the “giving to learn” dynamic works) that would have been difficult to pick up in any other way. By contrast, funding studies would introduce the management challenge (costly in terms of person-hours, our scarcest resource) of trying to align researchers’ work with our own interests, and could take years to produce actionable information.

The importance of committing to causes
To date, we’ve quite deliberately limited our involvement in – and commitments to – any given cause. We believe that this has placed limits on how well we’re able to get to know the fields in question. If we were to make a substantial “commitment” to a cause – intending to allocate a substantial number of person-hours to it for the next several years, accompanied by a substantial potential budget – we would be able to:

  • Network more extensively, get to know the relevant people and organizations, and communicate the nature of our interests.
  • Encourage people to come to us with ideas, without having to caveat that our interest is preliminary and we may not be able to provide stable/renewable funding over time.
  • Generally do more investigation and learning about the cause, leading to refinements in the specific types of projects we’re looking to fund – which in turn would affect the types of projects that would come to our attention.

We believe these activities would lead to deeper understanding of the few causes we investigate, which would likely inform how we approach our lower-depth investigations of all causes. Accordingly, we see a great deal of value in making such commitments relatively soon, rather than trying to be comprehensive in doing lower-depth investigations of every possible cause of interest.

Our writeup on criminal justice reform

[Added August 27, 2014: GiveWell Labs is now known as the Open Philanthropy Project.]

We’ve just published an extensive writeup on the cause of U.S. criminal justice reform, which was one of the causes we previously listed as a priority for investigation under GiveWell Labs.

We first became interested in this cause when our initial conversations around promising policy areas (in particular, our extended conversation with Steve Teles) highlighted it as having unusual political tractability. We heard from multiple sources that the combination of the adverse U.S. fiscal situation, low crime rates, and emerging conservative interest in an issue historically supported by progressives may have created a “unique moment” for criminal justice reform with a limited window.

Due to the significant amount of suffering and expense associated with incarceration, we believe there are high humanitarian stakes as well. And in initial conversations about the cause, we identified some approaches that we believe to have relatively little in the way of philanthropic support.

There are other causes that we see as having higher humanitarian stakes than criminal justice reform, and other causes that we see as being more philanthropically “neglected,” but such causes generally don’t have the same “window of opportunity” dynamic. So criminal justice reform is a plausible, though not the only plausible, candidate for “outstanding cause to work on.”*

When we started doing deep investigations of new causes, we chose to work on criminal justice reform first, because we had multiple leads already for potential grantees, and it seemed like a cause that we would quickly learn about and make progress on from the standpoint of finding giving opportunities. Since then, we have had many conversations and identified initial grantees that we find quite promising. These grantees’ activities include research, technical assistance and advocacy. They are taking what we see as promising approaches to reducing incarceration while preserving or improving public safety (and have substantial room for more funding).

We have paused work on this cause for the time being as we investigate other potential focus areas (more in future posts). However, we think it is quite likely that we will end up declaring criminal justice reform as an ongoing priority for GiveWell Labs.

Much more information at our full writeup:

Our writeup on criminal justice reform

*To put this in “cost-effectiveness analysis” terms: the “window of opportunity” aspect implies that chances for influencing policy are unusually high in this domain; our analysis of potential policy impacts implies that such influence would have high humanitarian value; the fact that there is little existing philanthropic support for what we consider promising approaches implies that more funding could make a difference. Thus, speaking broadly, the prospects for high impact per dollar spent seem relatively strong. Future posts will compare this cause more explicitly with other potential policy causes. As discussed previously, we believe that estimating the cost-effectiveness of policy-oriented philanthropy is particularly challenging, but that there are initial reasons to believe that it can be quite high relative to other styles of philanthropy.

A promising study re: Development Media International

Development Media International (DMI), an organization that “runs radio and TV campaigns to change behaviours and save lives in developing countries,” recently released preliminary midline results from a randomized controlled trial. These results imply that exposure to DMI’s content caused people to take up a variety of behaviors (including early initiation of breastfeeding and various measures for controlling diarrhea) at increased rates. We have not yet seen the full technical report and we have substantial questions about these results, but believe that the basic design of the study is a major improvement on previous evidence regarding similar programs.

If the results – and DMI’s cost-effectiveness calculations – held up, DMI could become a GiveWell top charity with substantially better estimated “cost per (equivalent) life saved” than our current top charities. Because of the nature of DMI’s intervention – attempting to change behavior through mass media – it is quite plausible to us that its cost per person reached is extremely small, and thus that the cost-effectiveness is extremely high. However, there are many questions we would need to look into before determining whether this is in fact the case.

As such, we plan to investigate DMI thoroughly this year. There is some possibility that we will decide to recommend it at the end of this calendar year, and some possibility that we will instead choose to hold off on deciding whether to recommend it until next year, when the study’s endline results (which we expect to give a more reliable picture of its impact) are released. Note that we recommend only charities that we have thoroughly vetted and spent substantial time considering alongside other contenders, and we have designed our process to be most up-to-date when it is most relevant (in December); so regardless of what we find, we are extremely unlikely to change any official recommendations ahead of that timeline. Our recommendations do not reflect real-time “best guesses,” but rather reflect the results of thorough investigation and deliberation.

More on DMI and the recent study results
We’ve long been interested in mass media for behavior change: aiming to use mass media (e.g., radio or television programming, often soap operas) to promote and disseminate messages on potential life-saving practices such as oral rehydration therapy, sleeping under bednets, breastfeeding, etc. It’s quite plausible to us that messages on TV or the radio could influence behavior, and could reach massive numbers of people for relatively low costs, leading to high cost-effectiveness in terms of lives saved or improved per dollar spent.

However, as we state in our current writeup on the subject, we have previously found research on such programs to have important limitations and to be generally prone to selection bias. As such, we have been keeping tabs on DMI’s ongoing study, which – unlike previous work – uses randomization to isolate the impacts of its programming.

Preliminary, public discussion of DMI’s midline results became available recently, and we conducted a conversation with DMI representatives about the results (notes from this conversation are forthcoming). We note that:

  • The preliminary results seem to indicate that DMI’s intervention had substantial impacts on behavior change (the easiest way to see this is to scan the rightmost column, “differences in differences (unadjusted)” in DMI’s table, though there is some other evidence available as well). We do not yet have complete information about the statistical significance of these findings (the public report gives p-values for four of ten behaviors).
  • It appears that the treatment and control groups differed substantially at baseline (in terms of their behaviors and other indicators), making the comparison less than ideal. In addition, the control group (which did not receive DMI’s intervention) saw substantial improvement on the key indicators, which we don’t have an explanation for at this time. With that said, skimming the summary table provided by DMI, it appears that in many cases, the treatment group started with lower rates of a given behavior, and then caught or surpassed the control group, which (when viewed in the context of a randomized selection process) does seem to indicate a positive effect to us, though the magnitude may prove hard to estimate reliably.
  • DMI provides an estimate that its intervention costs ~$60-300 per life saved, which would make it much stronger (around an order of magnitude stronger) than the strongest of our top charities. DMI also stated to us in conversation that the midline results are reasonably consistent with the assumptions that went into the estimate, though it appears to us that there will be substantial judgment calls in how to interpret the results in light of the estimate. We have not yet seen the technical details of DMI’s estimate. We expect our final estimate of DMI’s “cost per life saved” to be substantially less optimistic than DMI’s current estimate, but at this time it is hard to say by how much.
  • DMI’s midline results come entirely from self-reported data on behavior, which is potentially a highly problematic measure for assessing the impact of a mass media campaign (since the media campaign may impact people’s perceptions of how they should respond to surveys more than it affects their actual behavior). It appears to us that DMI is aware of this issue, and states that its endline results (slated for 2015) will measure mortality.
  • Based on DMI’s website and our discussion, we preliminarily perceive DMI so far to be unusually thoughtful and analytical in its self-evaluation and -analysis.

Our next steps
We hope to:

  • Obtain the technical report behind the midline results and examine it closely. DMI has offered to share this with us in the next few weeks.
  • Examine the technical details behind DMI’s “cost per life saved” estimate and assess them with our analysis of the midline results in mind. DMI has shared its model with us, though we have not yet examined it.
  • Get more information about DMI’s plans and room for more funding and consider how our cost-effectiveness estimate might change in light of these.
  • If warranted, further examine DMI via a site visit and create a full charity review.

Because of the strong study design, the degree of thoughtfulness and analysis we’ve seen so far from DMI, and the strong claimed cost-effectiveness and basic plausibility of the claim, we believe there is a substantial chance that DMI will eventually become a recommended charity, and that there is also a (smaller) chance that we will end up estimating substantially higher “bang for the buck” for DMI compared to our current top charities. If this turns out to be the case, we will hope to help close any funding gap of DMI’s.