We’re sometimes asked whether life-saving interventions (such as insecticide-treated net distribution) risk leading to overpopulation. A common response is that (as argued by Hans Rosling of Gapminder and the most recent Gates Foundation annual letter, among others) the reverse dynamic holds: population growth tends to slow as child mortality declines, possibly because parents feel less need to have multiple children in order to hedge against the risk of death.

Empirically, it appears generally true that lower child mortality tends to correlate with slower population growth. However, one might ask whether this relationship is causal (i.e., falling mortality leads to falling fertility, for reasons outlined in the previous paragraph) or merely correlational (i.e., falling mortality generally accompanies a host of factors related to general economic development, and including falling population growth). This question is important when assessing the impact of a life-saving intervention in isolation: it’s possible that health interventions, unaccompanied by improvements on other dimensions, do in fact lead to higher populations.

We’ve long wondered about this question, and we recently commissioned David Roodman – whom we have been impressed with in the past – to examine the rather extensive and complex literature on it. David has recently posted a draft of his writeup, and is seeking feedback.

We haven’t yet fully vetted the writeup, but it appears to me to give a deep and balanced examination of the available literature, along with interesting discussion of the general challenges of isolating causality in cross-country studies. Its conclusion:

I think the best interpretation of the available evidence is that the impact of life-saving interventions on fertility and population growth varies by context, above all with total fertility, and is rarely greater than 1:1 [rarely enough for the fertility decline to exceed the mortality decline in terms of effect on population size]. In places where lifetime births/woman has converging to 2 or lower, family size is largely a conscious choice, made with an ideal family size in mind, and achieved in part by access to modern contraception. In those contexts, saving one child’s life should lead parents to avert a birth they would otherwise have. The impact of mortality drops on fertility will be nearly 1:1, so population growth will hardly change.

Overall, it appears that life-saving interventions unaccompanied by other improvements, where access to contraception is weak, are likely to lead to some acceleration of population growth. With that said, we wish to note the following:

• No intervention takes place in isolation, and we expect population growth to slow in the future in most low-income areas as poverty falls.
• Acceleration of population growth should not necessarily be associated with overpopulation and its connotations of a net decline in standards of living.
• GiveWell’s current top charities are all focused on improving living standards rather than averting deaths, and Against Malaria Foundation (a former and potentially future top-rated charity) aims to avert deaths and bring about other benefits by reducing the burden of malaria.

We encourage our readers to check out the writeup and send in their thoughts. It’s a thorough review of the question and a generally interesting read.

More on why we commissioned this writeup

We have previously made some preliminary attempts to review the literature on the connection between mortality and fertility. However, we’ve found this literature to be (a) challenging to synthesize, as most of the key papers use relatively complex methodologies that would take significant work to interpret; (b) only a moderate priority for our research, since it has limited relevance to which charities we recommend and how we rank them.

Recently, we decided to explore the idea of working with David Roodman (whom we’ve generally been impressed with) as someone who might increase our capacity to produce thorough reviews of literature on key giving-relevant questions. For our first project together, we chose to work on this question because David had done substantial work on it in the past, and felt he could produce an initial writeup relatively quickly.

One of the core questions we’ve faced throughout GiveWell’s history has been whether to allocate our time to research (i.e., identifying and evaluating outstanding giving opportunities) or outreach (i.e., increasing the number of donors who are aware of GiveWell and use our research, and the amount of money we move to top charities). The question of “research vs. outreach” was one of the most common topics at board meetings in our early years, and new staff and supporters often ask us why we don’t focus more of our time on outreach.

Over the course of GiveWell’s history, we’ve taken many approaches to increase awareness of our work. We’ve tried all of these throughout our history, so we believe the experiences described below are reasonably representative of “what we’d expect to work today” rather than specific to a particular point in GiveWell’s development. The purpose of this post is to share our observations of what has worked and what hasn’t, in order to help people understand why we prioritize our time as we do and also in order to potentially help people thinking about their own outreach strategies.

The big picture is that:

• Proactive “marketing” has contributed little to our growth.
• Much of our most valuable publicity and promotion has come from enthusiastic people who actively sought us out.
• Supporters often take serious time, and vet our work, before becoming highly enthusiastic.

Because of this, we have generally felt that improving the quality of our research, and our existing audience’s understanding of it, has been the most important factor in our growth. We’ve consistently seen our best results come from focusing not on “How can we get people to find out about us?” but “How can we get the people who have already sought us out, shown interest and found us to become as enthusiastic as possible?”

More detail follows. We first discuss approaches that have not worked particularly well, then discuss the major factors in the growth we’ve seen.

These approaches either (a) have not had strong return-on-investment in terms of money-moved-per-hour-of-staff-time-invested or (b) do not seem likely to yield significantly better results if we put in additional time.

• Speaking opportunities. We’ve given a total of 50-100 talks – at companies (e.g., tech companies, banks, hedge funds, consulting firms), universities (undergrad, business and public health schools; both specific classes and broader community events), conferences, and smaller, personal events organized by GiveWell supporters. Few donors have found us via a talk. In hindsight, the costs of talks (e.g., practicing our talk, prepping slides, coordinating logistics, travel time) seem to have significantly outweighed the benefits. Over the past year or so, we’ve accepted invitations to speak, but only when (a) we can limit preparation time (e.g., by delivering our standard GiveWell talk rather than putting in serious preparation) and (b) travel time is low. We’ve also sometimes sent more junior staff to speak, partly for their training. Roughly speaking, I’d estimate that talks have yielded ~5-10 major supporters giving a total of approximately $200,000-$400,000/year. My best guess is that we have put in ~250-500 hours throughout our history into speaking opportunities.
• Networking with people who haven’t shown proactive interest in GiveWell. We’ve conducted hundreds of 1-on-1 meetings or phone calls with prospective donors. As a general rule, when we’ve initiated the meeting (i.e., a supporter of ours suggested we meet someone or we asked a supporter of ours to make an introduction), we’ve very rarely converted that person into a fan. A common pattern is that someone thinks GiveWell is a “great idea” but that this recognition rarely results in action, e.g., giving to GiveWell or our top charities. (This was a topic of discussion at a July 2010 board meeting.) Networking of this type has yielded a few major donors who collectively give ~$100,000-$150,000/year.
• Proactive media outreach. We’ve made various attempts to proactively encourage the media to cover us. We’ve had no notable success actively reaching out to the media; our media coverage has generally come via media reaching out to us. A fact that has further diminished our interest in allocating time to proactive media outreach is seeing the surprising (to us) results from what we would have considered major media. For example, we’ve seen limited returns – in terms of new donors – from appearances in newspapers like the Boston Globe and highly-trafficked major sites like Lifehacker, Reddit, and Gawker.
• Google AdWords. We have run AdWords consistently since GiveWell began, and it has consistently been our best performing (in ROI terms) proactive outreach strategy. However, while it has had a strong return on money invested, we have not found a significant number of large donors via AdWords. In 2009 (the latest year for which we have the data), we spent approximately $650 on AdWords during the last week of the year and counted $5,310 donated, see our 2009 metrics report (DOC). In 2013, we ran significantly more AdWords as part of the Google Grants program, leading to 202 donors. (We are not currently tracking the amount given, though we hope to be able to do this in the near future.)

We have also tried other outreach approaches at various times. We have (a) cold-called family foundations and corporate giving departments to encourage them to use our recommendations; (b) retained a professional designer to prepare materials as takeaways from talks and 1-on-1 meetings; (c) evaluated PR firms as a means for gaining broader media exposure. Both (a) and (b) resulted in significant time spent with little-to-no results. When we spoke with PR firms in 2009 and 2012, we found that the media coverage they expected to be able to generate for us was comparable to the media coverage we had already had which had not led to strong results (i.e., many donors/major donors) in the past.

(For those particularly interested in our early attempts at outreach, see our 2008 change of direction writeup.)

While the approaches described above have not worked particularly well, GiveWell has grown significantly. Some highlights follow; more information is available in our annual and quarterly metrics reports.

• Approximately 130,000 unique visitors came to GiveWell’s website in December 2013 compared with only 5,500 in December 2009. (December is the time of year when most individual donors give, so traffic then is a reasonable indication of GiveWell’s growth.)
• Over the course of 2013, we directly tracked more than 8,000 donors giving to our top charities compared with 693 in 2009.
• We tracked more than $17 million donated in 2013 to our top charities as a direct result of our research. As recently as 2009, we tracked only a bit over $1 million (and this figure included some funds committed in 2008).

Comparing our current reach to that which we had in December 2009 is instructive. By December 2009, we had already appeared in (a) a major New York Times article, (which also led to appearances on CNBC and NPR) and (b) Peter Singer’s book The Life You Can Save (which led to multiple additional media mentions during his book tour), and (c) Nick Kristof’s book, Half the Sky. Notwithstanding these major drivers of “eyeballs” to our website, GiveWell’s reach remained relatively limited.

Another interesting comparison is to our first December, in 2007, when we had an abnormal traffic spike due to being featured in an NYT story that reached #3 on the most-emailed articles list. We saw a total of ~50,000 visitors to our site that month – over 1/3 as many as in December 2013 – but our total money moved that month was only ~$30,000 (compare to more than $4.3 million in December 2013, excluding Good Ventures).

Our growth has been a function of audience enthusiasm and word-of-mouth, much more than of media exposure and attention.

We know the sources of our growth because we have surveyed and prioritized getting to know our major donors.

Most of our larger donors have told us that they either (a) found GiveWell while proactively looking for a resource to help them decide where to give or (b) heard about GiveWell from a trusted source (most commonly Peter Singer, other media, or a close friend or colleague).

Excluding Good Ventures (the single largest supporter of our top charities) and other institutions (which have different approaches to giving), over the past 3 years, 95 donors have accounted for close to 60% of the total non-institutional money GiveWell directed to top charities or received to support its operations. (Combining these two figures offers a reasonable picture of the most committed users/largest supporters of GiveWell’s work.)

We have data on how 84 of these 95 donors found us, which is shown in the table below:

See this footnote for definitions and more technical discussion.

It is important to note that the success we’ve had when someone refers us (either via writing something or referring friends) has often been a function of specific individuals’ repeatedly and enthusiastically promoting us or discussing our work in public. This includes people in the media (or who frequently contribute in media) such as Peter Singer, Ken Stern, Nick Kristof, Alex Tabarrok and Stephanie Strom but also individuals like Jonah Sinick (who played a large role in introducing the LessWrong community to GiveWell) and Jeff Kaufman and Julia Wise (who have promoted GiveWell to their coworkers, friends and on their respective blogs). In general, we feel that high-enthusiasm, lower-profile promoters have benefited us far more than higher-profile, one-time endorsements.

Good Ventures

We excluded Good Ventures from the above discussion, but its story is important because it is the single largest donor to our top charities (having directed $12 million to those organizations over the past 3 years). Broadly speaking, Good Ventures came to GiveWell in the same way other major donors have. In a 2011 blog post, Cari Tuna, Good Ventures’ President, wrote, “I first learned about GiveWell about a year ago while preparing to transition from reporting to working in philanthropy full time. I read about the organization in Peter Singer’s The Life You Can Save and, around the same time, met co-founder Holden Karnofsky through a mutual friend. Right away, I was struck by the rigor of GiveWell’s research, its commitment to transparency and the volume of thoughtful commentary about the nonprofit sector it already had produced in just three years. In April 2011, I joined GiveWell’s board. Since then, I’ve been increasingly impressed by the co-founders’ dedication to their work, humility about what they know and what they don’t, and ability to adapt the GiveWell model as they learn.”

After finding us, the first step most people take is, in one way or another, vetting the quality of our work. Some vet us intensively by reading the GiveWell website extremely carefully, following our blog, corresponding with us directly via email or phone, and participating in in-person research discussions (which we hold periodically in New York and San Francisco) before giving making a significant (for them) donation (i.e., giving the amount they eventually intend to give on an annual basis). Others decide to give significantly relatively soon after finding us (though in most cases report doing substantial vetting of our work before they do so).

We considered the donor patterns for our 50 largest donors (again excluding Good Ventures). We stopped at 50 solely in the interest of saving time and because as the dollar amounts fell, the overall picture was unlikely to change. We excluded 18 of these 50 donors because (a) we would guess that a change in donation size may have been driven by the fact that their first donation was early in GiveWell’s existence, and the change in donation size was likely a function of GiveWell’s growth rather than their comfort with our work, (b) gave so early that we don’t have easy access to their donation records, or (c) because 2013 was the first year in which they gave so we can’t yet know their “standard” donation size.Of the 32 remaining donors:

• 16 donors representing $4.5 million made significantly larger (>2x) donations in subsequent years than they had in the first year they gave.
• 8 donors representing $1.3 million followed us intensively for at least a year before making any donation.
• 8 donors representing $1.1 million made first donations that were more than half the size of their subsequent donations.

Again, Good Ventures mirrors the pattern followed by our major retail donors. Cari and Dustin met us in February 2011. After that meeting, Cari and Dustin expressed an intention to make a $100,000 grant to support GiveWell, and Cari joined GiveWell’s board and followed GiveWell’s work closely. In December 2011, Good Ventures donated $750,000 to our top two charities at the time. In 2012, Good Ventures contributed $2 million to our top charities, and in 2013, Good Ventures contributed more than $9 million. The most important/successful media we’ve had has come from the same dynamic: i.e., people who learned about/found GiveWell and often took the time to follow us reasonably closely until they were confident in the quality of our work. Some other media has simply come via reporters’ using referrals to find us.

Many of our supporters imagine that exposure is the main bottleneck to our growth – that our appeal speaks for itself, and that the most important challenge we face is making people aware that we exist. But the analysis above tells a different story: proactive outreach has generally done little for us, while strong relationships with enthusiastic supporters (which develop over a significant amount of time) have driven strong growth.We continue to pick “low-hanging fruit” when it comes to outreach, taking opportunities to gain exposure that require relatively little of our time and effort. However, the bulk of our energy goes into making our research as strong as possible and building relationships with the people who rely on it, and we believe this is the most important driver of our past and future growth.

Definitions:

Note that the sources we use for each donor are often not mutually exclusive. For example, many donors who proactively started looking for a source like GiveWell came across Peter Singer’s book, The Life You Can Save; others, had always intended to give significantly but had not started yet. Once they read Peter Singer’s book and learned about us, they started giving more. In the former case, we’ve catalogued the donor as “Proactive” and in the latter as “Peter Singer.” It is also true that Peter Singer has authored several articles in the New York Times and some donors may report either Peter Singer or New York Times if they found us via one of these articles. Peter Singer himself told us that he found GiveWell via our appearance in the New York Times Giving Section in November 2007, which we understand to have been caused by Times’ reporters finding and following our blog.