David Roodman’s draft writeup on the mortality-fertility connection

We’re sometimes asked whether life-saving interventions (such as insecticide-treated net distribution) risk leading to overpopulation. A common response is that (as argued by Hans Rosling of Gapminder and the most recent Gates Foundation annual letter, among others) the reverse dynamic holds: population growth tends to slow as child mortality declines, possibly because parents feel less need to have multiple children in order to hedge against the risk of death.

Empirically, it appears generally true that lower child mortality tends to correlate with slower population growth. However, one might ask whether this relationship is causal (i.e., falling mortality leads to falling fertility, for reasons outlined in the previous paragraph) or merely correlational (i.e., falling mortality generally accompanies a host of factors related to general economic development, and including falling population growth). This question is important when assessing the impact of a life-saving intervention in isolation: it’s possible that health interventions, unaccompanied by improvements on other dimensions, do in fact lead to higher populations.

We’ve long wondered about this question, and we recently commissioned David Roodman – whom we have been impressed with in the past – to examine the rather extensive and complex literature on it. David has recently posted a draft of his writeup, and is seeking feedback.

We haven’t yet fully vetted the writeup, but it appears to me to give a deep and balanced examination of the available literature, along with interesting discussion of the general challenges of isolating causality in cross-country studies. Its conclusion:

I think the best interpretation of the available evidence is that the impact of life-saving interventions on fertility and population growth varies by context, above all with total fertility, and is rarely greater than 1:1 [rarely enough for the fertility decline to fully offset the mortality decline in terms of effect on population size]. In places where lifetime births/woman has converging to 2 or lower, family size is largely a conscious choice, made with an ideal family size in mind, and achieved in part by access to modern contraception. In those contexts, saving one child’s life should lead parents to avert a birth they would otherwise have. The impact of mortality drops on fertility will be nearly 1:1, so population growth will hardly change.

Overall, it appears that life-saving interventions unaccompanied by other improvements, where access to contraception is weak, are likely to lead to some acceleration of population growth. With that said, we wish to note the following:

  • No intervention takes place in isolation, and we expect population growth to slow in the future in most low-income areas as poverty falls.
  • Acceleration of population growth should not necessarily be associated with overpopulation and its connotations of a net decline in standards of living.
  • GiveWell’s current top charities are all focused on improving living standards rather than averting deaths, and Against Malaria Foundation (a former and potentially future top-rated charity) aims to avert deaths and bring about other benefits by reducing the burden of malaria.

We encourage our readers to check out the writeup and send in their thoughts. It’s a thorough review of the question and a generally interesting read.

David Roodman’s draft writeup, “The impact of life-saving interventions on fertility”
 

More on why we commissioned this writeup

We have previously made some preliminary attempts to review the literature on the connection between mortality and fertility. However, we’ve found this literature to be (a) challenging to synthesize, as most of the key papers use relatively complex methodologies that would take significant work to interpret; (b) only a moderate priority for our research, since it has limited relevance to which charities we recommend and how we rank them.

Recently, we decided to explore the idea of working with David Roodman (whom we’ve generally been impressed with) as someone who might increase our capacity to produce thorough reviews of literature on key giving-relevant questions. For our first project together, we chose to work on this question because David had done substantial work on it in the past, and felt he could produce an initial writeup relatively quickly.

Our work on outreach

One of the core questions we’ve faced throughout GiveWell’s history has been whether to allocate our time to research (i.e., identifying and evaluating outstanding giving opportunities) or outreach (i.e., increasing the number of donors who are aware of GiveWell and use our research, and the amount of money we move to top charities). The question of “research vs. outreach” was one of the most common topics at board meetings in our early years, and new staff and supporters often ask us why we don’t focus more of our time on outreach.

Over the course of GiveWell’s history, we’ve taken many approaches to increase awareness of our work. We’ve tried all of these throughout our history, so we believe the experiences described below are reasonably representative of “what we’d expect to work today” rather than specific to a particular point in GiveWell’s development. The purpose of this post is to share our observations of what has worked and what hasn’t, in order to help people understand why we prioritize our time as we do and also in order to potentially help people thinking about their own outreach strategies.

The big picture is that:

  • Proactive “marketing” has contributed little to our growth.
  • Much of our most valuable publicity and promotion has come from enthusiastic people who actively sought us out.
  • Supporters often take serious time, and vet our work, before becoming highly enthusiastic.

Because of this, we have generally felt that improving the quality of our research, and our existing audience’s understanding of it, has been the most important factor in our growth. We’ve consistently seen our best results come from focusing not on “How can we get people to find out about us?” but “How can we get the people who have already sought us out, shown interest and found us to become as enthusiastic as possible?”

More detail follows. We first discuss approaches that have not worked particularly well, then discuss the major factors in the growth we’ve seen.

Approaches that have not worked particularly well

These approaches either (a) have not had strong return-on-investment in terms of money-moved-per-hour-of-staff-time-invested or (b) do not seem likely to yield significantly better results if we put in additional time.

  • Speaking opportunities. We’ve given a total of 50-100 talks – at companies (e.g., tech companies, banks, hedge funds, consulting firms), universities (undergrad, business and public health schools; both specific classes and broader community events), conferences, and smaller, personal events organized by GiveWell supporters. Few donors have found us via a talk. In hindsight, the costs of talks (e.g., practicing our talk, prepping slides, coordinating logistics, travel time) seem to have significantly outweighed the benefits. Over the past year or so, we’ve accepted invitations to speak, but only when (a) we can limit preparation time (e.g., by delivering our standard GiveWell talk rather than putting in serious preparation) and (b) travel time is low. We’ve also sometimes sent more junior staff to speak, partly for their training. Roughly speaking, I’d estimate that talks have yielded ~5-10 major supporters giving a total of approximately $200,000-$400,000/year. My best guess is that we have put in ~250-500 hours throughout our history into speaking opportunities.
  • Networking with people who haven’t shown proactive interest in GiveWell. We’ve conducted hundreds of 1-on-1 meetings or phone calls with prospective donors. As a general rule, when we’ve initiated the meeting (i.e., a supporter of ours suggested we meet someone or we asked a supporter of ours to make an introduction), we’ve very rarely converted that person into a fan. A common pattern is that someone thinks GiveWell is a “great idea” but that this recognition rarely results in action, e.g., giving to GiveWell or our top charities. (This was a topic of discussion at a July 2010 board meeting.) Networking of this type has yielded a few major donors who collectively give ~$100,000-$150,000/year.
  • Proactive media outreach. We’ve made various attempts to proactively encourage the media to cover us. We’ve had no notable success actively reaching out to the media; our media coverage has generally come via media reaching out to us. A fact that has further diminished our interest in allocating time to proactive media outreach is seeing the surprising (to us) results from what we would have considered major media. For example, we’ve seen limited returns – in terms of new donors – from appearances in newspapers like the Boston Globe and highly-trafficked major sites like Lifehacker, Reddit, and Gawker.
  • Google AdWords. We have run AdWords consistently since GiveWell began, and it has consistently been our best performing (in ROI terms) proactive outreach strategy. However, while it has had a strong return on money invested, we have not found a significant number of large donors via AdWords. In 2009 (the latest year for which we have the data), we spent approximately $650 on AdWords during the last week of the year and counted $5,310 donated, see our 2009 metrics report (DOC). In 2013, we ran significantly more AdWords as part of the Google Grants program, leading to 202 donors. (We are not currently tracking the amount given, though we hope to be able to do this in the near future.)

We have also tried other outreach approaches at various times. We have (a) cold-called family foundations and corporate giving departments to encourage them to use our recommendations; (b) retained a professional designer to prepare materials as takeaways from talks and 1-on-1 meetings; (c) evaluated PR firms as a means for gaining broader media exposure. Both (a) and (b) resulted in significant time spent with little-to-no results. When we spoke with PR firms in 2009 and 2012, we found that the media coverage they expected to be able to generate for us was comparable to the media coverage we had already had which had not led to strong results (i.e., many donors/major donors) in the past.

(For those particularly interested in our early attempts at outreach, see our 2008 change of direction writeup.)

GiveWell’s growth

While the approaches described above have not worked particularly well, GiveWell has grown significantly. Some highlights follow; more information is available in our annual and quarterly metrics reports.

Comparing our current reach to that which we had in December 2009 is instructive. By December 2009, we had already appeared in (a) a major New York Times article, (which also led to appearances on CNBC and NPR) and (b) Peter Singer’s book The Life You Can Save (which led to multiple additional media mentions during his book tour), and (c) Nick Kristof’s book, Half the Sky. Notwithstanding these major drivers of “eyeballs” to our website, GiveWell’s reach remained relatively limited.

Another interesting comparison is to our first December, in 2007, when we had an abnormal traffic spike due to being featured in an NYT story that reached #3 on the most-emailed articles list. We saw a total of ~50,000 visitors to our site that month – over 1/3 as many as in December 2013 – but our total money moved that month was only ~$30,000 (compare to more than $4.3 million in December 2013, excluding Good Ventures).

Our growth has been a function of audience enthusiasm and word-of-mouth, much more than of media exposure and attention.

How have donors found GiveWell?

We know the sources of our growth because we have surveyed and prioritized getting to know our major donors.

Most of our larger donors have told us that they either (a) found GiveWell while proactively looking for a resource to help them decide where to give or (b) heard about GiveWell from a trusted source (most commonly Peter Singer, other media, or a close friend or colleague).

Excluding Good Ventures (the single largest supporter of our top charities) and other institutions (which have different approaches to giving), over the past 3 years, 95 donors have accounted for close to 60% of the total non-institutional money GiveWell directed to top charities or received to support its operations. (Combining these two figures offers a reasonable picture of the most committed users/largest supporters of GiveWell’s work.)

We have data on how 84 of these 95 donors found us, which is shown in the table below:

How donor found GiveWell Number of donors Amount given (USD millions)
Proactive 22 $4.9
Peter Singer 21 $2.2
Personal 7 $1.6
New York Times 8 $1.1
Referral 11 $0.6
Marginal Revolution blog 6 $0.4
Other online 5 $0.4
Speaking opportunities 2 $0.2
Networking 2 $0.1
Total 84 $11.4

See this footnote for definitions and more technical discussion.

It is important to note that the success we’ve had when someone refers us (either via writing something or referring friends) has often been a function of specific individuals’ repeatedly and enthusiastically promoting us or discussing our work in public. This includes people in the media (or who frequently contribute in media) such as Peter Singer, Ken Stern, Nick Kristof, Alex Tabarrok and Stephanie Strom but also individuals like Jonah Sinick (who played a large role in introducing the LessWrong community to GiveWell) and Jeff Kaufman and Julia Wise (who have promoted GiveWell to their coworkers, friends and on their respective blogs). In general, we feel that high-enthusiasm, lower-profile promoters have benefited us far more than higher-profile, one-time endorsements.

Good Ventures

We excluded Good Ventures from the above discussion, but its story is important because it is the single largest donor to our top charities (having directed $12 million to those organizations over the past 3 years). Broadly speaking, Good Ventures came to GiveWell in the same way other major donors have. In a 2011 blog post, Cari Tuna, Good Ventures’ President, wrote, “I first learned about GiveWell about a year ago while preparing to transition from reporting to working in philanthropy full time. I read about the organization in Peter Singer’s The Life You Can Save and, around the same time, met co-founder Holden Karnofsky through a mutual friend. Right away, I was struck by the rigor of GiveWell’s research, its commitment to transparency and the volume of thoughtful commentary about the nonprofit sector it already had produced in just three years. In April 2011, I joined GiveWell’s board. Since then, I’ve been increasingly impressed by the co-founders’ dedication to their work, humility about what they know and what they don’t, and ability to adapt the GiveWell model as they learn.”

The importance of growing enthusiasm over time

After finding us, the first step most people take is, in one way or another, vetting the quality of our work. Some vet us intensively by reading the GiveWell website extremely carefully, following our blog, corresponding with us directly via email or phone, and participating in in-person research discussions (which we hold periodically in New York and San Francisco) before giving making a significant (for them) donation (i.e., giving the amount they eventually intend to give on an annual basis). Others decide to give significantly relatively soon after finding us (though in most cases report doing substantial vetting of our work before they do so).

We considered the donor patterns for our 50 largest donors (again excluding Good Ventures). We stopped at 50 solely in the interest of saving time and because as the dollar amounts fell, the overall picture was unlikely to change. We excluded 18 of these 50 donors because (a) we would guess that a change in donation size may have been driven by the fact that their first donation was early in GiveWell’s existence, and the change in donation size was likely a function of GiveWell’s growth rather than their comfort with our work, (b) gave so early that we don’t have easy access to their donation records, or (c) because 2013 was the first year in which they gave so we can’t yet know their “standard” donation size.Of the 32 remaining donors:

  • 16 donors representing $4.5 million made significantly larger (>2x) donations in subsequent years than they had in the first year they gave.
  • 8 donors representing $1.3 million followed us intensively for at least a year before making any donation.
  • 8 donors representing $1.1 million made first donations that were more than half the size of their subsequent donations.

Again, Good Ventures mirrors the pattern followed by our major retail donors. Cari and Dustin met us in February 2011. After that meeting, Cari and Dustin expressed an intention to make a $100,000 grant to support GiveWell, and Cari joined GiveWell’s board and followed GiveWell’s work closely. In December 2011, Good Ventures donated $750,000 to our top two charities at the time. In 2012, Good Ventures contributed $2 million to our top charities, and in 2013, Good Ventures contributed more than $9 million. The most important/successful media we’ve had has come from the same dynamic: i.e., people who learned about/found GiveWell and often took the time to follow us reasonably closely until they were confident in the quality of our work. Some other media has simply come via reporters’ using referrals to find us.

Our takeaways

Many of our supporters imagine that exposure is the main bottleneck to our growth – that our appeal speaks for itself, and that the most important challenge we face is making people aware that we exist. But the analysis above tells a different story: proactive outreach has generally done little for us, while strong relationships with enthusiastic supporters (which develop over a significant amount of time) have driven strong growth.We continue to pick “low-hanging fruit” when it comes to outreach, taking opportunities to gain exposure that require relatively little of our time and effort. However, the bulk of our energy goes into making our research as strong as possible and building relationships with the people who rely on it, and we believe this is the most important driver of our past and future growth.


Definitions:

  • Proactive – the donor told us that s/he was actively searching for a source like GiveWell at the time s/he found us.
  • Personal – the donor knew Elie or Holden personally before they started GiveWell.
  • New York Times – The New York Times has written about GiveWell many times; we have not tried to source donors to specific articles.
  • Referral – the donor told us that s/he found us from a friend or coworker who recommended GiveWell.
  • Other online – the donor told us that s/he found us online but we do not have the exact source or there was only 1 donor who found us via this source.
  • Speaking opportunities – the donor told us that s/he learned about GiveWell via a talk we gave.
  • Networking – we met the donor through general networking (i.e., asking people we knew to introduce us to people they know or people we knew making active introductions to their network).

Note that the sources we use for each donor are often not mutually exclusive. For example, many donors who proactively started looking for a source like GiveWell came across Peter Singer’s book, The Life You Can Save; others, had always intended to give significantly but had not started yet. Once they read Peter Singer’s book and learned about us, they started giving more. In the former case, we’ve catalogued the donor as “Proactive” and in the latter as “Peter Singer.” It is also true that Peter Singer has authored several articles in the New York Times and some donors may report either Peter Singer or New York Times if they found us via one of these articles. Peter Singer himself told us that he found GiveWell via our appearance in the New York Times Giving Section in November 2007, which we understand to have been caused by Times’ reporters finding and following our blog.

Against Malaria Foundation finalizes distribution agreement for ~676,000 nets

The Against Malaria Foundation (AMF) was our top-rated charity for approximately two years, before we suspended our recommendation last November for room for more funding related reasons.

We are happy to see that it has recently finalized an agreement to fund 676,000 nets – at a total cost of ~$2 million – in the Democratic Republic of Congo (DRC).

Our impression is that DRC is a very difficult country to work in; that said, our impression from speaking with AMF is that the agreement has met its high standards for monitoring and evaluation, and that we can expect strong reporting on this distribution. We will be following up and posting updates on the distribution’s progress.

We are not yet reinstating our recommendation of AMF, as it has still not committed the bulk of the funds it has available, but this is an important step forward – the largest single distribution AMF has finalized to date. AMF is still negotiating other potential distributions, and we are hopeful that it will commit the bulk of its funds and return to our top charities list.

The first case study from our history of philanthropy project

Benjamin Soskis, who has been working for us on our history of philanthropy project, has completed a case study (.docx) of the Robert Wood Johnson Foundation-Pew Memorial Trust Health Care for the Homeless (HCH) program. We have found this case study very interesting and believe it to be a helpful addition to the literature on the history of philanthropy and its impacts. Dr. Soskis had previously compiled a literature review for this program (.docx).

In brief, the HCH program consisted of the following:

“In December 1983, the Robert Wood Johnson Foundation (RWJF), in partnership with Pew Memorial Trusts, spurred by a recognition of how little was being done to address the health care needs of the homeless, issued a call for proposals to … develop means of incorporating the homeless into local outpatient health care systems…. Ultimately, the foundations funded 19 coalitions over the course of four years, spending a total of $25 million…. In June 1987, Congress passed the Stewart B. McKinney Homeless Assistance Act, the first major piece of federal legislation to address homelessness in more than half a century…. That the RWJF-Pew program should be granted a significant degree of causal impact in the incorporation of the program into the McKinney Act is undeniable.” (Case Study, Pg 1)

The full case study is available here (.docx).
The full list of sources is available here (.docx).

Dr. Soskis’s case study provides an example of how philanthropy can influence policy change. He writes:

“Incorporation of the HCH program within the McKinney Act was the result of a convergence of a well-designed demonstration project with a policy window opened by a campaign by homeless advocates begun earlier in the decade. The advocates pushed for the federal government to address the mounting crisis of homelessness and when that campaign came to fruition, the HCH program was available as a policy model. The RWJF-Pew HCH program, therefore, does not merely represent a model of a successful demonstration project, but of a particularly powerful convergence between philanthropic initiative and broader political currents.” (Case Study, Pg 3)

The picture that emerges from the case study seems importantly different from the picture we got from a shorter piece in a list of 100 philanthropic success stories. The shorter case made the project’s impact seem relatively more significant, and our interest in this case was partly due to the fact that we’ve been pointed to it repeatedly as one of philanthropy’s most clear-cut success stories. Dr. Soskis’s study does attribute significant impact to philanthropy, but also paints a more complex picture.

This is the first case study Dr. Soskis has completed. He is currently working on a similar study focused on the role played by philanthropy on the 2010 passage of the Affordable Care Act. We hope to share that (and other case studies) in the future.

Read the case study of the Health Care for the Homeless program (.docx)

GiveWell annual review for 2013: details on GiveWell’s money moved and web traffic

This is the final post (of six) we have made focused on our self-evaluation and future plans.

This post lays out highlights from our metrics report for 2013. For more detail, see our full metrics report (PDF).

1. In 2013, GiveWell tracked $17.36 million in money moved to our top charities, and an additional $0.90 million to recommendations made as a part of work on GiveWell Labs. This is a significant increase over past years.

2. GiveDirectly received about 60% of the money moved and our other top charities, Against Malaria Foundation, Schistosomiasis Control Initiative, and Deworm the World Initiative, each received 12-14%. Funding for GiveDirectly was driven in part by $5 million in matching funds from Good Ventures.

3. Growth in money moved was strong for donors giving less than $10,000. Money moved from these donors grew 90% in 2013, from $2.0 to $3.8 million. Unlike in previous years, growth in overall money moved was not driven by the largest donors.

This table excludes Good Ventures and donations for which we don’t have individual information. More in our full metrics report.

4. Money moved by donors giving more than $10,000 fell (by 9%), and fell substantially for those giving more than $50,000 (by 41%). To explain why, we reviewed each $50k+ donor’s situation individually and categorized them based on what we know about their giving decisions in 2013. Overall giving by these donors fell about $1 million. Most of the decrease among these donors was due to donors who decided to wait to give, thinking that giving options could be better in the next few years.

In addition, we asked more donors to support our operational costs and asked donors to support us at a higher level than we had in previous years. Unrestricted funding from individual donors (i.e. excluding foundations), particularly from large donors, grew substantially in 2013 (from $0.58 million to $1.36 million). Unrestricted funding from the $50k+ donors discussed above rose by $152,000.

Overall, we believe the decrease in money moved from large donors is likely to be temporary and is not a reflection of decreased interest in GiveWell. More in our full metrics report.

5. Web traffic continued to grow. Growth in search, direct, Google AdWords, and referral link traffic all contributed to overall growth. Traffic tends to peak in December of each year, circled in the chart below.

6. We surveyed donors who gave $2,000 or more to ask them more about themselves and their giving. The majority of respondents are in their 20s and 30s and work in finance or technology. Like last year, among the most common ways they find GiveWell are through online searches and referral links and through Peter Singer. Personal referral is a growing way in which major donors find GiveWell.

7. As in previous years, the most common response when we asked major donors ‘how has GiveWell changed your giving?’ was ‘I would have given a similar amount to another international charity’ in the absence of GiveWell.

What effect has GiveWell had on your giving?

For donors who responded that GiveWell caused them to reallocate their giving, where would you have given in GiveWell’s absence?

8. GiveWell’s website now processes slightly more than Charity Navigator’s, though it offers far fewer charities as options. (We were unable to measure progress against Guidestar because it changed its processing system in 2013.) This comparison provides evidence that the growth we saw in 2013 is due not to generalized increases in online giving or use of charity evaluators, but rather to GiveWell-specific factors. (Note that the GiveWell figure in this chart includes only what was processed through our website – not all money moved – in order to provide a valid comparison to the others, for which we only have online-giving data.)

GiveWell is hiring

We’re actively hiring because we are capacity constrained. Filling any of the roles below would make a substantial difference to our research.

If you follow GiveWell and want to help us, please share this post with anyone whom you think might be a good fit for the jobs listed below.

  • Research Analyst. Research Analysts are GiveWell’s primary staff and work on all parts of our research process. We hope to add a few entry-level research analysts this year. More
  • Summer Research Analyst. We offer summer positions to students entering their final year in school with the hopes that they will become full-time employees following graduation. More
  • Administrative Assistant. The Administrative Assistant is a 3-day-a-week, part-time job based in our office in San Francisco. The Administrative Assistant helps with duties such as collecting and processing donations made through the mail and proofreading materials before they are made public. More
  • Conversation Notes Writer. Conversation notes are a key part of our research process. Conversation Notes Writers listen to conversations conducted by GiveWell staff and produce summaries. This job is flexible: it can be done from anywhere in the world at any time of day, but we ask for people who can commit at least 10 hours per week. More