The GiveWell Blog

Key questions about philanthropy, part 3: Making and evaluating grants

(updated on: ) | by Holden

This post is third in a series on fundamental (and under-discussed) questions about philanthropy that we’ve grappled with in starting a grantmaking organization (see previous link for the series intro, and this link for the second installment). This post covers the following questions:

  • When making a grant, should we focus most on evaluating the strategy/intervention, the leadership, or something else? We think both are very important; for a smaller grant we hope to be excited about one or the other, and for a larger grant we hope to thoroughly assess both. A couple of disanalogies between philanthropy and for-profit investing point to a relatively larger role for evaluating strategies/interventions, relative to people. More
  • For a given budget, is it best to make fewer and larger grants or more numerous and smaller grants? We currently lean toward the former. Most of the grants we’ve made so far are either (a) a major grant that we’ve put major time into or (b) a smaller grant that we’ve put less time into, in the hopes of seeding a project that could raise more money down the line. More
  • What sort of paperwork should accompany a grant? Funders often require grantees to complete lengthy writeups about their plans, strengths, weaknesses, and alignment with funder goals. So far, we’ve taken a different approach: we create a writeup ourselves and work informally with the grantee to get the information we need. We do have a standard grant agreement that covers topics such as transparency (setting out our intention to write publicly about the grant) and, when appropriate, research practices (e.g. preregistration and data sharing). More
  • What should the relationship be between different funders? How strongly should we seek collaboration, versus seeking to fund what others won’t? It seems to us that many major funders greatly value collaboration, and often pursue multi-funder partnerships. We don’t fully understand the reasons for this and would like to understand them better. Our instincts tend to run the other way. All else equal, we prefer to fund things that are relatively neglected by other funders. We see a lot of value in informal contact with other funders – in checking in, discussing potential grants, and pitching giving opportunities – but a more formal collaboration with another staffed funder would likely introduce a significant amount of time cost and coordination challenges, and we haven’t yet come across a situation in which that seemed like the best approach. More
  • How should we evaluate the results of our grants? Of all the questions in this series, this is the one we’ve seen the most written about. Our approach is very much case-by-case: for some grants, we find it appropriate to do metrics-driven evaluation with quantifiable targets, while for others we tend to have a long time horizon and high tolerance for uncertainty along the way. More

Change of leadership at Evidence Action

| by Natalie Crispin

Evidence Action — which runs the Deworm the World Initiative, one of GiveWell’s top charities — announced today that Alix Zwane will be stepping down as Executive Director on August 3. She is leaving to join the Global Innovation Fund as CEO. Laliteswar Kumar, currently Director, Africa Region, will serve as Interim Executive Director. Dr. Zwane expects to remain involved in the organization until August. Evidence Action aims to identify a new Executive Director within a few months.

Dr. Zwane’s departure does not change our recommendation of the Deworm the World Initiative and we would guess that it will not be a significant factor in our view of the Deworm the World Initiative in the future. Our recommendation is largely based on the strength of evidence and cost-effectiveness of its program and its track record of carrying out that program.

If this change has more of an effect on our funding recommendations than we expect, this will likely be due to one or more of the following factors:

  • We have limited experience with changes in senior leadership at our top charities. All of our other current top charities are led by the organizations’ founders. It is possible that the new Executive Director will have a different vision for the organization or may be unable to generate similar results.
  • Strong communication with each of our top charities is a key part of our research process. We have found Dr. Zwane particularly easy to communicate with. Although we have had substantial communication with other staff, much of our communication with the Deworm the World Initiative, particularly around issues related to room for more funding, has been with her. It is possible that communicating with other staff will not be as smooth and could lead to lower confidence in the Deworm the World Initiative’s work.
  • Evidence Action’s new Executive Director may have a different approach to transparency. Evidence Action has been highly transparent to date, a quality which we have found to be relatively rare among charities. Dr. Zwane told us that she does not expect Evidence Action’s approach to transparency to change.
  • We would not be surprised if Evidence Action fails to identify a new Executive Director within a few months. This search, particularly if it takes a while, could distract from oversight of current programs and planning for the future.

Overall, our impression is that Dr. Zwane has been a highly effective leader of Evidence Action and her departure risks disruptions that could lead to us changing our view of the organization, though we would guess that this will not be the case.

In addition to recommending the Deworm the World Initiative, we have also recommended that Good Ventures provide funding for Evidence Action Beta, with the goal of supporting the development of new top charities (e.g., a planning grant and a grant for a seasonal income support project).

Dr. Zwane’s departure may have more of an effect on our work with Evidence Action Beta, where all of our communication to date has been with her, where the track record is more limited, and where our positive view of Dr. Zwane’s leadership plays a larger role in our confidence in the program.

Finally, the Global Innovation Fund is an organization that aims to “invest in social innovations that aim to improve the lives and opportunities of millions of people in the developing world” and has significant resources (at least $200 million over the next five years) at its disposal. We are excited about its future under Dr. Zwane’s leadership.

Geomagnetic storms: Using extreme value theory to gauge the risk

(updated on: ) | by David Roodman

This is the third post in a series about geomagnetic storms as a global catastrophic risk. A paper covering the material in this series was just released.

My last post examined the strength of certain major geomagnetic storms that occurred before the advent of the modern electrical grid, as well as a solar event in 2012 that could have caused a major storm on earth if it had happened a few weeks earlier or later. I concluded that the observed worst cases over the last 150+ years are probably not more than twice as intense as the major storms that have happened since modern grids were built, notably the storms of 1982, 1989, and 2003.

But that analysis was in a sense informal. Using a branch of statistics called Extreme Value Theory (EVT), we can look more systematically at what the historical record tells us about the future. The method is not magic—it cannot reliably divine the scale of a 1000-year storm from 10 years of data—but through the familiar language of probability and confidence intervals it can discipline extrapolations with appropriate doses of uncertainty. This post brings EVT to geomagnetic storms.

For intuition about EVT, consider the storm-time disturbance (Dst) index introduced in the last post, which represents the average depression in the magnetic field near the equator. You can download the Dst for every hour since 1957: half a million data points and counting. I took this data set, split it randomly into some 5,000 groups of 100 data points, averaged each group, and then graphed the results. Knowing nothing about the distribution of hourly Dst—does it follow a bell curve, or have two humps, or drop sharply below some threshold, or a have long left tail?—I was nevertheless confident that the averages of random groups of 100 values would approximate a bell curve. A cornerstone of statistics, the Central Limit Theorem, says this will happen. Whenever you hear a pollster quote margins of error, she is banking on the Central Limit Theorem.

Key questions about philanthropy, part 2: Choosing focus areas and hiring program staff

(updated on: ) | by Holden

This post is second in a series on fundamental questions about philanthropy that we’ve grappled with in starting a grantmaking organization (see link for the series intro). In this post, we discuss the following questions:

  • Should a funder set explicit focus areas, and if so, how should they choose the focus areas? We believe it generally makes sense to declare focus areas (i.e., causes that one plans to focus on). While it’s common for philanthropists to choose focus areas based on preexisting personal passions, we are taking a different approach. This choice is arguably the most important a funder makes, and we are trying to be strategic about it. More
  • How many focus areas should one work in? We’re highly undecided about this question. We see most funders working in relatively few focus areas, with a high level of depth and expertise. But we also see some appeal in the idea of pursuing more breadth at the cost of depth. More
  • What does a program staffer do on a day-to-day basis? We use the term “program staff” to refer to the staff who have primary responsibility for finding and evaluating giving opportunities (though not necessarily making the final grant decision). One clear part of their role is checking in on existing grants. But it’s less obvious what the most effective activities are for finding new giving opportunities. Our sense is that the most valuable activities include articulating and refining one’s grantmaking priorities, as well as networking. It strikes us that these activities may often lead to higher returns than running an open-ended grant application process. More
  • What sort of person makes a good program staffer? We are still forming our views on this question. Some qualities we think are generally important include strong alignment and communication with the people who will ultimately be responsible for grantmaking decisions (e.g., any executives and funders working with the program staffers); a strong sense of one’s strengths and weaknesses; and strong interpersonal skills. More

Has violence declined, when large-scale atrocities are systematically included?

(updated on: ) | by Holden

Note: I wrote the following on my personal time, then cleaned it up slightly for public consumption. This post is not directly related to GiveWell’s work, but we thought readers might find it interesting anyway. It provides a simple supplementary analysis to the argument presented in The Better Angels Of Our Nature that violence has declined over time. I conclude that the book’s big-picture point stands overall, but my analysis complicates the picture, implying that declines in deaths from everyday violence have been significantly (though probably not fully) offset by higher risks of large-scale, extreme sources of violence such as world wars and oppressive regimes.

Thanks to Steven Pinker for reviewing a draft of this post.

One of my favorite nonfiction books is The Better Angels of our Nature by Steven Pinker. It argues that “violence has declined over long stretches of time, and today we may be living in the most peaceable era in our species’ existence … it is an unmistakable development, visible on scales from millennia to years, from the waging of wars to the spanking of children.” For the most part, I think the book is quite convincing on this point.

This post focuses on what I see as the biggest missing piece of its analysis. The major large-scale atrocities of the 20th century – particularly the two World Wars and the regimes of Josef Stalin and Mao Zedong – stand as an obvious challenge to the book’s theme of declining violence over time. Better Angels does address these events, arguing that they are not as historically anomalous as they may seem. However, the book does not give a comprehensive, quantified picture of how recent centuries compare to older ones in terms of total deaths from such large-scale atrocities. It also does not compare the relative death toll of large-scale atrocities to that of other sources of violent deaths it discusses (homicide, witch hunts, executions, etc.) to determine whether the atrocities of the 20th century were violent enough to offset other kinds of improvements. While many critics have highlighted the atrocities of the 20th century, I don’t believe any of them have done this sort of analysis either, with the exception of a partial analysis on the Uncommon Descent blog.

Using some of the data cited in Better Angels, I’ve done a simple analysis to lay out estimated “deaths from major atrocities” for each century, going back to the 5th century BC. I’ve also looked a bit into how these figures would look if we included deaths from everyday violence as well. Having done this, four points stand out:

  • There are two other centuries (13th and 17th) that look to have been at least as bloody as the 20th, though this observation is very sensitive to very imprecise death toll estimates of a very small number of atrocities. (“Bloody” here refers to high violent deaths per capita per year; “atrocity” means an enormous large-scale mass killing, like a war or conquest or democide.) The 13th century death toll comes almost entirely from estimates of the damage done by Genghis Khan, while the 17th century death toll comes mostly from estimates about the fall of the Ming Dynasty. I don’t see a clear trend overall on “death risk from large-scale atrocities” from the 13th through 20th centuries.
  • Prior to the 13th century, it looks like per-century death tolls from the largest atrocities were consistently lower, and I doubt that this is an artifact of the data.
  • Around the 15th century, a documented fall in the homicide rate seems to have started. The homicide rate decline and the rise in deaths from very large-scale atrocities that took place between the 13th and 15th centuries seem to be in the same ballpark as each other, consistent with the idea that violence shifted from individuals to regimes. I would guess that the net effect was a decline in violent deaths, especially when bearing certain issues with the data in mind, but it isn’t clear.
  • Large-scale atrocities account for enormous numbers of violent deaths. While Better Angels describes multiple trends, it does not compare them to each other in an apples-to-apples way. My sense is that large-scale atrocities account for far more violent deaths than most of the other sources of violence the book discusses – so the lack of a positive trend means that the overall global risk of dying from violence may not have improved greatly over time (though it probably has improved). To make this point vivid, the global rate of violent deaths from the “big four” atrocities alone (two World Wars, regimes of Josef Stalin and Mao Zedong) – spread out over the entire 20th century – is ~50 violent deaths per 100,000 people per year; that’s comparable to the very worst national homicide rates seen today, whereas the homicide rate for high-income countries such as the U.S. tends to be less than 1/10 as high. In other words, the two World Wars + Stalin and Mao alone were enough to make the 20th century as a whole more dangerous than homicide makes today’s homicide-heaviest countries, and they were enough to offset the benefit of the European homicide rate decline that Better Angels describes from Medieval times through the Enlightenment.

Looking purely at quantified violent death risk by century, the picture that emerges from these figures is one of falling everyday violence that is significantly (though probably not fully) offset by higher risks of large-scale, extreme sources of violence such as world wars and oppressive regimes. The net impact is probably lower levels of violence, but it’s not entirely clear. The key transition looks like it was around the 13th-15th centuries; I don’t see much reason to think that the Scientific Revolution should bear much blame for rising atrocity tolls (the timing doesn’t work), but the “rule of law and rising power of governments” dynamic that Better Angels credits for much of the decline in everyday violence could be argued to have had a significant cost in terms of rare mass atrocities.

The dynamics of violent deaths discussed above are consistent with a picture of modernization as improvement in everyday conditions, accompanied by larger rare catastrophic events. This picture can be applied to more recent times as well, even as death tolls from atrocities have fallen: everyday peacefulness has continued to improve, but the potential maximum damage of global catastrophic risks (such as power grid failures, natural and engineered pandemics, climate change and artificial intelligence) seems to be on the rise as well. Today, the potential bad news is even more unlikely and infrequent, but potentially even more enormous, than ever before.

Taking a more holistic view – looking at ways in which non-fatal violence has declined, the phenomenon of the “long peace” since the mid-20th century, and other improvements over time – I think it remains the case that the modern world has become greatly less violent, as well as a better place to live in other ways. I do think that the overall point of Better Angels stands with my analysis in mind, though there is some added complexity to it.

Finally, I note that the literature on this topic appears extremely thin. Steven Pinker is not a historian, yet I believe his systematic examination of historical trends in violence is the first of its kind. Many critics of Better Angels highlight the question of how 20th century atrocities compared to past atrocities. However, I’ve seen only one critic who did either of the following: (a) spelled out a more systematic comparison Pinker could have done; (b) performed a rough version of this comparison. This critic was Uncommon Descent, a blog whose main purpose appears to be arguing for Intelligent Design.

Details follow. From this point on I abbreviate Better Angels as BA.

  • I go through BA’s discussion of the major atrocities of the 20th century, and discuss why I believe more analysis is called for. BA’s argument and the need for more analysis
  • I discuss my own rough attempt to make these comparisons, and what it shows: a lack of clear trend in deaths from large-scale atrocities from the 13th through 20th centuries, a smaller death toll from large atrocities but a higher toll from homicides prior to the 15th century, and the relative importance of large-scale atrocities vs. other sources of violence. More: My analysis
  • I reflect on how one should think about long-term historical trends in violence and quality of life with these corrections in mind. Reflections

Geomagnetic storms: History’s surprising, if tentative, reassurance

(updated on: ) | by David Roodman

This is the second post in a series about geomagnetic storms as a global catastrophic risk. A paper covering the material in this series was just released.

My last post raised the specter of a geomagnetic storm so strong it would black out electric power across continent-scale regions for months or years, triggering an economic and humanitarian disaster.

How likely is that? One relevant source of knowledge is the historical record of geomagnetic disturbances, which is what this post considers. In approaching the geomagnetic storm issue, I had read some alarming statements to the effect that global society is overdue for the geomagnetic “Big One.” So I was surprised to find reassurance in the past. In my view, the most worrying extrapolations from the historical record do not properly represent it.

I hasten to emphasize that this historical analysis is only part of the overall geomagnetic storm risk assessment. Many uncertainties should leave us uneasy, from our incomplete understanding of the sun to the historically novel reliance of today’s grid operators on satellites that are themselves vulnerable to space weather. And since the scientific record stretches back only 30–150 years (depending on the indicator) and big storms happen about once a decade, the sample is too small to support sure extrapolations of extremes.

Nevertheless the historical record and claims based on it are the focus in this and the next post. I’ll examine four (kinds of) extrapolations that have been made from the record: from the last “Big One,” the Carrington event of 1859; from the July 2012 coronal mass ejection (CME) that might have caused a storm as large if it had hit Earth; a more complex extrapolation in Kappenman (2010); and the formal statistical extrapolation of Riley (2012). I’ll save the last for the next post.