The GiveWell Blog

Evidence of impact for long-term benefits

(updated on: ) | by Holden

We’ve recently published our updated review on the evidence on cash transfers. It elaborates on a claim we’ve made previously – that there is evidence for long-term benefits from cash transfers at high average rates of return.

Some people have expressed skepticism of this evidence, pointing to several limitations: there are not many studies, some of the key data comes from people’s reports of their own spending, and programs studied may not be representative of GiveDirectly‘s program.

We think that some of these limitations are less concerning than they may appear at first glance. More importantly, though, the same limitations broadly apply to the evidence of long-term impact (aside from bednets’ impact on mortality) for our recommended health interventions. The situations are not exactly analogous, but the question of which interventions have stronger vs. weaker evidence for long-term impact (aside from bednets’ impact on mortality) does not have an obvious answer.

We speculate that individual donors instinctively imagine that the evidence around many programs is more robust than it is. (We know that we did so when we first started GiveWell.) If this is the case, we’re glad that our recommendation of a cash transfer program – which many people find intuitively unappealing – has prompted some of our followers to take a closer, more skeptical look at this evidence.

Determining that a given intervention – whether health, cash transfers, or anything else – has long-lasting impacts on quality of life is extremely challenging for a multitude of reasons. It requires researchers to track the same people for a long period of time, to collect accurate and relevant data from these people that can shed light on their quality of life. It requires both funders and researchers to have substantial patience and foresight, making plans 5-10 years in advance (and a lot can change in 5-10 years). As a result, informative data about long-term impacts can be hard to come by, and interpreting such data requires substantial judgment calls.

(An easier form of long-term impact to assess is the impact on mortality. Since most people who make it past their fifth birthday live past age 60, we believe it is relatively safe to equate averting a child’s death to long-term impact. However, different people have different intuitions on how to value averting the death of a child under five vs. improving someone’s long-term quality of life. In addition, while there is strong evidence that bednets avert mortality, the case for deworming rests on life improvement. In this post, we focus on the evidence for long-term life improvement rather than on the evidence for mortality reduction, which is quite robust for bednets.)

Learning about the limitations described in this post has made us (a) more confident that using rigid criteria and definitions of “evidence-based” is the wrong path; (b) more favorably inclined toward interventions that seem to require unusually low burdens of proof (a description that we believe all of our top charities currently fit).

The rest of this post goes into more detail on the limitations to the evidence of long-term impact for cash transfers, and how these compare to the limitations to the evidence for bednet distribution and deworming.

We will discuss the relative cost-effectiveness of cash transfers next week.

Limitation 1: not many studies
The case for the long-term benefits of cash transfers rests largely on one high-quality (randomized) long-term study of conditional cash transfers as well as one high-quality long-term study of unconditional grants to microenterprises. Neither of the programs studied is exactly like GiveDirectly’s program, and both could be have taken place in a substantially different context; this issue is discussed in the following section. This section addresses the simple fact that there are not many studies on the topic.

We have long argued that no one study should be considered a “final word” on the effectiveness of a program, even if the program studied was exactly the same as the program of interest; there are many reasons that a single study might be unreliable, and that its results might fail to hold up upon replication. (For more on this topic, see our discussion of meta-research as well as John Ioannidis’s work on replicability in biomedical research). Thus, having a small number of relevant studies is a significant concern.

However, similar concerns apply in other cases.

The case for the long-term benefits of deworming rests on two studies. One is a high-quality randomized study. The other is a retrospective (non-randomized) examination of a hookworm eradication program in the American South in the early 20th century.

The case for the long-term developmental impacts of insecticide-treated nets includes very little in the way of direct studies: just one retrospective (non-randomized) analysis similar to the second study of deworming mentioned above. However, there are other reasons to be optimistic about the long-term impacts of bednets. One is that bednets have been shown to avert deaths, an impact that can be measured over the short run but has clear long-term significance (though how one ought to value this impact, relative to something like “improved income in adulthood,” is an open question). Another reason is that multiple studies have found substantial short-term health impacts for children under five, and there are studies in a variety of other areas making a case for the connection between under-five health and later-in-life developmental benefits. (We have not written extensively about the latter, though we will do so eventually.)

Even putting aside lives saved, if I had to bet on one intervention to have long-term impacts, I’d bet on nets – though if one rigidly requires top-quality randomized studies of the exact intervention itself, the case for nets is weakest. Regardless, the case for all three interventions is quite limited.

Limitation 2: limited representativeness
The case for the long-term benefits of cash transfers rests on one study of conditional cash transfers (which were made with certain requirements, and which were structured as small recurring transfers while GiveDirectly’s grants are structured as larger one-time transfers) and on one study of grants to microenterprises (which, while made with no strings attached and were made one time only, were targeted specifically at people running microenterprises and were also smaller than GiveDirectly’s transfers).

We don’t believe conditionality is a major issue. In examining the impacts of cash transfers, we have focused on impacts that we feel aren’t plausibly related to the conditions. Data about how people spend their money, and what returns they earn on it, seem unlikely to be driven by the sorts of conditions imposed in these programs, which generally pertain to sending children to school, bringing them in for health checkups, etc. (In fact, we would guess that following conditions would be likely to reduce rather than increase consumption and investment returns for adults, by reducing child labor and/or reallocating time and other resources toward children.) The size and structure of cash transfers may be a major issue, though we would guess (as reasoned in our writeup) that GiveDirectly’s version would be more conducive to higher rates of investment and thus greater long-term returns.

Again, there are similar issues with the evidence for nets and deworming.

  • The key study of deworming was of an annual deworming program in an area with extremely high rates of infection (particularly of schistosomiasis – see our recent post on the matter). Because of the proximity to Lake Victoria and the role El Nino played in the study, we’d guess that most of SCI’s work takes place in areas of much lower infections.Much of SCI’s work involves deworming people less frequently than they were dewormed in the studies (details)
  • The high-quality studies of nets involved unusually intensive programs, with constant replacement and checking up on nets. In some cases, programs were structured quite differently – involving treatment of existing nets (rather than distribution of long-lasting insecticide-treated nets), social marketing of nets (rather than free distribution), etc. As far as we can tell, the conditions of AMF’s distributions are likely to resemble those in the studies in relevant ways (particularly usage of nets), but this isn’t something we have definitively established.

In addition to these differences, there is a broader difference that we feel is quite important: studies took place at very different places and times, and with different populations, which could be particularly significant for economic impacts – the main things we are taking as evidence of long-term impact. This applies to all the studies in question.

Limitation 3: reliance on self-reported data
The studies of cash transfers rely on recipients’ reports of their earnings and/or consumption.

Of the limitations discussed in this post, this is the one we’re least concerned about. We do believe that self-reported data is likely to be highly misleading in certain contexts, such as when it is clear to the person being surveyed what sort of answer the surveyor is hoping for (or when some answers are more socially acceptable than others). We feel this is a valid reason to be skeptical of e.g. GiveDirectly’s data on how people spent their transfers (and GiveDirectly concedes as much). However, the studies of cash transfers take a quite different approach: they randomly assign people to treatment and control groups and perform highly extensive surveys of people in both groups, attempting to quantify consumption and other factors. (To get a sense for how extensive such surveys can be, see GiveDirectly’s survey instrument for its ongoing study; the survey instrument we examined for our reanalysis of the evidence for deworming was similarly extensive.) It generally seems to us to be a consensus among scholars that more complex surveys of this type are more reliable, because it becomes easier to answer straightforwardly than to intuit what sorts of answers are being sought (for example, see our notes on speaking with Richard Cibulskis of the World Malaria Report (DOC)).

The long-term followup on deworming we recently discussed relies on similar survey data (i.e., participants’ self-reports of their earnings). And we also rely on such survey data in estimating the rate at which insecticide-treated nets are used.

Two more considerations regarding the relative strength of the evidence for cash transfers vs. deworming

  • We have recently completed a thorough reanalysis of the main study on deworming, and have not done similar analysis of the studies on returns to cash transfers.

 

 

Revisiting the case for developmental effects of deworming

(updated on: ) | by Alexander

This post discusses our detailed examination (including, with help from the authors, reanalyzing raw data) of the Miguel and Kremer 2004 study on deworming (treating people for parasite infections) as a way to raise school attendance, and a followup study (Baird et al. 2012) on the later-in-life impacts.

Our current #3 charity, SCI, focuses on deworming. Deworming is quite cheap (we estimate that SCI spends ~$0.50 per person treated, including all costs) but the benefits are not as obvious and tangible as those of many other health programs because the parasites treated cause few deaths and their effects may be subtle. (So much so that a recent Cochrane review fails to find statistically significant impacts of population deworming on the outcomes that have been most studied). In our view, the case that deworming is a “good buy” depends heavily on the idea of developmental effects: the possibility that deworming children has a subtle, lasting impact on their development, and thus on their ability to be productive and successful throughout life.

The main evidence for this idea comes from (a) Bleakley 2004, a study of the Rockefeller Sanitary Commission’s campaign to eradicate hookworm in the American South in the early 20th century; (b) a series of studies in Kenya, in which school deworming was rolled out on a purposefully arbitrary (randomization-like) basis, and children who received more years of deworming were compared to children who had received fewer. This post focuses on the latter.

We have long had questions about these studies, relating both to the possibility of publication bias and to possible ways in which the setting of the studies was unrepresentative. This year, we decided to ask the study authors for the data and code behind their studies, so that we could run additional analyses to gain more information about the seriousness of our concerns. The authors graciously shared their data and code and helped us to interpret it.

The full details of our reanalysis are available here. A big-picture summary of our concerns and findings follows.

Is data-mining a concern?
The concern

Data-mining is a form of publication bias, in which researchers look at many possible analyses of the data they collect, and present only the analyses most favorable to the conclusions they’re hoping to find. We were concerned about this issue for the series of deworming studies because

  • The first study (Miguel and Kremer 2004) drew a great deal of attention for its positive result (regarding the positive impact of deworming on school attendance), raising the possibility that researchers had the incentive to declare positive results from subsequent studies as well.
  • Different studies used different definitions of “treatment group,” and emphasized different outcomes (for example, the initial study emphasized school attendance; the second emphasized height; the third emphasized earnings).

What we found

After performing our own analysis, we are less concerned about this issue than we were previously. Changing our definition of the “treatment group” didn’t have much impact on the findings, and while the authors did share some outcomes with us that had not been reported in the paper, there were not a lot of these and they didn’t significantly change the picture.

That said, there do remain some reasons to be concerned about this issue. The authors of the most recent follow-up study (the one emphasizing earnings) shared not only data but also their funding proposal and survey questionnaire with us, and we note that

  • The survey was extensive, and included a lot of information that wasn’t included in the data that researchers analyzed. We weren’t surprised to see that this was the case (the follow-up study aimed to collect a rich data set, not just data customized for the questions it was asking), but it still raises the possibility that bias could have crept into the process of transforming raw survey answers into analyzable data.
  • The funding proposal expressed an interest in a wide variety of outcomes, including educational attainment, labor market outcomes (measurable in multiple ways, though the authors stated to us that one of the statistically significant positive effects described in the paper, labor market earnings, is the “canonical” measure for the field of labor economics), cognitive performance, happiness, health measures, and more; it did not clearly declare that any particular one was the primary outcome of interest.We attempted some rudimentary analysis of whether this fact could have facilitated spurious findings, and didn’t find strong reason to think that the findings were spurious. However, we struggled with the question of how a study’s findings should be adjusted for the fact that it had a larger universe of multiple outcomes that it could have chosen to emphasize; normal statistical adjustments for multiple comparisons do not perform well in cases with large numbers of ambiguous outcomes.

We feel that the issues above would have been much easier to resolve if the authors of the studies had preregistered their studies, declaring in advance what their primary and secondary outcomes and analyses of interest were. This is not to say that the authors have done anything unusual or wrong; our understanding is that preregistration was extremely rare (perhaps nonexistent) in economics at the time the study was done. In fact, Ted Miguel, one of the authors of the study, has gone on to co-author one of the first studies we’ve seen in this field that does utilize (and discuss) preregistration. Our point here is just that the practice of preregistration carries substantial credence benefits with us as consumers of research, and would affect our qualitative assessment of these findings.

How well would the studies generalize to other settings?
The area in which the deworming experiment was conducted had unusually high infection rates; in fact, infection rates rose substantially over the course of the study, as shown in Tables II and V of Miguel and Kremer 2004:

Measure Year 1 Prevalence Year 2 Prevalence
Moderate to heavy schistosomiasis infection 7% 18%
Moderate to heavy hookworm infection 15% 22%
Moderate to heavy roundworm infection 16% 24%
Moderate to heavy whipworm infection 10% 17%

The above table likely substantially understates the degree of change, because the second-year figure includes the benefits of treatment externalities experienced by the control group (discussed above). A calculation sent to us by the authors implied that the 18% prevalence of moderate to heavy schistosomiasis infection in the control group in year 2 shown above should be augmented by the 22 percentage point externality effect of treatment to get a genuine counterfactual infection rate of 40% – despite the fact that the initial prevalence was (as shown above) only 7%. This implies that without the program, the area would have seen an extreme rise in prevalence of moderate-to-heavy schistosomiasis infections. A footnote in Miguel and Kremer 2004 attributes this phenomenon to “the extraordinary flooding in 1998 associated with the El Niño weather system, which increased exposure to infected fresh water (note the especially large increases in moderate-to-heavy schistosomiasis infections), created moist conditions favorable for geohelminth larvae, and led to the overflow of latrines, incidentally also creating a major outbreak of fecal-borne cholera” (Pg 174).

Because of this unusual situation, we worry that the results of studies from this place and time may not generalize well to other circumstances in which rates are at lower, more typical levels.

What we found

We did a couple of analyses to see whether the headline effects were sensitive to the prevalence of moderate-to-heavy infections, particularly schistosomiasis infections. As we expected, we did see some reason to believe that deworming had had larger impacts in higher- than in lower-prevalence areas, and that it had had larger impacts for schools with substantial schistosomiasis prevalence. That said, what we found was far from sufficient to completely explain away the studies’ findings. In particular, dividing the schools into those that did and didn’t have substantial schistosomiasis prevalence left us working with fairly small sample sizes, from which it is difficult to conclude anything.

Overall, we are moderately less concerned about this issue than we were before.

Other alternative hypotheses
The findings of the studies discussed here – particularly the follow-up showing substantially improved earnings, resulting from just a couple of rounds of additional deworming treatment in childhood – have always struck us as quite counterintuitive and surprising. As such, we’ve been particularly attentive to alternative explanations of the findings.

One possible alternative explanation is that parents/students may have sought to switch into the schools that “won” early deworming treatments, which could cause the treatment and control groups to differ in ways not picked up by the baseline data measured by the studies. Further discussion with the authors made this concern appear far less likely to us (our understanding is that the deworming program was announced very close to the time when students were registered as “treatment” or “control,” and that school transfers after the start of the program were rare and relatively symmetric between schools).

Another possibility that has become more salient to us in the course of analyzing these studies is that efforts to encourage students to attend school in order to receive treatment might have bled over to later days, increasing attendance in treatment schools over the following years. The particular piece of data that led us to examine this possibility is that within schools, there is no statistically significant difference in attendance rates for treated and untreated students (the effects only appear across schools). (The authors assume that this phenomenon occurred due to the presence of within-school externalities.) In the course of analyzing the studies more closely, we learned that treatment dates were announced at the school in advance in an attempt to boost take-up, and that some efforts were undertaken to boost attendance on drug distribution days.

Finally, we wondered whether the results of these studies might be driven by a few “outlier” schools, but after the analysis we’ve done of the raw data, we are now convinced that this is not an issue.

Bottom line
By sharing their data, code, and other materials with us, the authors of these studies helped us to perform analyses that ultimately gave us more confidence in their results. Several of the aspects of these studies that most worried us (particularly regarding publication bias) turned out not to be important to the conclusions.

We still have substantial reservations about the studies. Preregistration would have been an additional measure that could have increased our confidence and lowered our concerns. In fact, this is the single case we’ve seen in which preregistration would have had the most influence on our conclusions. Had the authors preregistered hours worked or income amongst those employed (the key metrics showing improvement in Baird et al. 2012) as their main outcome of interest prior to collecting follow-up data, we would have far more confidence in the validity of the findings.

Going forward, field replications (carrying out similar deworming programs, and similar analysis to see whether similar results are obtained) would – in our view – greatly improve the robustness of the evidence.

In our view, the vast majority of aid interventions have almost no rigorous evidence behind them. A very small set of interventions – including LLIN distribution – have a broad, impressive evidence base. Deworming is somewhere in between. The studies discussed here are rigorous, have highly encouraging findings, held up to the best scrutiny we could bring to them. At the same time, many questions remain unanswered. This is one of the areas in which an additional long-term study would have the most effect on our views.

Conference call discussing our top charities, Thu. Dec 6th, 7pm Eastern

| by Natalie Crispin

We put a lot of effort into making our research process and reasoning transparent so that anyone can understand and vet the thinking behind our charity recommendations.

Consistent with this, we will be holding a conference call on Thursday, December 6th at 7pm Eastern to discuss our recently updated recommendations. The call is open to anyone who registers via our online form. Staff will take questions by email and answer them over the conference line. If you can’t make this date but would be interested in joining another call at a later date, you can indicate this on the registration form.

If you’re thinking of giving to one of our top charities this year, or you’re just curious about our thinking, we welcome you to join.

We’ve recently held similar discussions with smaller groups of GiveWell supporters. Audio and transcripts from these are available on the conference call page on our website.

Register for the December 6th GiveWell Conference Call

Our top charities for the 2012 giving season

(updated on: ) | by Holden

Over the past year, we’ve continued to follow and investigate our existing top charities, and we’ve also looked for more outstanding giving opportunities. Today we are announcing our updated top charities:

1. Against Malaria Foundation (AMF). AMF focuses on distribution of insecticide-treated nets to prevent malaria; of all the charitable interventions we know of that have clear room for more funding, this one has the strongest evidence of effectiveness and cost-effectiveness. AMF has outstanding self-evaluation and transparency. It first became our #1 charity in late 2011 and has continued to impress us.

2. GiveDirectly. GiveDirectly’s goal: for every $1 it spends, deliver $0.90 directly into the hands of extremely poor people in the developing world, with no strings attached. We first wrote about GiveDirectly in 2011 when it was just getting off the ground, and we now have enough information from its first year to believe that it is succeeding in this goal, and is an outstanding young organization with extremely strong self-evaluation and transparency.

Our definition of “evidence of impact” includes evidence that “wealth is being transferred to low-income people.” We feel that direct cash transfers face an unusually low burden of proof because the link is so tight between cash transfers and giving additional wealth (and choice/empowerment) to recipients. That said, cash transfers have been more heavily studied than any other non-health intervention we know of. The evidence says that they increase short-term consumption, especially of food, and there is suggestive evidence that they may be invested at very high financial rates of return (e.g. ~20% per year). We are not as confident in the investment returns as we are in the evidence for some other interventions, especially bednets, but we believe that the case for increasing consumption on its own terms is strong. The many high-quality studies on cash transfers also provide little support for common concerns about such transfers (such as the idea that they are spent mostly on alcohol).

Different donors will come to different conclusions on the value of this outcome (directly increasing recipients’ wealth) as compared to health interventions, and cash has some salient strengths (putting choice in the hands of recipients) and weaknesses (potential harms if it is poorly spent) relative to health interventions. For our part, we would guess that our other two top charities’ interventions (taken in isolation from the organizations delivering them) do more good on a per-dollar basis, but we are not confident in this, and we see a strong case for supporting GiveDirectly.

3. Schistosomiasis Control Initiative (SCI). SCI focuses on deworming: treating people for parasitic infections. Deworming is extremely inexpensive (~$0.50 per person treated, including all costs), and there is evidence linking it with substantial developmental benefits (people dewormed in childhood may attend school more and earn more later in life); the evidence is not as strong as for insecticide-treated nets but is still far above what we’ve seen for most charitable interventions. We have at times struggled to understand SCI’s activities and impact, and are not as confident in the organization as a whole as in AMF and GiveDirectly, but overall it is an organization with an impressive track record and a strong commitment to transparency, and we consider it an outstanding giving opportunity.

We have published updated reviews of all three charities, as well as audio and a transcript from a conference call we did last week, at which we discussed our preliminary thinking about how to rank these charities with a group of especially involved GiveWell followers. We also give some initial thoughts in this post on the relative strengths and weaknesses of the three charities.

We plan to publish substantial additional content in the coming weeks that will give more context for the thinking behind our rankings. This will include updated reports on insecticide-treated nets, deworming and cash transfers, and blog posts going into more detail on how we see the major considerations (in particular, how the different interventions compare on cost-effectiveness and how we respond to common objections/concerns regarding cash transfers).

This year, more than in past years, our top charities have very different strengths and weaknesses, and we see substantial room for individual donors’ judgment calls in how much to give to each. We encourage donors to read on and review our take on this question. The rest of this post will (a) discuss what we’ve done over the last year that has led us to our current rankings; (b) lay out what we see as the major considerations regarding which of the above three charities to give to.

Our work over the past year
Over the past year, we’ve done the following:

  • Continued to follow AMF and SCI, which were our top two charities as of December 2011. We published the following updates:

    Both made progress and continued to be highly transparent. We had an easier time understanding AMF’s activities (and the impact of GiveWell-sourced donations) than understanding SCI’s activities.

  • Continued to investigate key questions about insecticide-treated nets and deworming.
    • Regarding insecticide-treated nets, we did a major internal vetting of our work and published a post revisiting the cost-effectiveness of this intervention; we also wrote about the possibility of insecticide resistance. Our big-picture stance on this intervention has not changed much – we still consider it an intervention with outstanding evidence of effectiveness and cost-effectiveness – though we have noted some new concerns and caveats.
    • Regarding deworming, we published a post about a new systematic review of the evidence on deworming’s benefits, and we have a post forthcoming revisiting one of the major studies arguing that deworming has developmental benefits (i.e., that it leads to better outcomes later in life). We now find the short-term case for deworming’s health benefits to be weaker than we found it to be in 2011, and we find the long-term case for its developmental benefits to be stronger.
  • Kept tabs on other promising organizations and did a thorough investigation of GiveDirectly. We have considered GiveDirectly promising since its launch, but only recently did we feel that it had enough of a track record to facilitate a deep assessment. In early October of this year, we determined that it was the most promising contender for a top-charity slot (aside from AMF and SCI); we completed a thorough investigation including multiple rounds of reviewing documents and interviewing its CEO; a more thorough investigation of the academic literature on cash transfers (writeup forthcoming); and a multiple-day visit to see GiveDirectly’s operations in Kenya.
  • Looked for other outstanding giving opportunities in the area of global health and nutrition. As discussed earlier this year, we focused on finding ways to deliver the most promising interventions, rather than repeating our earlier approach of scanning a large list of charities. The two most promising interventions we looked into, immunizations and salt iodization, do not appear to have the sort of room for more funding that our top charities do (i.e., we do not see opportunities to translate additional dollars directly into additional people reached). We will be writing more about this in the future.

How much should you give to each charity?
We don’t believe in “diversifying” donations to charity, for the sake of “reducing risk” – we believe in giving in order to maximize the “expected total good accomplished,” which – by default – means finding the best giving opportunity and allocating it 100% of one’s charitable dollars. However, we do see legitimate reasons to divide one’s donation:

  • If you are giving a large enough amount of money, it’s possible to hit diminishing returns by giving it all to one charity. A simple example is that if I were giving $1 billion this year, I wouldn’t give it all to AMF, because that amount would well exceed AMF’s room for more funding.
  • A more subtle version of this idea pertains to learning opportunities. In a sense GiveWell is like a “large donor” with a few million dollars of anticipated money moved. If we direct major funding to more than one charity, we will have improved access to each such charity and will have improved opportunities to track its progress and learn from it. In addition, though we don’t anticipate moving enough money to overwhelm any of the three charities’ room for more funding, there is an argument that each marginal dollar means less to the charity in terms of improving its prominence, ability to experiment and plan, probability of turning out not to be able to scale further, etc.
  • For donors who think of themselves as giving not only to help the charity in question but to help GiveWell, we encourage allocating your dollars in the same way that you would ideally like to see the broader GiveWell community allocate its dollars. If every GiveWell follower follows this principle, we’ll end up with an overall allocation that reflects a weighted average of followers’ opinions of the appropriate allocation. (By contrast, if every GiveWell follower reasons “My personal donation won’t hit diminishing returns, so I’ll just give exclusively to my top choice,” the overall allocation is more likely to end up “distorted.”)

We’ve polled GiveWell staff and core supporters on how they plan to allocate their own funds, and after considering these subjectively, we have settled on a recommended allocation of 70% to AMF, 20% to GiveDirectly, and 10% to SCI. However, we believe there is substantial room for judgment calls and value disagreements here, and we expect to see many individuals deviate significantly from this allocation due to their judgment calls and values. Our rough picture of the judgment calls and value choices that matter most is as follows.

Program Program evidence Program cost-effectiveness Our confidence in the organization Potential for innovation /upside
Against Malaria Foundation Malaria control Very strong Very strong Very high High
Schisto-somiasis Control Initiative Deworming (treating parasitic infections) Fairly strong Highly uncertain, though may be strongest High Low
Give Directly Direct cash transfers to the very poor Strong on short-term consumption; moderate on investment Highly uncertain; may be strong Very high Very high

For donors who place high value on supporting strong overall organizations (in light of the fact that much of the impact of a donation depends on who’s in charge of allocating it), and/or promoting innovation and experimentation, we suggest a relatively higher allocation to GiveDirectly, and lower to SCI.

  • We believe that GiveDirectly has significant “upside”: it is experimenting with a type of intervention that is extremely rare in the nonprofit world, that it is hoping to experiment with many different approaches to this intervention, and that its approach may ultimately evolve and change significantly and may have a major impact on other parts of the nonprofit world.
  • We believe AMF has significant “upside” as well, though in a different way (and probably to a lesser extent than GiveDirectly). Its intervention is one that has a large community around it and a lot of funding behind it. Through its unusual degree of transparency and self-evaluation, AMF may set examples for others and/or produce information/insights that are helpful to the broader malaria control community.
  • While SCI is an outstanding organization, we do not believe it is as strong as the other two on this criterion.

For donors who are particularly “harm-averse” – such that they place significant weight on the “do no harm” principle as opposed to the “maximize expected good accomplished” principle (we subscribe to the latter) – we suggest a relatively lower allocation to GiveDirectly. Cash transfers (particularly as GiveDirectly structures them) have a higher risk of unintended consequences than the other two interventions. We see very little risk of harm for insecticide-treated nets or deworming.

For donors seeking to save rather than improve lives, we suggest a relatively higher allocation to AMF, the only charity of the three whose intervention has been linked with reduced mortality. For donors seeking to improve rather than save lives, we believe AMF should still get the highest allocation overall–the benefits of reducing the burden of malaria likely go well beyond saving lives. (GiveWell staff tend to prefer improving lives over saving them, and our suggested allocation takes this into account.)

For donors who are inclined to be less skeptical than we are of academic evidence and cost-effectiveness analysis, we suggest a relatively higher allocation to SCI, and lower to GiveDirectly. If one takes the evidence on deworming at face value, it indicates extremely impressive long-term benefits. Similar benefits may exist for reducing the burden of malaria, but deworming is substantially cheaper (~$0.50 per person per year, as opposed to ~$1.25) on a per-person basis.

For donors who are more skeptical than we are of academic evidence and cost-effectiveness analysis, and place a higher weight on the reasoning that “people are likely to make better decisions for themselves than we can make for them,” we recommend a relatively higher allocation to GiveDirectly.

We’ll be very interested to see what people end up doing. We encourage donors to post their planned allocations, and reasoning, as comments (or to email us if they’d prefer that their thinking stay private).

7 tips for giving efficiently

(updated on: ) | by Holden

If you’re planning on giving to charity this holiday season, there are a few simple steps you can take that can save a lot of money – allowing you to give more at the same cost to yourself – as well as reduce hassle.

1. Don’t wait until the last minute. Many donors wait until the very end of the calendar year to give. (Over 20% of our 2011 money moved, excluding gifts of $100,000 and more, came through in the last three days of the year alone.) Doing this will make it very difficult to execute some of the steps below (such as giving appreciated stock). And if something unexpected happens (as it often does with large credit card donations), you may have little time to react.

We recommend setting a target date of December 24 or earlier for finalizing your gift (for a week’s cushion).

2. Try to get a tax benefit. Details vary by country and personal situation, but a tax deduction can allow you to give much more to charity at the same cost to yourself. (That said, as discussed below, we believe it is more important to give to the most effective possible charity than to get the maximum tax benefit.)

Our #1 charity, Against Malaria Foundation, offers tax deductibility to citizens of the U.S., the U.K., and Canada. Our current #2 charity is a U.K. charity only, but U.S. donors can get a tax deduction for giving to GiveWell for the support of SCI.

If you’re in another country, you may wish to look into groups in your country that provide the service of (a) taking deductible donations themselves; (b) transferring the funds (minus fees) to charities abroad. While they can charge substantial fees, you may still come out ahead after tax considerations.

3. Avoid the large transaction fees and delays associated with large online donations. When donating via credit card, you will almost always be charged standard credit card processing fees. For donations under $5000, our feeling is that it’s worth paying the fees to avoid the hassle (both for you and the charity) of another donation method. But fees on a $5000 donation will generally exceed $100. In addition, credit card transactions of $5000 and up are often flagged by credit card companies (though a call to the credit card company can generally resolve the situation quickly).

This year, Google Checkout stopped offering fee-free processing, and we were unable to find a viable alternative with lower fees. Therefore, we are recommending that people giving $5000 and up consider writing a check or making a bank transfer. (Details)

4. Give appreciated stock. In the U.S., if you give stock to a charity, neither you nor the charity will have to pay taxes on capital gains (as you would if you sold the stock yourself). If you have stock that you acquired for $1000 (and has a cost basis of $1000) but is now worth $2000, you can give the stock to charity, take a deduction for $2000, and not have to pay capital gains tax on the $1000 of appreciation. This can result in significant savings. (More at Vanguard’s write-up on this topic.)

Taking advantage of this requires having some of your net worth invested in appreciated stock (or other securities), knowing (or being able to obtain) the cost basis of such securities, and arranging to transfer the stock directly to a charity, which is generally a fairly easy process, but varies from broker to broker; waiting until December 31 can be especially problematic if you’re trying to do this.

GiveWell can take direct transfers of stock for the support of our top charities, so let us know if you’re interested in this. A more robust way to smooth the process of giving appreciated securities is to take advantage of a donor advised fund, discussed immediately below.

5. Look into donor-advised funds to make the process smoother and more consistent year-to-year. Donor advised funds allow donors to make a charitable donation (and get a tax deduction) now, while deciding which charity they’d like to support later. The donation goes into a fund that is “advised” by the donor, and the donor may later recommend a grant from the fund to the charity of his/her choice.

We see a couple of advantages to this setup. One advantage is that you can separate your “decision date” (the date on which you decide which charity you’d like to support) from your “transaction date.” That means that if you aren’t ready to decide which charity to support yet, you can still get started on the process of transferring funds and getting a tax deduction for the appropriate year. Another advantage is that if you change the charity you support from year to year, you’re still working with the same partner when it comes to transactions, so the process for e.g. donating stock will not change from year to year. Donor advised funds are often set up to easily accept donated stock, whereas charities may or may not be.

Many large investment companies – Vanguard, Fidelity, Schwab – offer donor advised funds. They generally charge relatively modest management fees.

We do not recommend delaying this year’s donation while you look into donor advised funds. Donor advised funds can make giving more convenient over the long term, year-to-year, but you can get all the advantages of efficient giving this year without one. We also maintain our own donor-advised fund for donors interested in supporting our recommended charities.

6. Consider the political environment. If you believe that your tax rate is likely to be higher in 2013 than in 2012, you may wish to give at the very beginning of 2013 rather than the very end of 2012.

We’ll likely be counting donations made in the first week or so of 2013 toward our 2012 money moved (we’ve made similar adjustments in the past).

7. Choose your charity wisely. Saving money on taxes and transaction fees can be significant, approaching or exceeding a 50% increase in the amount you’re able to give. However, we believe that your choice of charity is a much larger factor in how much good your giving accomplishes.

Our charity recommendations make it possible to support outstanding, thoroughly vetted organizations – which we’ve investigated by reviewing academic evidence, interviewing staff, analyzing internal documents, conducting site visits, assessing funding needs, and more – without needing to do your own research. We publicly publish the full details of our process and analysis, so you can spot-check whatever part of our work and reasoning you’d like to.

Final notes.

  • If you support our recommended charities (on the basis of our recommendation) but you don’t give through our website, please use the donation report form to let us know about your gift; doing so helps GiveWell track our impact.
  • We believe that even when dealing with a relatively complicated gift (for example, a gift of stock), it’s possible to give quite quickly and with only minor hassle. The much more difficult challenge is choosing a charity, and we’ve tried to make that easy as well. So we hope you’ll give this season, even if you’re just starting to think about it now; if you’d like more advice or help, don’t hesitate to contact us.

GiveWell’s annual charity recommendations refresh

| by Elie

As we do every year, we’re planning to release updated charity recommendations by December 1st. We’re currently weighing what we’ve learned about the organizations that could receive our top ratings. Currently, the contenders for top spots are Against Malaria Foundation (AMF) (our current #1 charity), Schistosomiasis Control Initiative (SCI) (our current #2 charity), and GiveDirectly (which we have written about before, but haven’t previously recommended due to its newness; now that it has more of a track record we are taking a closer look).

As part of this work, we’ve recently released updates on AMF (latest update here) and SCI (latest update here). I also recently returned from a trip to Kenya to see GiveDirectly’s work in the field, and we are working on an updated review of GiveDirectly that incorporates all we’ve learned recently.