The GiveWell Blog

7 tips for giving efficiently

If you’re planning on giving to charity this holiday season, there are a few simple steps you can take that can save a lot of money – allowing you to give more at the same cost to yourself – as well as reduce hassle.

1. Don’t wait until the last minute. Many donors wait until the very end of the calendar year to give. (Over 20% of our 2011 money moved, excluding gifts of $100,000 and more, came through in the last three days of the year alone.) Doing this will make it very difficult to execute some of the steps below (such as giving appreciated stock). And if something unexpected happens (as it often does with large credit card donations), you may have little time to react.

We recommend setting a target date of December 24 or earlier for finalizing your gift (for a week’s cushion).

2. Try to get a tax benefit. Details vary by country and personal situation, but a tax deduction can allow you to give much more to charity at the same cost to yourself. (That said, as discussed below, we believe it is more important to give to the most effective possible charity than to get the maximum tax benefit.)

Our #1 charity, Against Malaria Foundation, offers tax deductibility to citizens of the U.S., the U.K., and Canada. Our current #2 charity is a U.K. charity only, but U.S. donors can get a tax deduction for giving to GiveWell for the support of SCI.

If you’re in another country, you may wish to look into groups in your country that provide the service of (a) taking deductible donations themselves; (b) transferring the funds (minus fees) to charities abroad. While they can charge substantial fees, you may still come out ahead after tax considerations.

3. Avoid the large transaction fees and delays associated with large online donations. When donating via credit card, you will almost always be charged standard credit card processing fees. For donations under $5000, our feeling is that it’s worth paying the fees to avoid the hassle (both for you and the charity) of another donation method. But fees on a $5000 donation will generally exceed $100. In addition, credit card transactions of $5000 and up are often flagged by credit card companies (though a call to the credit card company can generally resolve the situation quickly).

This year, Google Checkout stopped offering fee-free processing, and we were unable to find a viable alternative with lower fees. Therefore, we are recommending that people giving $5000 and up consider writing a check or making a bank transfer. (Details)

4. Give appreciated stock. In the U.S., if you give stock to a charity, neither you nor the charity will have to pay taxes on capital gains (as you would if you sold the stock yourself). If you have stock that you acquired for $1000 (and has a cost basis of $1000) but is now worth $2000, you can give the stock to charity, take a deduction for $2000, and not have to pay capital gains tax on the $1000 of appreciation. This can result in significant savings. (More at Vanguard’s write-up on this topic.)

Taking advantage of this requires having some of your net worth invested in appreciated stock (or other securities), knowing (or being able to obtain) the cost basis of such securities, and arranging to transfer the stock directly to a charity, which is generally a fairly easy process, but varies from broker to broker; waiting until December 31 can be especially problematic if you’re trying to do this.

GiveWell can take direct transfers of stock for the support of our top charities, so let us know if you’re interested in this. A more robust way to smooth the process of giving appreciated securities is to take advantage of a donor advised fund, discussed immediately below.

5. Look into donor-advised funds to make the process smoother and more consistent year-to-year. Donor advised funds allow donors to make a charitable donation (and get a tax deduction) now, while deciding which charity they’d like to support later. The donation goes into a fund that is “advised” by the donor, and the donor may later recommend a grant from the fund to the charity of his/her choice.

We see a couple of advantages to this setup. One advantage is that you can separate your “decision date” (the date on which you decide which charity you’d like to support) from your “transaction date.” That means that if you aren’t ready to decide which charity to support yet, you can still get started on the process of transferring funds and getting a tax deduction for the appropriate year. Another advantage is that if you change the charity you support from year to year, you’re still working with the same partner when it comes to transactions, so the process for e.g. donating stock will not change from year to year. Donor advised funds are often set up to easily accept donated stock, whereas charities may or may not be.

Many large investment companies – Vanguard, Fidelity, Schwab – offer donor advised funds. They generally charge relatively modest management fees.

We do not recommend delaying this year’s donation while you look into donor advised funds. Donor advised funds can make giving more convenient over the long term, year-to-year, but you can get all the advantages of efficient giving this year without one. We also maintain our own donor-advised fund for donors interested in supporting our recommended charities.

6. Consider the political environment. If you believe that your tax rate is likely to be higher in 2013 than in 2012, you may wish to give at the very beginning of 2013 rather than the very end of 2012.

We’ll likely be counting donations made in the first week or so of 2013 toward our 2012 money moved (we’ve made similar adjustments in the past).

7. Choose your charity wisely. Saving money on taxes and transaction fees can be significant, approaching or exceeding a 50% increase in the amount you’re able to give. However, we believe that your choice of charity is a much larger factor in how much good your giving accomplishes.

Our charity recommendations make it possible to support outstanding, thoroughly vetted organizations – which we’ve investigated by reviewing academic evidence, interviewing staff, analyzing internal documents, conducting site visits, assessing funding needs, and more – without needing to do your own research. We publicly publish the full details of our process and analysis, so you can spot-check whatever part of our work and reasoning you’d like to.

Final notes.

  • If you support our recommended charities (on the basis of our recommendation) but you don’t give through our website, please use the donation report form to let us know about your gift; doing so helps GiveWell track our impact.
  • We believe that even when dealing with a relatively complicated gift (for example, a gift of stock), it’s possible to give quite quickly and with only minor hassle. The much more difficult challenge is choosing a charity, and we’ve tried to make that easy as well. So we hope you’ll give this season, even if you’re just starting to think about it now; if you’d like more advice or help, don’t hesitate to contact us.

GiveWell’s annual charity recommendations refresh

As we do every year, we’re planning to release updated charity recommendations by December 1st. We’re currently weighing what we’ve learned about the organizations that could receive our top ratings. Currently, the contenders for top spots are Against Malaria Foundation (AMF) (our current #1 charity), Schistosomiasis Control Initiative (SCI) (our current #2 charity), and GiveDirectly (which we have written about before, but haven’t previously recommended due to its newness; now that it has more of a track record we are taking a closer look).

As part of this work, we’ve recently released updates on AMF (latest update here) and SCI (latest update here). I also recently returned from a trip to Kenya to see GiveDirectly’s work in the field, and we are working on an updated review of GiveDirectly that incorporates all we’ve learned recently.

Insecticide resistance and malaria control

A recent Scientific American post discusses a study that may indicate an emergence of “behavioral resistance” to insecticide-treated nets. Mosquitoes in Benin were found to shift their “mean catching time” (roughly speaking, the time of day when they were most likely to be active) from 2-3am to 5am; this may indicate changing behavior to attack humans closer to the daytime (when they’re more likely to be away from their insecticide-treated nets) as opposed to the middle of the night.

Along with a study released last year showing increased insecticide resistance following a net distribution campaign, this study highlights a major concern for supporters of insecticide-treated nets (ITNs): ITNs may become less effective as mosquitoes adapt to them – whether by evolving resistance to insecticide or changing their behavior. We are concerned about this issue, seeking to learn more about it, and even considering recommending funding research on it. That said, we think the balance of considerations still points to ITN distribution as an outstanding intervention.

Below, we lay out what we know regarding key questions around ITN resistance (defined broadly as “any ways in which populations of mosquitoes adapt to the presence of ITNs in order to make them less effective”). We then give some general considerations in favor of not weighing this factor too heavily against ITN distribution for the time being.

We address the following questions:

  • What is known about the current state of resistance? A fair amount of data has been collected on some forms of resistance; less has been collected on other forms. The resistance data that is available points to widespread (but not universal) resistance in western Africa, little resistance in southern Africa, and very little data for central Africa. More
  • Is the prevalence of resistance increasing and is this caused by ITN distributions? The consensus view among malaria scholars seems to be that prevalence of resistance has been increasing, but we have not been able to find evidence for this view that clearly distinguishes “increasing prevalence of resistance” from “increasing efforts to monitor resistance and thus increasing reports of resistance.” If prevalence of resistance has been increasing, it may be due to multiple factors, including but not limited to ITN distributions. More
  • How does resistance affect the effectiveness of ITN distributions? Unfortunately there is little data available to assess this issue; it isn’t standard to collect data on malaria cases as part of an ITN distribution, and it seems that cases in which one can pinpoint the timing of both an ITN distribution and subsequent trends in malaria cases are relatively rare.It appears that there has only been one confirmed case of malaria control (in this case, insecticide spraying) failing to control a mosquito population due to resistance. There are several cases of ITNs’ apparently remaining effective despite widespread resistance, and many plausible explanations for why this could be the case. However, it appears that relatively little is known about this issue; it appears that more study could shed significantly more light on the matter. More
  • What can aid groups do in order to manage resistance? Various techniques for mixing/rotating different insecticides may be used to manage resistance in areas where it is problematic; unfortunately, however, there is currently only one class of insecticide that can be used on ITNs. So in areas where resistance is confirmed to be an issue, other control methods can be brought in, but in areas where little is known and only ITNs are used, mixing/rotating insecticides is not an option.Despite this, the World Health Organization recommends the continued use of ITNs even in places where resistance is confirmed; they have several ways of being effective and are expected to retain partial or full effectiveness.

    Longer-term, work is underway developing new technologies, including ITNs that use different insecticides, to deal with this issue. More

We see resistance as a major concern; a reason that it’s important to continue collecting data on the real-world impact of LLINs (as Against Malaria Foundation is attempting to do); and an area of research that may represent a good giving opportunity itself.

That said, our positive view on LLINs remains in place. There is strong evidence that LLINs reduce malaria and save lives and only preliminary/suggestive/mixed evidence that insecticide resistance may reduce their impact. In addition, it appears to us that the malaria control community has been devoting at least some attention and investigation to this issue for a long time, has developed a reasonable knowledge base (if one that has plenty of room to grow), and still recommends the use of LLINs regardless of the resistance situation.

Indeed, the fact that we’re discussing this issue at all speaks to the extraordinarily and unusually strong evidence base (and supply of data) behind ITN distribution. For most aid interventions that donors can fund, the set of “things that could go wrong” is large and broad, and we have little evidence to address most of them, but when looking at LLIN distribution, the salient concerns are few and specific enough that the malaria control community is able to put substantial resources into specifically investigating them.

Note that

  • This post does not give citations, but the technical version of this writeup does.
  • Against Malaria Foundation, our #1 charity, has made a post of its own on this topic.
  • Throughout this writeup, we primarily rely on two review papers (World Health Organization and Ranson 2011) whose claims we have not vetted (by, for example, examining the studies they cite); for the most part the claims cited here are fairly straightforward and we have relatively little cause for skepticism about them.

What is known about the current state of resistance?
“Resistance” can be used to refer to genetic properties of mosquitoes, to refer to behavioral properties of mosquitoes, or to refer directly to failures of mosquito control. In the context of malaria control, we’ve generally seen it used to refer to the first two; as discussed below, there is relatively little evidence of control failure due to these factors, so “resistance” is usually used to discuss specific risk factors for control failure.

Ranson (2011) provides a map summarizing the available information about resistance, using publications from 2000-2010. In this context, “resistance” is defined by the percentage of mosquitoes that die after a set period of exposure to insecticide under laboratory conditions; anything under 80% is considered “resistant” and anything under 97% is considered “possibly resistant.” This measure is intended for sensitive, early detection and does not necessarily indicate ineffectiveness of control measures.

Resistance appears common (though far from universal) in the area of western African running from Cote d’Ivoire through Cameroon; it is less common in the countries west of that and in southeast Africa; data from other parts of Africa is thin or nonexistent.

Another map in the same document lists what is known about the spread of genes known to be associated with resistance; detecting genes is believed to be an even more sensitive and earlier way to detect threats of resistance. If these genes were fully responsible for the observed resistance in laboratories, that would be good news, since resistance associated with these genes appears to us to have been most heavily studied and (as discussed below) appears to be associated with relatively lower risks of control failure. However, not all of the resistance observed in the map above can be attributed to these genes, and metabolic resistance (a different type) has been implicated in 10 studies in 8 countries.

The above covers only target-site and metabolic resistance, which make insecticides less effective in killing mosquitoes; by the nature of the laboratory test, it does not address behavioral resistance (mosquitoes’ changing behavior in order to avoid nets). There is currently relatively little evidence regarding behavioral resistance and no clear consensus about its importance. (Though we would encourage readers to take recent reports of possible behavioral resistance with a grain of salt. Mosquitoes’ altering their behavior could reflect many things and doesn’t necessarily imply adaptations to nets; behavior adaptations such as seeking to feed during the daytime may make mosquitoes easier to detect and thus more vulnerable; and most importantly, we would always caution against putting too much weight on a single study, particularly a recent one).

Is the level of resistance increasing? Is this caused by the use of ITNs?
It has been difficult for us to reliably distinguish the increase in reports of resistance (caused by increased efforts to assess resistance) from an increase in actual prevalence of resistance; reports on resistance often do not draw clear distinctions between the two. We also have been unable to find a comprehensive review regarding the question of what factors appear to have contributed to the development of resistance. Ranson (2011) states that there is reason to believe that ITNs contribute to the development of resistance, but resistance may also be brought about through the use of agricultural pesticides and other factors:

the primary source of this selection pressure [for resistant mosquitoes] is still unclear although evidence is accumulating to incriminate the agricultural use of insecticides in selection for insecticide resistance in malaria vectors. For example, seasonal fluctuations in results from WHO bioassays in major agricultural regions of Sudan and Cameroon reflect the timing of application of insecticides to the key crops in the region [45, 54]. In addition several reports have documented a higher incidence of insecticide resistance in areas of intensive agriculture than found in comparable non-agricultural sites [58, 59, 128]. There are also examples where use of pyrethroids for malaria control appears to be the primary factor in selecting for resistance [62, 129, 130]. Other sources of selection pressure including the use of consumer products such as coils or aerosols, or selection due to other toxins permeating mosquito breeding sites may also be contributing to the resistance phenotype [25, 131].

What is the impact of resistance on ITN distribution and other control measures?
It appears to us that

  • There is relatively little data available to assess this issue. It isn’t standard to collect data on malaria cases as part of an ITN distribution, and it seems that cases in which one can pinpoint the timing of both an ITN distribution and subsequent trends in malaria cases are relatively rare. We’ll be writing more about this general issue in the future; our impression is that what data there is implies that malaria has been in decline and (less strongly) that ITNs have been effective, in general, but the case is far from comprehensive.
  • There aren’t many cases in which a study found a failure of malaria control as a result of resistance. The clearest-cut case, according to both of the sources we’ve relied on most for this discussion, was around 2000 on the border between South Africa and Mozambique. Ranson 2011 cites another, less clear-cut, case from the island of Bioko on the West African coast, and both sources cite additional cause for concern from studies using experimental huts (though Ranson 2011 notes that the evidence from these studies is mixed).
  • There are multiple studies implying that ITNs can still be impactful even in areas where resistance is confirmed. However, these studies may be of limited use since they are focused only on a particular kind of resistance.
  • The reviews we’ve focused on express the belief that resistance is likely to become a problem eventually and that addressing it should be considered an urgent priority.
  • Our impression is that resistance was also a factor in earlier malaria control efforts, but may not have been as rigorously studied; the sources we’ve reviewed do not refer to these earlier efforts.

There are multiple reasons that ITNs may retain effectiveness even against mosquitoes that are “resistant” (in the sense of demonstrating low mortality rates in laboratory settings). In addition to the fact that ITNs provide a physical barrier, the insecticide may also repel mosquitoes (and cause them to seek out other targets) even when it does not kill or fully disable them. In addition, it’s possible that mosquitoes are still killed by the insecticide (despite reduced susceptibility) when they have enough contact with it; that insecticide may inhibit them in other ways that stops them from transmitting malaria; that resistant mosquitoes are less fit overall or less prone to transmitting malaria; or that mosquitoes that are resistant to insecticides at young ages may become less resistant as they age (and that older mosquitoes are more relevant to malaria transmission.) We have come across one study that argues for the last of these phenomena.

What can aid actors do about this issue?
One approach to managing resistance is to periodically change the type of insecticide being used, so as to reduce selective pressure for resistance to particular insecticides. (This may reverse the spread of resistance, if resistant mosquitoes are less fit in other ways.) However, there is only one class of insecticide (pyrethroids) approved for use on LLINs, so in areas where LLINs are the main form of malaria control, options are relatively limited (though in cases where resistance is confirmed to be having a major effect, other methods may be brought in specifically in order to get the benefits of using multiple insecticides). The World Health Organization recommends that in all circumstances – even where resistance is confirmed and is interfering with control efforts – LLINs should continue to be promoted.

Hopefully, there will be LLINs with different insecticides that are usable in the future; research has been done toward this goal and has made preliminary progress.

Closing notes
We’re very concerned about insecticide resistance. It appears that relatively little is known about the extent, causes, and control implications of resistance; that resistance could substantially effect (or even negate) the effectiveness of malaria control; and that more research has great potential to improve the impact of the substantial amounts of money spent on LLIN distribution (and other forms of mosquito control). This investigation has highlighted research on insecticide resistance as a potentially outstanding giving opportunity itself, and we hope to look into it more.

That said, this issue doesn’t change our bottom line that LLIN distribution is a highly cost-effective intervention or that Against Malaria Foundation is our #1-rated charity. There is strong evidence that LLINs reduce malaria and save lives and only preliminary/ suggestive/ mixed evidence that insecticide resistance may reduce their impact. Importantly, it appears to us that the malaria control community has been devoting at least some attention and investigation to this issue for a long time, has developed a reasonable knowledge base (if one that has plenty of room to grow), and still recommends the use of LLINs regardless of the resistance situation (as noted in the previous section). We have previously been impressed with the thoughtfulness and data behind malaria scholars’ answers to our concerns over the question of why universal coverage is now being pursued (as opposed to targeting children under five), the question of whether ITNs delay the development of immunity and thus merely delay malaria deaths, and the question of whether ITNs are used by recipients (though we’ve had more trouble when it comes to information about how long ITNs remain in use in the field), and we are inclined to put reasonably high weight on recommendations such as this.

Finally, we’d like to note that

  • We see it as very important to monitor and report changes in the malaria burden following ITN distributions, as Against Malaria Foundation is attempting to do. This isn’t a substitute for studying resistance specifically, which may help the malaria control community anticipate problems before they arise. But it seems important as a way to know when resistance – or something else – has become a big enough problem to interfere with the effectiveness of ITNs in the field.
  • We think the fact that we’re discussing this issue at all speaks to the extraordinarily and unusually strong evidence base (and supply of data) behind ITN distribution. For most aid interventions that donors can fund – water infrastructure, agriculture interventions, etc. – the set of “things that could go wrong” is large and broad, and we have little evidence to address most of them. By contrast, when looking at LLIN distribution, we see strong basic evidence for impact and reasonably good (if not conclusive) answers to the most obvious concerns about how real-world effectiveness compares to the efficacy seen in studies. The remaining concerns are few and specific enough that the malaria control community is able to put substantial resources into specifically investigating them.

Giving to GiveWell’s recommended charities helps GiveWell

GiveWell does not solicit donations from the general public. We cover our operating expenses mostly by privately soliciting donations from people (and institutions) who are big fans of our work. This allows us to operate – when dealing with those outside our “inner circle” – as a pure advisor, without having to “compete with our own top charities.”

Occasionally this policy creates some confusion for donors interested in giving to promote “effective giving” in general, not just interested in directly funding proven interventions. Such donors sometimes express interest in donating directly to GiveWell, and when we state that we aren’t seeking donations from them, they conclude that their giving can’t help us. But it can, significantly, if they give to our recommended charities (in such a way that we can track the donation).

The main way this takes place is through our money moved metric, which tracks the donations given to our recommended charities on the basis of our recommendations. This figure has several important implications for our mission:

We advertise our “money moved” figure for charities in order to make the case that our investigations are worth their time. Since our founding 2007, we’ve seen continuously improving access to the people whom we need to speak with in order to do our research well: charity representatives, academics, etc. We believe our growing “money moved” has been a key factor. Because of this dynamic, higher “money moved” directly improves the quality and efficiency of our research process.

More generally, our “money moved” figure serves as a general indicator for interest in effective giving. We hope that in the future, the “effective giving” world will be much larger, with more organizations devoted to helping to find the best giving opportunities. People who would consider starting such organizations can learn about the size of the potential “market” from our figures. Charities that are outside our current scope may still wish to start planning for a future in which our sort of analysis plays a larger role, and our figures may serve as an indication of how likely this future is.

Our “money moved” figure is a major tool for our own fundraising, and a major input into the size of our operating budget. The higher the figure, the stronger the case for a growing operating budget, and the easier time we have raising the funds we need for it.

Our growth in money moved has been strong so far. If this weren’t the case, we’d have had far more trouble raising money, doing research, and making the case for our influence.

If you’re a donor looking to help GiveWell in our mission, please consider

  • Donating to our top charities.
  • Making sure that the donation is attributable to GiveWell and thus countable in our “money moved” metric. You can do this by giving using GiveWell’s website (note that in the case of our #1 charity, AMF, our website points to AMF’s website and AMF provides the tracking function) or by using our donation report form.
  • Telling your friends about us using Facebook, Twitter, email, or in-person conversations. Note that we track the number of donations we influence as well as the number of dollars, and we pay attention to (and advertise) the robustness of our “money moved” figure as well as the level.

Evaluating people

A crucial consideration, when evaluating a giving opportunity, is: “Who are the people involved?”

I believe that any given project is likely to run into many factors that are unexpected (or can’t be captured in the evaluation process), and indeed that such unexpected factors often dominate the expected factors. Therefore, no matter how promising a project looks on paper, I expect it to fail if the people behind it aren’t capable enough; conversely, if the people behind a project are sufficiently capable, even a project that sounds terrible to me could turn out to be excellent (or be adjusted as it progresses to be excellent) based on factors that I fail to understand at the time of evaluation. Bottom line – I expect funding great people to lead at least to good (if not great) things, and funding incompetent people to lead to failure, in many cases regardless of most other factors. (Two important counterpoints: the choice of sector is important, as great people often work on issues they’re passionate about and the good they can accomplish can be limited by their choice of issue, and room for more funding is always crucial: even the most capable people generally have limits to how much funding they can productively use).

I’ve believed this for a long time, but GiveWell hasn’t used “evaluation of people” as a significant explicit factor in our evaluations (though we have taken the position that much of what we look for implicitly selects for good people). Lately, we’ve found ourselves putting more explicit weight on our impressions of an organization’s people. This post discusses why we’ve previously been hesitant to explicitly weigh and discuss this factor, what has changed, and how we evaluate people.

Challenges of explicitly evaluating people

It’s worth noting that GiveWell’s process to date has contained a great deal of implicit evaluation of the people involved in the charities we’ve examined.

  • One of our main goals is evaluating a charity’s track record. We’ve argued in the past that evaluating an organization’s track record can be a good way to evaluate its people. In the nonprofit world, there’s an additional wrinkle in assessing track records, which is that most nonprofits don’t have the data to make such assessment possible; we think those that do are likely to have important values in common with us, as discussed below.
  • More generally, we tend to ask many critical questions of a given charity, and to publish its responses along with our evaluation of those responses. We feel that asking critical questions about a person’s work and evaluating how much the answers seem to make sense is another good method for evaluating people.
  • Our emphasis on transparency has created an interesting way of distinguishing between charities: some groups take a near-“open book” attitude, others aren’t willing to put any of their materials or communications with us on the record, and others fall somewhere in between. We believe that this distinction may (imperfectly) capture important values, as discussed below.

That said, there are observations we make and opinions we form of people that don’t necessarily show up on their organizations’ GiveWell reviews. We notice not just how good an organization’s track record is, but how intelligently (from our perspective) its staff discuss the relevant issues; we notice not just how good its final answers to our critical questions are, but how efficient are the exchanges leading up to these answers. We form impressions that can’t always be pinned down to specific (or public) occurrences.

So far, we’ve been hesitant to put much weight on, or publicly discuss, these sorts of factors. A major hesitation is that we’re wary of putting too much weight on informal, intangible impressions that are likely to end up being “tests of how similar charity representatives are to us” rather than “tests of how well-suited charity representatives are to do their work.” It seems easy for evaluations of people to end up being driven by charisma, by similarity to the evaluator, and/or by characteristics that don’t translate well to nonprofit work. (Regarding the latter: many philanthropists have been successful in other domains, and are used to being able to pick out people who can succeed in the domains they know well; we fear that they aren’t able to build up the same experience and reliability of intuition in the nonprofit world, where it’s hard to know who has and hasn’t succeeded, and that the qualities they’re looking for may not translate well.)

Reasons our stance is shifting

Despite the concerns above, we’re now starting to factor our views on people more heavily into our prioritizations and recommendations, and we’re starting to feel that it’s important to write explicitly about our views on this front (not just about charities’ performance on our standard criteria). This is because

  • We’ve had enough experience to start feeling that we recognize patterns in what sorts of people/organizations eventually do well vs. poorly in our process, and as our process has improved we’ve become more confident in its conclusions.
  • Improvement in our process has also led us to have less confidence in any given empirical case (example). We’ve come to feel that we need to supplement empirical analysis with whatever other sources of views are available, even if they’re somewhat intangible.
  • We’ve been thinking more about how to evaluate charities without relying on our “proven cost-effective” framework; when doing so, explicit evaluation of people is more important.
  • We’ve been learning how rare it is that one can see how a charity has performed over time; because of this, it’s become more important to us to support groups that could contribute to changing this, i.e., groups that share some of our core values such as valuing self-evaluation and transparency.

That said, the concerns listed in the previous section still apply, and thus we are still cautious about placing heavy weight on our reads of people.

How we evaluate people

We look for the following qualities:

  • Clear, direct communication. We view clear, direct communication as a sign that someone welcomes – rather than prefers to avoid – (a) substantive discussion of their work; (b) honest criticism leading to improvement. It’s also generally easier for us to communicate with people who have this quality, which makes us more confident in our evaluation of them and their organization.
  • A sense that the person is following their own agenda rather than catering to ours. We try to avoid people whose primary goal seems to be telling us what we want to hear. If someone seems averse to contradicting or criticizing us – or seems more interested in the question of “what plans GiveWell would like the organization to carry out” than “what plans the organization would like to carry out” – we don’t consider this a good sign. We prefer organizations that set their own agenda and directly engage our questions.

    Clear, direct communication is a positive sign on this front, particularly when it includes criticizing, contradicting, and educating us.

    (We discussed this factor in a previous discussion of our stay in India.)

  • Thoughtful, plausible answers to our critical questions. In our view, one of the marks of an exceptionally capable person is that they have thought extremely deeply and thoroughly about the work they specialize in, to the point where they have thought of nearly any critical question that we could raise, and have thought about it more deeply than we have. This doesn’t necessarily mean they will be able to give an answer we find fully satisfying on any given critical question, but it means they will usually give an answer that shows they have thought hard about the issues and have an answer that we could plausibly find satisfying if we knew more.
  • Personal and organizational history. We look for generally impressive accomplishments and for experience relevant to the work being done. How we weigh these factors is highly context-dependent, and particularly depends on the organization’s mission. There are some types of work for which we’re comfortable with people who have little directly relevant experience, and others for which we find the personal and organizational resume to be more important.
  • Self-skepticism and valuing self-evaluation. This is an area where we see a significant disanalogy between for-profit and nonprofit work. When speaking to for-profit investors, we’ve heard the sentiment expressed that it’s natural and healthy for an entrepreneur to be overconfident; we can imagine that in the for-profit world, feedback loops and course correction often play out despite overconfidence, since people know they need to make adjustments when their deliverables (traffic, revenue, profit, etc.) are below expectations. But in the nonprofit world, there’s often (or always) no way to learn about whether your work is helping people unless you make a substantial, explicit effort to learn. People with high confidence and charisma and low self-skepticism worry us: we fear that they will continually succeed in raising money from donors but will never do the investigation and adjustment necessary to optimize their work for helping people.

    From what we’ve seen, it’s extremely rare to find people in the nonprofit world who (a) concede many possible ways in which their programs could be failing and (b) have put substantial thought and effort into investigating the relevant issues. People who do have these qualities tend to produce unusually meaningful and informative self-evaluation. We value such self-evaluation greatly not just because it’s informative but because of what it reflects about the people involved.

  • Valuing transparency. While there are sometimes good reasons to keep particular information confidential, a general preference to share more information publicly is likely, in our view, to go hand-in-hand with (a) self-skepticism and interest in being critiqued; (b) an interest in creating public goods (i.e., helping others to learn from one’s own experience) and helping the world as a whole rather than in focusing on the success of one’s organization. We see significant, tangible variation in attitudes toward transparency: some organizations want everything kept off the record, others are willing to share some information but wish to withhold whatever does not reflect well on them; others take more of an “open book” attitude.

Perhaps not surprisingly, these qualities mirror the qualities we believe ourselves/GiveWell to have. This makes sense for a couple of reasons. First, we’d guess that people are usually most impressed by people who are similar to them in important ways – sharing basic values, sharing communication styles, etc.; while we’d like to be able to identify people who are outstanding at what they do despite being poor at communicating with us, this is simply practically difficult. Second, we perceive ourselves as bringing important and undervalued values and qualities to the nonprofit sector, and part of what we find ourselves looking for in charities is others who will spread similar values and qualities.

We recognize that we may be overrating the importance of the qualities discussed above, but we feel that we maximize our own value-added by pursuing and promoting them, and part of the goal of this post is to ensure that our audience knows these are the values we’re seeking to pursue and promote.
We believe that other funders often look for substantially different qualities – in particular, they often seem to place more weight on charisma and fundraising ability, perhaps because they are hoping their investment will be leveraged into donations from many others. There is sound logic behind this preference. However, our feeling is that the current nonprofit sector overemphasizes storytelling ability and fundraising success, relative to thoughtfulness and transparency – with the result that we see a lot of the former and less than we’d like to see of the latter – so the qualities we look for are different..

Over the coming weeks, we will be adding sections to the pages on our top charities that discuss our views on each organization as a whole (including its people).

Revisiting the case for insecticide-treated nets (ITNs)

GiveWell seeks to constantly revisit and rethink our content, to make sure that it continues to represent a reasonable interpretation of the currently available facts. To this end, we assigned Jonah Sinick, a Research Analyst, to perform a thorough review of the content most relevant to our #1 charity (the Against Malaria Foundation): the AMF review page and the page on distribution of insecticide-treated nets. We asked Jonah to check our footnotes, think critically about our reasoning, and list as many potential problems with our content as he could find; we committed to publishing a list of all such potential problems along with our responses.

The full output of this exercise is here; a summary of the major issues raised and our responses is here. This post focuses on one particular issue Jonah raised – a question about the extent to which small-scale studies of ITN efficacy are representative of real-world conditions (in particular, real-world malaria burdens) – and describes the steps we took to investigate the matter. In brief:

  • Jonah found some initial reason to suspect that our “cost per life saved” for ITN distribution was unrealistically low. He noted that based on available country-level estimates for malaria mortality, ITNs could not save lives with the frequency we were claiming, even if they prevented all malaria deaths.
  • In response, I did some initial estimates that implied the correct “cost per life saved” figure might be 5-10x higher than our current figure, and accordingly marked the issue as a high priority.
  • After further investigation, we concluded that, in fact, no major revision to our “cost per life saved” figure is called for. The deaths averted by ITNs appear to include deaths that are not strictly classified as “malaria deaths,” and available data does not give us reason to believe that the ITN efficacy studies are unrepresentative.
  • However, a smaller issue – the decline in all-cause child mortality between the time that ITN efficacy studies took place and today – does potentially call for a relatively small adjustment, and also adds a new source of uncertainty regarding the cost-effectiveness of ITN distributions.
  • There are multiple issues that we continue not to account for in our cost-effectiveness estimate, including other positive impacts of nets: prevention of non-lethal malaria cases, reduction of malaria deaths among those over the age of 5, and contribution to the prevention of other diseases transmitted by mosquitoes such as lymphatic filariasis.

Our bottom line after going through this exercise remains that insecticide-treated net distribution remains one of the most cost-effective programs we know of and we continue to rank the Against Malaria Foundation as our #1-ranked charity.

The rest of this post describes our investigation in greater detail than we usually do; since we are publishing the results of the vetting, we thought we’d take this opportunity to give interested parties a more-in-depth-than-usual look at the kind of back-and-forth and analysis we are constantly engaging in to get a better understanding of our top charities.

Overall, we take this episode as another example of how difficult it is to have high confidence in a “cost per life saved” type figure. Our estimate represents the figure that we come to when we use the best evidence we can to estimate each parameter as fairly as we can, but it does not include a Bayesian adjustment to account for the substantial uncertainty, both on account of issues we’ve thought of and issues we haven’t thought of; as such, it should not be taken literally.

That said, we also think the issues discussed in this post illustrate a good reason to do cost-effectiveness estimation: it is a way of disciplining ourselves to make sure we’ve addressed every input and question that matters on the causal chain between interventions (e.g., nets) and morally relevant outcomes (e.g., lives saved). In this case, Jonah’s examination of our cost-effectiveness estimate led to many new questions and observations, and we feel that our overall understanding of the strengths and weaknesses of the case for ITN distribution has improved. This benefit of cost-effectiveness estimation is only realized when estimates are continually revisited and rechecked, which in turn requires that they be fully transparent to those checking them.

Note that a future post will discuss another concern we have investigated this year: the question of whether mosquitoes develop resistance and/or adapt their behavior to ITNs over time.