Comments

• Greg Simon on December 21, 2007 at 6:10 pm said:

  GiveWell’s Not Getting It Well—Medical Research Is Not Charity

  FasterCures President Greg Simon responds to The New York Times piece by Stephanie Strom “2 Young Hedge-Fund Veterans Stir Up the World of Philanthropy,” http://www.nytimes.com/2007/12/20/us/20charity.html

  The article states, “Mr. Karnofsky and Mr. Hassenfeld, both 26, founders and sole employees of GiveWell, which studies charities in particular fields and ranks them on their effectiveness. GiveWell is supported by a charity they created, the Clear Fund, which makes grants to charities they recommend in their research.”

  After reading this, the third article I’ve seen about these refugees from Wall Street, I feel moved to point out a few things. First, it’s unfortunate that what has drawn attention to their work is using blogs to flame the admittedly flawed ratings groups like Charity Navigator. Second is this obsession of equating philanthropy with “charity.” I do not know any medical research group that considers itself a charity. They are nonprofit disease research organizations developing cures for deadly and debilitating diseases. Charity has nothing to do with it – these orgnizations reprsent investments – and yes I feel silly pointing that out to people who used to do research for hedge funds.

  Once you realize organizations that are trying to save lives through research are an investment, you evaluate these organizations differently. You look at their strategies, their resources, their connections, board members, partners and risk taking and then you can create a diversified portfolio of disease cure investments going forward. That is what FasterCures is doing with our Philanthropy Advisory Service with grants from Gates and Robert Wood Johnson – and it takes time and more than asking a hundred questions and seeing who sends you their annual report faster than others – that really only measures receptionists.

  Unfortunately, GiveWell’s approach of “cost per life saved” is the equivalent of what investors call “chasing performance.” Some groups may be efficient in handing out condoms but to discuss “charities” saving lives and not mention groups like IAVI that are developing vaccines for AIDS is like focusing on iron lung distributors instead of a polio vaccine.

  We are all trying to get more out of our philanthropy, but saving lives requires strategies that try to create the future – and measuring that is harder than counting dollars and cents – or condoms.
• Shakespeare's Fool on December 21, 2007 at 9:05 pm said:

  That picture in the Times?
  Definitely the next Boris Karloff and Bela Lugosi.
  John
• Basti on December 22, 2007 at 1:23 pm said:

  Hi guys!

  Just read about your organization in the NYT and was already quite thrilled. After reading your blog I got enthusiastic.

  I think the work you are doing is not just useful but really necessary and I support you to speak out clearly even at the risk to offend somebody.

  It was also interesting to see how controversial these topics are even in the US. Compared to my country (Germany) your charities are already much more transparent. We have no Guidestar and our version of the Charity Navigator only covers 200 organizations whereas they just publish the positives and if you want to receive a “report” (only 3 pages) you have actually send “real” stamps to them so they can send you a “real” letter (nothing online). Well, in this way they at least make sure that no one reads their reports and gets confused with the straw ratio…
• Carl Shulman on December 23, 2007 at 12:38 am said:

  Greg,

  It’s not clear to me why you’re so incensed by the term ‘charity’ as opposed to ‘investment.’ The law defines these organizations as 501(c)(3) “public charities.” Funders don’t internalize much of the benefits of their donations, as they would with an actual investment, which helps to explain why philanthropists rarely examine their funding efforts as rigorously as their personal investments. Vaccine development provides its benefits over time, but so do charities distributing vaccines or bed nets.

  Also, the investment metaphor can lead to mistaken inferences, like this one: “then you can create a diversified portfolio of disease cure investments going forward.” Diversification of financial investments is beneficial because personal wealth has diminishing marginal benefits (having your investment portfolio fall to zero from $100,000 is much worse than having it fall from $200,000 to $100,000). That argument doesn’t apply to charity, e.g. funding medical research. Unless you’re giving so much that diminishing marginal returns set in (and areas like malaria vaccine research can absorb many millions of dollars) you maximize the benefits of your donation by spending all your contributions on the highest-return area and have no general reason to diversify.

  I agree that many extremely valuable research projects do not yet have quantifiable results, e.g. many medical research efforts or GiveWell itself, as Holden and Elie acknowledge. What is the status of your Advisory project? I found a page(http://www.fastercures.org/sec/meria) describing it and love the idea, but I didn’t see anything about progress over the first year. How do you assess the expected benefit of research? QALYs/DALYs? Do you plan to measure the effectiveness of your metrics/service in predicting research successes, comparing the research productivity of the selected groups to those funded by other sources?
• Gregory Simon on December 23, 2007 at 9:49 pm said:

  Carl, thank you for your thoughtful reply. First, I am not incensed about the confusion between “charities” and investments, more like irritated. For example you note that donors don’t internalize the benefits of their donation. Well, unlike speculative investments which may return zero, donors realize an immediate return through reduction of their taxes; so there is an immediate and predictable return. The fact that they do not examine these philanthropic investments as rigorously as their speculative financial ones is precisely why FasterCures is developing the database to allow them to do that.
  Second case in point about the confusion. You say diversified portfolios don’t make much sense in the medical research area because donors should put their maximum donation into high return areas. Pray, which ones are those? How would average philanthropists — even very wealthy sophisticated ones — know whether to fund immunotherapy or chemotherapy in treating melanoma? How would they know which of the warring theories about Alzheimer’s diseases is right? Should they have a 5, 10 or 30 year window for a cure breakthrough? NIH takes 30 years for most things ands thinks it is doing a great job. Most of the organizations we work with have 5 or 10 year plans and are willing to change the rules (re IP, publication, data sharing etc)to get there. In our opinion, without diversification of risk, time horizons, and therapeutic theories, donors are guessing. But where are they to get the data to figure out where any particular nonprofit sits on the continuum from high to low risk, short or long term strategies? That is what we are trying to provide through our program.
  Our Philanthropy Advisory Service is a two year project. Our first reports will be out by third quarter 2008. We have a team of people and advisors developing a system of metrics that make sense for medical research. We will do due diligence interviews with organizations in diseases we are studying. Most importantly, we are not trying to pick winners and losers. We are providing the information on every nonprofit in a given disease area so donors can compare them across multiple qualities and then make their own decisions about which ones to fund. We are not going to look at all the groups in a given area and then say “that one.” I think that is folly. We are trying to match a donor’s proclivities and expectations with the capabilities of organizations. We hope through transparency the innovative groups will stand out. But we are not trying to predict the future. We are trying to make clear what disease research groups say they are trying to do and then examine them to see if that is what in fact they are doing. Is what they’re doing novel or stale? Is it going to matter to patients or just to other scientists? Is it properly supported by the right people and resources to get it done? But no one can say whether one strategy is going to cure cancer versus another, believe me if we could do that, we would already be glad to be out of business. And as for measuring our success, no one can guarantee a donor that they can expect a return by funding one course of medical research over another. But we can increase the amount and diversity of philanthropy in both global and domestic diseases by giving donors more assurance that they know what they are funding and what they can expect in terms of risk, effort and timeline.
  Thank you again,
  Greg
• Holden on December 24, 2007 at 1:34 am said:

  Greg:

  First, I want to clarify a couple things. We do not try to “guarantee” certain results for donors, nor is our aim to say “that one” definitively and take away donor choice. Like what you describe, our goal is to put the information out there to let donors choose for themselves.

  However, I think it would be a real mistake to try to do this without producing our own (explicitly and admittedly debatable and guesswork-based) ranking. I’m not sure I want to get into a huge debate about why this is – it may be a future blog post. I’ll just say briefly that (a) a lot of donors simply don’t have the time to root through everything themselves – they’d rather just go with the best guess of people who’ve spent more time than they have, which is an entirely reasonable wish and one that we can improve the world by accommodating; (b) providing rankings gives context and shape to the information we provide, making it easier for us to focus on finding out what we need and presenting it in a way that matters; (c) we are already seeing evidence that donors ARE using our site to choose for themselves (our higher-ranked charities have gotten more donations through our site than lower-ranked ones, but not by as much as you’d think).

  We also have no ironclad commitment to “charity” over research. There are advantages to both – certainty vs. magnitude – and like all other donors, we are taking our best guess first (I’ve written a little about our choice here). We hope eventually to cover research, though if you guys do it better than we could, we won’t.

  That’s all in defense of what we’re doing, which I think you’ve mischaracterized. That said, if you guys can produce a useful report for donors that helps them choose which medical research projects to fund, AWESOME. I agree with you that research is valuable and important. I agree with you that donors need better information and help with their choices.

  I predict that if you choose not to rank projects, you will end up with a site that is both confusing/overwhelming AND incomplete (i.e., not providing all the information one would want in order to make the most informed decision possible). Ranking projects while giving 100% of the reasoning and information behind your rankings (as we do) will make the site more useful both for those with no time (who just want to use your recs) and those with a ton of time (who want to get the WHOLE story – not a standardized set of facts, but everything that one would want in order to make the most informed decision possible).

  But that’s a minor point, a detail, and your call. What matters is this: if you can help solve this problem of insufficient donor resources, and cover an area – research – that we would probably struggle with, we will want to help you in whatever way we can. Even if that way is getting slammed by you for publicity purposes. So shoot me an email when the report is out, because if it’s good, we will want to link to your site and promote it on ours, even if your entire promotion strategy consists of making fun of my big nose. Best of luck.
• Carl Shulman on December 24, 2007 at 1:59 pm said:

  “Second case in point about the confusion. You say diversified portfolios don’t make much sense in the medical research area because donors should put their maximum donation into high return areas. Pray, which ones are those?”

  When making decisions about future investments or donations one decides on the basis of expected returns, the sum of the products of the probability of each outcome and the corresponding benefit of that outcome. Here’s a stylized example for the benefit of our readers:

  1. A philanthropist offers you a game, in which she will roll a six-sided die and you can bet on the outcome using a pool of $6,000 she provides.
  2. If you correctly bet a dollar that a particular side will come up, then you will get a number of dollars that varies by side:

  ‘Side 1’-$6.00
  ‘Side 2’-$12.00
  ‘Side 3’-$18.00
  ‘Side 4’-$24.00
  ‘Side 5’-$36.00
  ‘Side 6’-$54.00
  3. Before you place your bets, the philanthropist will roll the die and offer to ‘sell’ you information about the result in exchange for part of your $6,000 budget. If you pay out $1800 she tell you whether the die came out even or odd.

  So what should you do in this situation?

  One option would be to spread your funds evenly between the possibilities, betting $1,000 on each side. The expected value of this allocation would be the sum of the payoffs for each outcome ($6k, $12k, $18k, $24k, $36k, $54k) divided by 6 (reflecting the 1/6th probability that any particular side will come up), or $25,000, 25 lives saved.

  But you could instead put all your money on ‘Side 6.’ You would then have a 1/6th chance of getting $324,000=324 lives saved, and a 5/6 chance of not doing any good at all, for an expected value of $54k=54 lives saved.

  Or, still better, you could first spend $1800 to find out if the die is even or odd. If you learn that the die is even, then you can put all of your remaining $4200 on Side 6, giving you a 1/3 chance of $226,800, and an expectation of saving 75.6 lives. If the die is even you put your money on Side 5, saving an expected 50.4 lives. When we average across the outcomes in which the philanthropist tells us that the die is odd and those in which she tells us that it is even, we find that in expectation we will save 63 lives.

  Medical research will always involve choice under uncertainty and, although we should reduce that uncertainty by expending resources to acquire information, we will do the most good if we are not irrationally afraid of uncertainty (particularly when we know that many other funders are engaging in diversification).

  “How would average philanthropists — even very wealthy sophisticated ones — know whether to fund immunotherapy or chemotherapy in treating melanoma?”
  Initially they could look at things like the quantity of research funding going into each area (noting that academic careers benefit from a steady flow of papers and positive results, so that researchers will tend at the margin to work on the approaches that are most likely to succeed rather than those for which one additional researcher is most likely to generate benefits for patients), get the independent opinions of a scientific advisory board (and hopefully publish those opinions) including both researchers and health economists, etc. Indeed, I would hope and expect that the Gates Foundation is doing precisely that analysis. If it were to make that information public then smaller sophisticated donors could use the information rather than reinventing the wheel, adjusting for different preferences (e.g. discount rates/time horizons, ethnocentrism, etc).

  “In our opinion, without diversification of risk, time horizons, and therapeutic theories, donors are guessing.”
  Donors are still guessing after diversifying across time horizons and therapeutic theories!

  They’re just funding things other than their *best* guess as well. If the donors were giving to maximize the chances that when a cure is developed they can say, “I gave some money to support that” then diversification would make sense as it would in an investment context, but if all they want to do is save as many lives as possible then they should invest in information (via scientific advisors, FasterCures analysis, etc) to identify the highest marginal value option, recognizing that it is still quite likely not to pan out, and put their donations there (until serious diminishing marginal returns set in).

  This analysis will need to be modified if the donors don’t value lives equally, e.g. if they prefer saving Americans to Africans, young people rather than older ones, people alive today versus people who will be alive in 25 years, but those adjustments don’t change the conclusion that a thoroughly altruistic donor should not diversify unless she is giving so much that diminishing marginal returns drive her from the initally best cause to the next-best cause. Diversification is about the self-image of the donor, not the welfare of the people the donation is supposed to help.
• Phil on December 24, 2007 at 2:45 pm said:

  Carl – for the most part, I agree with your analysis, but I think you underplay the benefits of ‘charity diversification’ a bit.

  Given charities A, B and C, where A saves 5 lives per $10K donated, by fighting malaria, B saves 4 lives per $10K donated, by fighting malnutrition, and C saves 3 lives, by educating the next generation of third world nurses (numbers totally made up, for illustration only), many donors would want to participate to some extent in all three areas, even though they could apparently save the most lives by allocating all of their donations to A.

  I can think of at least two reasons for this:

  1) Donors want to make sure they have SOME positive impact. Many charitable causes offer the possibility of positive impact, but may ultimately fail to achieve that. In disease research, this may be a matter of a promising approach not panning out. In funding malaria treatments, it may be a matter of government interference or corruption disrupting good intentions. It may even be a matter of a donor not thoroughly researching a charity and getting conned in one way or another. While you state that donors should be willing to fund the highest E.R. disease research, and let the breadth of the donor community spread out any risk, I think for many donors (myself included), the possibility that I might come to realize years down the line that all my donations had been for naught is something I wish to avoid. Perhaps this is a flaw in my mental model, or I am not being perfectly altruistic, but if so, I think I am not alone.

  2) I think for many donors, the feeling of giving $1000 each to three different charities is likely to be significantly more intense than the feeling of giving $3000 to a single charity. I think this is independent of, and somewhat different from, the sort of risk aversion I’m discussing above. Again, this may not be entirely rational, but I think it’s human nature.

  I think one way for donors to deal with this is to take an approach similar to one that some people use in investing. In that field, there is a conflict between the broad desire for risk aversion (rational, according to most economists), and the desire to gamble, either in the literal sense (trips to Vegas), or in an investment sense (making an undiversified bet on a long-shot tech company). One approach to deal with this is to compartmentalize – invest perhaps 95% of your portfolio by the book – a nice diversified portfolio. Take the remaining 5% and make a few riskier plays (though of course, don’t be a fool even with this 5%).

  For charity, focus on a limited number of efforts with the highest expected return. But perhaps allocate a portion of donations for more heartfelt causes, even if they can’t be clearly justified on expected results alone.
• Carl Shulman on December 24, 2007 at 4:26 pm said:

  Phil,

  We’re mostly in agreement. There are psychological reasons why people tend to diversify contributions, and they result in bad giving. That’s why the last sentence of my comment was:

  “Diversification is about the self-image of the donor, not the welfare of the people the donation is supposed to help.”

  The reason I make this argument is because people supposedly offering advice *ON HOW TO HELP PEOPLE* frequently recommend diversification. If people are told that inefficient diversification is actually a good idea from an altruistic perspective, rather than just a concession to our selfish and fallible natures, then they will tend to do more of it.

  People generally tend not to give it all, or to give based on on ethnic/racial/religious/educational/national affinities. People who take a more universalist approach can recognize and respect the right of people to act in this way, but we shouldn’t go around saying, “giving only to help people in your religious or racial group is just great, and to be heartily endorsed.” Instead we should accept an unfortunate human tendency and work to counterbalance its effects to the extent that we can do this productively.

  “the broad desire for risk aversion (rational, according to most economists),”
  To be clear, economists say that risk aversion in investments is rational because we get diminishing marginal utility from wealth, a rationale that does not apply to altruistic donation. Imagine if you split your wealth into 10 equal accounts, and then were highly risk averse within each account rather than across your entire portfolio. That would be clearly irrational.

  Similarly, from the perspective of doing good one should look at risk across society. Many people will be spending on causes 1, 2, 3, 4, 5, and 6 so society is already diversified against risk. You, as a sophisticated and thoughtful donor, can look at that existing societal diversification and put your money in the best place.
• Rob Klein on December 24, 2007 at 6:11 pm said:

  How do you not put up a picture of a bear with this post?
• Phil on December 25, 2007 at 2:24 pm said:

  Carl – I was thinking about how GiveWell can justify what it does to the world at large.

  I think your 6-sided die example is a good one, that can be tweaked a bit better for this purpose.

  Imagine every gift to charity yields an impact equal to the size of the donation times the (random) result of a die roll. Currently, without significant effort on the part of the donor, it is difficult to evaluate the effectiveness of a charity. I think it is entirely plausible that many charities are at least 6 times as effective as other charities.

  Somewhat like in your example, dice are rolled ahead of time, with one die roll per charity. The roller offers to provide you some information about the result for each charity.

  The first round of charity websites, like Charity Navigator, are basically offering to tell you which charities rolled a ‘1’. They help rule out the worst charities – those which spend egregious amounts on fundraising and admin. But they don’t really provide information on program effectiveness. Still, there is value even in their limited information – donors want to avoid the real stinkers (the 1s).

  GiveWell has the potential to tell donors which charities rolled a 4 or better. I doubt it will be able to be super-precise (to find the 6s with certainty), but I think the charities it finds, factoring in the uncertainties and guesswork involved, will be 4s or higher.

  In turn, if GiveWell influences donors who give $1 million, of which perhaps 40% would have been spent on inferior charities (2s and 3s), pushing that money up to charities rated 4, 5, or 6, then it’s moved $400K in donations up from an average effectiveness of 2.5 to an average effectiveness of 5. If GiveWell influences $10 million, or $100 million, then obviously the impact is greater.

  While my numbers are simplistic, perhaps the analogy is useful a basic tool to understand the potential impact of a well-done charity effectiveness site.
• Carl Shulman on December 25, 2007 at 3:25 pm said:

  Phil,

  Sure. I would add that we should also think about the incentive effects: if large amounts of donations start to be sensitive to charity quality, then charities will start to improve their programs in order to attract the ‘smart money.’

  I think that the variation in effectiveness will be more than tenfold, although the biggest differences in efficiency may stem from philosophical differences, e.g. willingness to help poor foreigners vs fellow rich country citizens, willingness to spend on indirect activities like research or evaluation, concern or lack of it for future generations, and willingness to postpone direct benefits.
• Gregory Simon on December 26, 2007 at 12:04 pm said:

  Holden,
  Short note now, longer one later …I hope you do not think I was “slamming” you, certainly not my intention. Because medical research is in a class by itself in my opinion, our effort is a very different one than what I know of yours but of course there are synergies and we would want to work with you as well and i greatly appreciate your offer of linking etc. And since Lucy B. advises both of us it is only natural. We would welcome the chance to work/think with you about the bigger issues here. And finally, i did not mention your nose and if you saw mine you would know why..
  best,
  Greg
• JohnR on January 24, 2008 at 3:04 pm said:

  Increasing accountability will increase donor confidence which will ultimately increase donor participation. In the age of information and with the clear lack of accountability in today’s markets, it’s amazing that the need for GiveWell hasn’t been heralded as the next big thing in the non-profit sector.
• Michelle on February 14, 2008 at 4:23 pm said:

  “Increasing accountability will increase donor confidence which will ultimately increase donor participation.”

  In the age of information, what is your evidence for this statement?

  Has this been demonstrated anywhere, by anyone? What is to make us believe that accountability=increased confidence=increased participation?

Comments are closed.