The GiveWell Blog

Self-evaluation: GiveWell as a project

This is the second post (of four) we’re planning to make over the next two weeks focused on our self-evaluation and future plans. The first post is here.

This post answers the “additional questions for stakeholders” that we posed to ourselves in January about the state of GiveWell. For each question, we discuss

  • Our progress over the last year (specifically, since our last business plan in 11/2008);
  • Where we stand (compared to where we eventually hope to be);
  • What we can do to improve from here.

Is GiveWell’s research process “robust,” i.e., can it be continued and maintained without relying on the co-Founders?

Progress since 11/08

As of 11/08, we had only made one extended attempt to hire a third employee, and had ended the relationship because our research process was not developed enough to make it possible for us to provide the necessary oversight (more).

In early 2009, we made several more unsuccessful attempts to integrate a new hire into our research process, before again concluding that our process wasn’t developed enough. We tried again with more success in mid-2009, hiring:

Details at our work plan review.

We feel that a key factor in being able to productively hire people has been the progress on our international aid report. We now have many available examples of charity reviews and a consistent set of questions we ask and criteria we use. We have thus been able to be more concrete with new hires about their responsibilities and the expectations on them.

Where we stand

We feel that for the cause of international aid specifically, we are approaching the point where another person can maintain and expand the report with little oversight from the co-Founders. However,

  • This applies only to our international aid report. Covering more causes will require heavy involvement from the co-Founders.
  • The role of maintaining the international aid report is still not well-defined enough that we feel it can be passed on from one person with a given set of credentials and qualifications to another, reliably and systematically. At this point we would stick to saying that Natalie Stone can maintain it with relatively little oversight.

In short, our research process is not at all robust.

What we can do to improve

Our experience to date has been that hiring and training people (a) involves substantial time investment by the co-Founders; (b) can easily fail to add value to the organization; (c) is more likely to succeed when we are able to be concrete rather than open-ended about employees’ responsibilities, which in turn requires that we have a well-defined research process (i.e., many available examples of charity reviews; a consistent set of questions we ask and criteria we use). With that in mind:

  • We can research more causes. We expect research on new causes to be initially open-ended and improvisational (as past research has been), but we also expect to gradually produce more consistent criteria, rules of thumb, and examples, with which we can train others to do the research.
  • We can continue hiring people and training them by slotting them into our research process, where we have well-defined enough responsibilities to make this work.
  • We can attempt to make our process more formal, to the point where much of what people need to know is captured in writing and as little as possible relies on specific people. We feel at this point that work in this area would be premature. As we work to train new hires, we learn more about how to communicate and formalize our process.

Does GiveWell present its research in a way that is likely to be persuasive and impactful (i.e., is GiveWell succeeding at “packaging” its research)?

Progress since 11/08

A year ago, it was extremely difficult for donors even to understand our work, much less engage with it. We have put significant effort into improving the clarity, organization and credibility of our research, and our efforts/progress on this front are discussed in our previous post under the heading of “Is it practical for donors to evaluate and use GiveWell’s research in the areas it has covered?”

We have not put effort into making our research more emotionally appealing, psychologically persuasive, etc.

Where we stand

Our current content aims for clarity over persuasiveness. We feel that our clarity is now reasonably strong (as discussed in our previous post under the heading of “Is it practical for donors to evaluate and use GiveWell’s research in the areas it has covered?”) But our presentation remains dry, making next to no attempts at emotional engagement.

What we can do to improve

  • We can make a concerted effort to “repackage” our existing content, presenting the same ideas in more emotionally compelling ways. Such an effort would involve thinking and working in very different ways from what we’ve spent our time on to date, and may require substantial involvement from people who are more focused on, experienced with and skilled in communications than anyone on our current staff.
  • A lower-cost, “low-hanging fruit” version of the above option is to put more effort into integrating our main website with our blog – we have created a lot of content on the blog, much of which is not duplicated in any way on the main website and is interesting/engaging in ways that the main website is not.
  • We can obtain and collect endorsements from people whose support donors are likely to find meaningful. Doing so would overlap heavily with the activities described in our previous post under the heading of “Is it practical for donors to evaluate and use GiveWell’s research in the areas it has covered?”

Does GiveWell reach a lot of potential customers (i.e., is GiveWell succeeding at “marketing” its research)?

Progress since 11/08

Our metrics file shows that traffic to our site has grown significantly over the past year. Several factors have likely contributed to the improvement, and it is difficult (and in our view not necessary) to break down the improvement between them:

Where we stand

Our website traffic is still a fraction of the traffic commanded by two other donor resources, Charity Navigator and GuideStar, and our research is not used in any significant “partnerships.” Our impact remains small.

What we can do to improve

There are many possible strategies for expanding our audience.

  • We can focus on improving the persuasiveness and clarity of our research, as discussed in the previous section.
  • We can pursue partnerships with other donor/consumer resources.
  • We can pursue partnerships with donor-advised funds and wealth advisors, which might connect us to wealthy individuals seeking help with their giving decisions.
  • We can pursue partnerships with people and organizations that focus on fundraising for particular causes, rather than particular organizations. GiveWell’s research could help such organizations fundraise for outstanding organizations within their causes (and outsource responsibility for finding such organizations and justifying their choices).
  • We can pursue partnerships with corporate giving programs.
  • We can advertise directly.
  • We can make concerted efforts at “earned media,” i.e., focus our research on issues that are likely to interest the media (which are frequently the opposite of the issues we find most interesting and important) and engage in concerted public relations efforts around these issues.

Is GiveWell a healthy organization with an active Board, staff in appropriate roles, appropriate policies and procedures, etc.?

Progress since 11/08

As of 11/08, we were operating without an official Executive Director, reflecting the fact that we felt current staff did not have adequate resources in place for professional development and supervision (see our 9/08 meeting).

Our Board of Directors consisted of only five people: Bob Elliott (then the Chair, but unable due to his schedule to remain engaged in the project, which ultimately led him to resign his position in July 2009); Greg Jensen (then the Treasurer), Holden Karnofsky (Secretary), Lindy Miller and Tim Ogden.

Since then,

  • We have substantially added to and changed the composition of the Board (see our current Board members and positions). The Board now has seven members, each of whom is highly accessible and engaged with the project.
  • We have replaced biweekly email reports with a monthly conference call, substantially increasing engagement from the Board. Turnout at conference calls has been high.
  • We have substantially increased the ongoing professional development and supervision of staff, leading the Board to confer the co-Executive Director role on the two co-Founders. (See our 7/09 meeting.)
  • We have adopted key policies.

Where we stand

Unlike a year ago, we feel that our Board of Directors and our policies are appropriate for this stage of our project. We voluntarily comply with all BBB Wise Giving Alliance standards.

What we can do to improve

We are always open to acquiring more Board members and mentors with substantial experience and ability to contribute to our project.

What is GiveWell’s overall impact, particularly in terms of donations influenced? Does it justify the expense of running GiveWell?

Progress since 11/08

As discussed above, we have seen substantial improvement in our “donations influenced” figures. A couple of questions remain as to the ultimate value/impact of influencing donations:

  • Where would the donors we’re influencing be giving, if not for GiveWell?
  • How does the impact-per-dollar of GiveWell’s recommended charities compare to that of the charities our “customers” would otherwise be giving to?

We have only hunches regarding the first question, and at some point should do a formal survey of our donors (including factual questions about their past giving as well as hypotheticals about where they would give without our existence) to get more meaningful information.

On the second question, we have done our best to answer “What do you get for your dollar?” for our top charities – however, answering this question meaningfully for our non-recommended charities is much more difficult (non-recommended charities are generally not recommended because so little information is available about what they’re accomplishing). We have argued that differences in charities can be enormous.

It is also worth addressing our impact on general perceptions regarding, and public discussion of, giving.

Where we stand

GiveWell is not having nearly the impact that we would need to see in order to consider it a success. We still consider our project an experiment at this point.

What we can do to improve

The key paths to increasing GiveWell’s impact (as we see them) are laid out in the earlier sections of this post. Working on any of our many possible areas for improvement could lead to increased impact – the challenge is prioritizing. We’ll discuss broad priorities in a future post.