A crucial consideration, when evaluating a giving opportunity, is: “Who are the people involved?”

I believe that any given project is likely to run into many factors that are unexpected (or can’t be captured in the evaluation process), and indeed that such unexpected factors often dominate the expected factors. Therefore, no matter how promising a project looks on paper, I expect it to fail if the people behind it aren’t capable enough; conversely, if the people behind a project are sufficiently capable, even a project that sounds terrible to me could turn out to be excellent (or be adjusted as it progresses to be excellent) based on factors that I fail to understand at the time of evaluation. Bottom line – I expect funding great people to lead at least to good (if not great) things, and funding incompetent people to lead to failure, in many cases regardless of most other factors. (Two important counterpoints: the choice of sector is important, as great people often work on issues they’re passionate about and the good they can accomplish can be limited by their choice of issue, and room for more funding is always crucial: even the most capable people generally have limits to how much funding they can productively use).

I’ve believed this for a long time, but GiveWell hasn’t used “evaluation of people” as a significant explicit factor in our evaluations (though we have taken the position that much of what we look for implicitly selects for good people). Lately, we’ve found ourselves putting more explicit weight on our impressions of an organization’s people. This post discusses why we’ve previously been hesitant to explicitly weigh and discuss this factor, what has changed, and how we evaluate people.

It’s worth noting that GiveWell’s process to date has contained a great deal of implicit evaluation of the people involved in the charities we’ve examined.

• One of our main goals is evaluating a charity’s track record. We’ve argued in the past that evaluating an organization’s track record can be a good way to evaluate its people. In the nonprofit world, there’s an additional wrinkle in assessing track records, which is that most nonprofits don’t have the data to make such assessment possible; we think those that do are likely to have important values in common with us, as discussed below.
• More generally, we tend to ask many critical questions of a given charity, and to publish its responses along with our evaluation of those responses. We feel that asking critical questions about a person’s work and evaluating how much the answers seem to make sense is another good method for evaluating people.
• Our emphasis on transparency has created an interesting way of distinguishing between charities: some groups take a near-“open book” attitude, others aren’t willing to put any of their materials or communications with us on the record, and others fall somewhere in between. We believe that this distinction may (imperfectly) capture important values, as discussed below.

That said, there are observations we make and opinions we form of people that don’t necessarily show up on their organizations’ GiveWell reviews. We notice not just how good an organization’s track record is, but how intelligently (from our perspective) its staff discuss the relevant issues; we notice not just how good its final answers to our critical questions are, but how efficient are the exchanges leading up to these answers. We form impressions that can’t always be pinned down to specific (or public) occurrences.

So far, we’ve been hesitant to put much weight on, or publicly discuss, these sorts of factors. A major hesitation is that we’re wary of putting too much weight on informal, intangible impressions that are likely to end up being “tests of how similar charity representatives are to us” rather than “tests of how well-suited charity representatives are to do their work.” It seems easy for evaluations of people to end up being driven by charisma, by similarity to the evaluator, and/or by characteristics that don’t translate well to nonprofit work. (Regarding the latter: many philanthropists have been successful in other domains, and are used to being able to pick out people who can succeed in the domains they know well; we fear that they aren’t able to build up the same experience and reliability of intuition in the nonprofit world, where it’s hard to know who has and hasn’t succeeded, and that the qualities they’re looking for may not translate well.)

Despite the concerns above, we’re now starting to factor our views on people more heavily into our prioritizations and recommendations, and we’re starting to feel that it’s important to write explicitly about our views on this front (not just about charities’ performance on our standard criteria). This is because

• We’ve had enough experience to start feeling that we recognize patterns in what sorts of people/organizations eventually do well vs. poorly in our process, and as our process has improved we’ve become more confident in its conclusions.
• Improvement in our process has also led us to have less confidence in any given empirical case (example). We’ve come to feel that we need to supplement empirical analysis with whatever other sources of views are available, even if they’re somewhat intangible.
• We’ve been thinking more about how to evaluate charities without relying on our “proven cost-effective” framework; when doing so, explicit evaluation of people is more important.
• We’ve been learning how rare it is that one can see how a charity has performed over time; because of this, it’s become more important to us to support groups that could contribute to changing this, i.e., groups that share some of our core values such as valuing self-evaluation and transparency.

That said, the concerns listed in the previous section still apply, and thus we are still cautious about placing heavy weight on our reads of people.

We look for the following qualities:

• Clear, direct communication. We view clear, direct communication as a sign that someone welcomes – rather than prefers to avoid – (a) substantive discussion of their work; (b) honest criticism leading to improvement. It’s also generally easier for us to communicate with people who have this quality, which makes us more confident in our evaluation of them and their organization.
• A sense that the person is following their own agenda rather than catering to ours. We try to avoid people whose primary goal seems to be telling us what we want to hear. If someone seems averse to contradicting or criticizing us – or seems more interested in the question of “what plans GiveWell would like the organization to carry out” than “what plans the organization would like to carry out” – we don’t consider this a good sign. We prefer organizations that set their own agenda and directly engage our questions.

  Clear, direct communication is a positive sign on this front, particularly when it includes criticizing, contradicting, and educating us.

  (We discussed this factor in a previous discussion of our stay in India.)

• Thoughtful, plausible answers to our critical questions. In our view, one of the marks of an exceptionally capable person is that they have thought extremely deeply and thoroughly about the work they specialize in, to the point where they have thought of nearly any critical question that we could raise, and have thought about it more deeply than we have. This doesn’t necessarily mean they will be able to give an answer we find fully satisfying on any given critical question, but it means they will usually give an answer that shows they have thought hard about the issues and have an answer that we could plausibly find satisfying if we knew more.
• Personal and organizational history. We look for generally impressive accomplishments and for experience relevant to the work being done. How we weigh these factors is highly context-dependent, and particularly depends on the organization’s mission. There are some types of work for which we’re comfortable with people who have little directly relevant experience, and others for which we find the personal and organizational resume to be more important.
• Self-skepticism and valuing self-evaluation. This is an area where we see a significant disanalogy between for-profit and nonprofit work. When speaking to for-profit investors, we’ve heard the sentiment expressed that it’s natural and healthy for an entrepreneur to be overconfident; we can imagine that in the for-profit world, feedback loops and course correction often play out despite overconfidence, since people know they need to make adjustments when their deliverables (traffic, revenue, profit, etc.) are below expectations. But in the nonprofit world, there’s often (or always) no way to learn about whether your work is helping people unless you make a substantial, explicit effort to learn. People with high confidence and charisma and low self-skepticism worry us: we fear that they will continually succeed in raising money from donors but will never do the investigation and adjustment necessary to optimize their work for helping people.

  From what we’ve seen, it’s extremely rare to find people in the nonprofit world who (a) concede many possible ways in which their programs could be failing and (b) have put substantial thought and effort into investigating the relevant issues. People who do have these qualities tend to produce unusually meaningful and informative self-evaluation. We value such self-evaluation greatly not just because it’s informative but because of what it reflects about the people involved.

• Valuing transparency. While there are sometimes good reasons to keep particular information confidential, a general preference to share more information publicly is likely, in our view, to go hand-in-hand with (a) self-skepticism and interest in being critiqued; (b) an interest in creating public goods (i.e., helping others to learn from one’s own experience) and helping the world as a whole rather than in focusing on the success of one’s organization. We see significant, tangible variation in attitudes toward transparency: some organizations want everything kept off the record, others are willing to share some information but wish to withhold whatever does not reflect well on them; others take more of an “open book” attitude.

Perhaps not surprisingly, these qualities mirror the qualities we believe ourselves/GiveWell to have. This makes sense for a couple of reasons. First, we’d guess that people are usually most impressed by people who are similar to them in important ways – sharing basic values, sharing communication styles, etc.; while we’d like to be able to identify people who are outstanding at what they do despite being poor at communicating with us, this is simply practically difficult. Second, we perceive ourselves as bringing important and undervalued values and qualities to the nonprofit sector, and part of what we find ourselves looking for in charities is others who will spread similar values and qualities.

We recognize that we may be overrating the importance of the qualities discussed above, but we feel that we maximize our own value-added by pursuing and promoting them, and part of the goal of this post is to ensure that our audience knows these are the values we’re seeking to pursue and promote.
We believe that other funders often look for substantially different qualities – in particular, they often seem to place more weight on charisma and fundraising ability, perhaps because they are hoping their investment will be leveraged into donations from many others. There is sound logic behind this preference. However, our feeling is that the current nonprofit sector overemphasizes storytelling ability and fundraising success, relative to thoughtfulness and transparency – with the result that we see a lot of the former and less than we’d like to see of the latter – so the qualities we look for are different..

Over the coming weeks, we will be adding sections to the pages on our top charities that discuss our views on each organization as a whole (including its people).