A donor of ours earmarked $10,000 for regranting to a local charity in India, and in deciding how to give this away (and for general learning) we conducted 20+ site visits to small NGOs during our travels. In a sense, this was a chance for us to try out a more traditional method of giving: heavily based on referrals, site visits, and informal impressions rather than desk research.

This post, a followup to posts on our general thoughts on India (from myself, Elie and Natalie) summarizes our thoughts from these site visits and this decision. Note that we have posted detailed notes (and in some cases pictures) from the site visits at the official page of GiveWell’s trip to India.

Criteria

Our normal criteria weren’t a good fit with the organizations we visited, most of which were tiny and without the capacity for extensive monitoring, evaluation and documentation. So we followed the same basic principles that led us to our original criteria: look around, ask questions, and try to articulate our reactions in ways that lead to consistent principles.

The questions that ended up mattering most to us were:

Is the organization serving a population that is clearly in need? Is the organization run by people who seem thoughtful, competent and well-intentioned? These are the questions that seem most amenable to being answered by site visits, and need little explanation.

To what extent do the organization’s activities flow from clients’ needs, as opposed to donors’ strategies? It’s easiest to articulate the categories we mentally placed organizations in through a few examples:

• Room to Read, a large international NGO, seemed focused, to its core, on supporting and promoting libraries. I’d refer to this as a “strategic” organization, consistently pursuing a particular theory of change.
• Helping Hands, a small community NGO, seemed much more “improvisational.” When we asked the woman running it how she chose whom and what to fund, she told stories of individuals and one-off events (girls who needed money, schools that needed new supplies, etc.)
• Seva Mandir, an NGO working in many areas around the city of Udaipur, had a set of relatively large-scale programs, each of which seemed (from conversations) to have sprung from specific, consistent, repeated needs and requests of clients. Its described process for deciding which programs to execute revolved around formal village meetings. I’d term this a “systematically bottom-up” organization.

The approach we felt most comfortable with was the “systematically bottom-up” approach. “Improvisational” organizations seemed to rely too heavily on specific personalities and relationships, while “strategic” organizations left us concerned about the extent to which they were run for clients as opposed to donors, and the appropriateness of their programs for the people with whom they were working. These concerns mean not that we would never recommend such organizations, but that the need for strong monitoring and evaluation (which our normal research process emphasizes and our site visits did not) becomes more central for them.

How insulated is the organization’s management from potential problems? Some organizations are structured for infrequent, limited interactions with clients, while others are built on constant and close contact with clients. Some organizations try to make very long-term or difficult-to-observe differences in people’s lives (examples: sports for character development, advocacy for changing sexual and other private practices), while others aim for more tangible, short-term help.

Both of these distinctions relate to what we came to refer to as the “insulation level” of an organization. Some organizations seem, by their basic structure, likely to become aware of any failure to help clients; for organizations without this feature, monitoring and evaluation become more important to us.

Related to this, we often (when we felt it was appropriate) asked staff to tell us about specific clients, and preferred organizations where they seemed to know a lot about the lives, histories and particular needs of individuals.

How do staff prioritize clients vs. funders?

At one orphanage we visited, we were told that the children were sad today because they couldn’t go outside; when we asked why they couldn’t outside, we were told that they were staying in to meet us. On another visit, we joined a village meeting and were told that the people had been waiting for us for over an hour. We were embarrassed and unnerved by these situations.

We feel strongly that clients ought to come first, which for small organizations (often without specialized fundraising staff) may mean that funders and other visitors have to wait their turn. The orphanage we felt best about was the one where the children assembled for a brief greeting and then dispersed to play computer games and otherwise enjoy themselves. The director of the organization we ultimately awarded the $10,000 to was clear and unapologetic about the fact that her work with clients was more important than her time with us (more below).

Thoughts on different types of organizations

The organizations we found ourselves most interested in were the ones that focused on orphans and/or street children. It seemed to us that these were the populations with the greatest risks and needs, and that the help they needed had a “low-insulation” quality: organizations working with them aim to ensure that they are having basic needs met, are being raised in healthy environments closer to “normal” than what they would have otherwise, and end up healthy and educated.

There are concerns about orphanages and the extent to which they may be taking children away from their families. We definitely felt more comfortable with some orphanages than with others (examples above), and as mentioned below, we ultimately felt most comfortable with a center that provided services to children (including food and shelter) while not taking complete control of their lives. One thing that does appeal to me about orphanages, though, is that the children in them seem generally to have excellent English – something that I believe is a very real and strong advantage in the Mumbai economy.

We were also very interested in organizations focused on keeping children out of the sex trade, simply because the need is so extreme and the goal seems so valuable to us. However, the specific organizations we visited in this category didn’t leave us feeling confident, partly because of the greater challenges one faces in knowing whether their activities are working.

Given the striking and visible needs we saw all around us, we were less interested in organizations focused on more intangible, theoretical benefits. For example, Magic Bus facilitates soccer programs for children; it’s hard for me to justify this focus when those same children are underfed and under-sheltered. I have heard the argument that sports have a character-building quality that ultimately results in more good being done, but this sort of reasoning strikes me as theoretical and romanticized, and I wouldn’t accept this hypothesis without strong evidence (which I do not believe exists).

We have mixed feelings about education-focused programs.

• The evidence that education is causally helpful for later success in life is surprisingly weak – there isn’t strong evidence either for or against the idea. Our intuition is that education is indeed very important; based on conversations we’ve had, it seems a strong prerequisite to many of the more desirable career tracks. More at our writeup on developing-world education.
• We’ve seen no organizations that can make a compelling case, using academic performance data, for their positive impact on education. The best we’ve seen in this area is Pratham, which we visited twice while in Mumbai.
• Our site visits to education charities left us with little sense of the quality of the programs; we weren’t even sure what to look for. See our notes on Pratham for more on this.

Salaam Baalak Trust
Salaam Baalak Trust was the organization we felt performed best on the metrics above:

• It is a drop-in center for children living on the street. Unlike orphanages, it encourages children to live primarily with their existing families, but it provides shelter, meals, counseling and other services for children to use as they wish.
• This basic program is strong on the insulation- and strategy-related criteria above, and the organization is small enough that the two site visits we made gave us a look at a good proportion of what it does.
• Its director, Dinaz Stafford, left a good impression on us.
  • We had a definite sense that she was putting clients before visitors: she interrupted a meeting to speak to a child who had come in, was 45 minutes late to another meeting (explaining she had been dealing with a child who was having issues) and told us we had to leave when it was time for her to attend a meeting about children who were struggling.
  • She also invited us to examine her case files (on condition that we not record or disseminate names), which gave basic information on each child’s family situation and health over time, and when we asked her questions about specific children, she was easily able to rattle off specifics about them.