This post is more than 7 years old

In recent years, we’ve added a new source for potential GiveWell top charity recommendations: GiveWell Incubation Grants. This post will highlight a GiveWell Incubation Grant recipient, No Lean Season, that we see as a top charity contender for 2017.

GiveWell has traditionally identified our top charities through our standard process, during which we examine a charity’s track record and funding needs. Our goal with GiveWell Incubation Grants, outlined in an earlier blog post, is to grow the pipeline of potential future top charities, in part by supporting organizations at an earlier stage than we would traditionally consider them for a top charity recommendation. We generally expect there to be a lag of a few years between receiving an Incubation Grant and being considered for a top charity recommendation.

Good Ventures, a large foundation with which we work closely, funds GiveWell Incubation Grants. Good Ventures made its first Incubation Grant to No Lean Season in 2014 and we now believe it is a top charity contender when we update our recommendations at the end of 2017. We’re planning to highlight another 2017 contender, Zusha!, in a future post.

Seasonal income support program

No Lean Season offers subsidies to low-income agricultural workers in Bangladesh to incentivize them to temporarily migrate from rural areas to urban areas, where they may earn higher wages seasonally. These subsidies (which may be made as grants or loans) are around $8-19 USD and cover travel costs and a couple days of food. Follow-up studies found that individuals who once received an incentive to migrate chose to do so again—without a subsidy—at a higher rate than would otherwise be expected, suggesting they found migrating to be useful.

The below infographic from No Lean Season shows a high-level overview of the intervention (click for detail):

Evidence Action, the parent organization of GiveWell top charity Deworm the World Initiative, started No Lean Season as part of Evidence Action Beta, its program to test interventions that could be significantly scaled up.

How we decided to support No Lean Season

We approached Evidence Action in late 2013 to express our interest in supporting the creation of new GiveWell top charities.

In March 2014, Good Ventures made a $250,000 grant to Evidence Action to support the investigation and scale-up of promising programs. Since then, Good Ventures has made three additional grants totaling approximately $2.7 million to support the program’s scale-up; the write-up for the most recent grant, made in December 2016, is forthcoming.

No Lean Season as a GiveWell top charity contender

We assess potential GiveWell top charities along four criteria: evidence of effectiveness, cost-effectiveness, transparency, and room for more funding. No Lean Season appears as a plausible contender when reviewed along these dimensions. We plan to spend significantly more time reviewing No Lean Season this year as we move forward in our top charity review process and update our views on its work.

Evidence of effectiveness. A number of randomized controlled trials (RCTs) have studied the effects of seasonal income support in northern Bangladesh, where Evidence Action is scaling up the program. Trials conducted in 2008 and 2014 found significant effects on household expenditures and income, respectively, during the relevant season. In addition, later follow-ups of households that were incentivized to migrate found that they did so again at higher rates, even in the absence of a continued incentive; Mushfiq Mobarak, a Yale economics professor and a lead researcher on No Lean Season, estimates that the effects persist for three additional years. However, the 2008 and 2014 studies did not measure directly comparable or combinable outcomes, so we can’t compare them or combine the results. We take these RCTs as strong evidence that No Lean Season’s seasonal migration subsidies lead to improved economic outcomes in northern Bangladesh.

Analysis is not yet complete for a separate RCT, conducted in 2013, a year when labor unrest was unusually high (see Figure 1). Given the possibility of mitigating circumstances, we’re unsure how informative the 2013 RCT results will be to predicting future success of the program.

Potential risks of the program could include negative impacts at the destination labor market (e.g. on job availability or food prices) or vulnerability of migrants or family members left at home after migration. As of January 2016, our impression was that Evidence Action planned to monitor possible negative effects on the destination labor market; Evidence Action said it had not found indication in surveys that migrants or their families were less secure (see p. 4).

Evidence Action is planning to run an RCT at scale during the 2017 lean season. Due to our current best estimate of the program’s cost-effectiveness and expectation that No Lean Season will collect and share high-quality monitoring data from its 2016 work (discussed below), we think the evidence base may be sufficient for the organization to qualify as a top charity at the end of 2017, before results from this RCT are available.

Cost-effectiveness. We currently estimate that No Lean Season is between 5-14 times as cost-effective as direct cash transfers, a baseline we use for comparison among global health and development interventions. At scale, we estimate that individuals will experience a consumption benefit of $15 for every $3 No Lean Season spends. These benefits and costs are averages over the population that is eligible for and offered the program; we believe that the benefits are actually larger for households that send a migrant and smaller for households that don’t. We’re very uncertain about the baseline per capita lean season consumption in this population, but for comparison purposes, we estimate it at roughly $116 for the entire 5-month lean season. We also expect, based on previous studies, remigration without further incentive for about two future years, as well as at least one additional migration during the lesser lean season.

Our estimate of No Lean Season’s cost-effectiveness is in the range of our current top charities. We believe that the four deworming charities we recommend are ~4-10x as cost-effective as cash transfers, and the two charities we recommend for their work to prevent malaria, the Against Malaria Foundation and Malaria Consortium, are ~4x as cost-effective as cash transfers. However, our cost-effectiveness estimates typically become worse—the cost-effectiveness decreases—as we spend more time on our analysis and incorporate additional inputs and discounts. We expect this is likely to occur with our current estimate of No Lean Season’s cost-effectiveness, as well.

Monitoring. We have not yet reviewed No Lean Season’s monitoring. However, we are quite familiar with its parent organization, Evidence Action, as a result of our recommendation of the Deworm the World Initiative as a top charity since 2013, and our previous reviews of its work, and feel confident Evidence Action will share its monitoring of No Lean Season based on its track record. We expect this monitoring to be of a high quality.

Organizational strength and transparency. We have a positive view of Evidence Action as an organization, based on our significant experience communicating with its staff. Our impression is that Karen Levy, Evidence Action’s Director of Global Innovation and Beta, played a key role in scaling the Deworm the World Initiative, suggesting the organizational capacity exists to similarly scale a program like No Lean Season. We also believe that Evidence Action, based on its track record and our experience, will operate No Lean Season transparently.

Room for more funding. Evidence Action currently estimates that No Lean Season could productively use approximately $16 million over five years in Bangladesh and Indonesia, and additional funding to expand to India and Ghana; this estimate may be adjusted in the future.

We remain unsure whether this program will be successful in locations beyond Bangladesh. If it isn’t a good fit in other locations, No Lean Season’s overall room for more funding could be quite limited.

Progress to date and future plans

As of early 2016, Evidence Action planned to scale up its program in Bangladesh to offer a total of 16,000 subsidies and reach 9,000 households with its implementing partner, RDRS Bangladesh, in 2016, the first of a four-year scale-up. By 2019, No Lean Season provisionally plans to offer ~295,000 subsidies and reach ~165,000 households in Bangladesh (see p. 5 here; No Lean Season staff plan to update these figures going forward).

Evidence Action is also exploring the possibility of working in locations beyond Bangladesh. It visited Zambia and Malawi in 2014 to assess whether the program might help alleviate seasonal hunger in those locations. Our understanding is that Evidence Action is not planning to expand in Malawi based on its findings. We believe Evidence Action is also not planning to scale up in Zambia.

Mobarak, the No Lean Season researcher, has conducted two research studies in Indonesia, which he says suggest the country may have similar underlying conditions to Bangladesh. As of August 2016, Evidence Action was also considering expanding into India, particularly states close to Bangladesh, although it had not yet done any research there. Evidence Action also considered Ghana as a potential future location, although we are not aware of concrete plans to begin implementation there.[1]

Path to GiveWell top charity

By November 2017, we expect to see results from No Lean Season’s first year of a four-year scaling effort (the September-December 2016 seasonal effort to reach 9,000 households described above). This, combined with the fact that there are multiple rounds of randomized controlled trials in the past and a large forthcoming RCT at scale, maybe sufficient for No Lean Season to qualify as a 2017 top charity.

Notes

[1]
“Indonesia

Mushfiq Mubarak has completed two research studies related to No Lean Season in Indonesia:
1. An exploratory study, similar to those done in Zambia and Malawi.
2. A small-scale pilot in West Timor conducted by the Southeast Asian office of the Abdul Latif Jameel Poverty Action Lab (J-PAL). This study was similar to a previous trial in Bangladesh, but was not a randomized evaluation.

The studies’ initial results are promising. Indonesia appears to have similar underlying conditions to Bangladesh, including: lean season migration; availability of jobs in urban areas; and migration from rural to urban areas and between islands.

India

No Lean Season is also considering expanding into India but has not yet conducted research there. Its first activity in India would likely be a small-scale project, such as a pilot in one village. The underlying conditions in Indian states bordering the Rangpur region of Bangladesh are similar to those that exist in Rangpur.

Ghana

No Lean Season is considering expanding into Ghana but does not yet have concrete plans to initiate a project there. In general, it does not intend to expand into new countries until it has sufficient capacity to do so.”

From a conversation with Dr. Karen Levy and Guillaume Kroll, August 2, 2016, p 1.