We’re often asked why GiveWell doesn’t recommend any organizations that focus on providing surgeries. This post will describe:

• Work we did previously to try to find surgery charities to recommend. In brief, our inability to identify organizations with room for more funding and high-quality monitoring data prevented us from recommending surgery charities in general.
• Our current (rough, preliminary) view that cataract surgery’s cost-effectiveness may be competitive with that of our priority programs, and some of the major open questions we have about our estimate.
• Organizations implementing cataract surgery programs that we’ve spoken with. They run a variety of programs, and our impression is that they do not yet have the type of high-quality monitoring information we’re interested in.
• Our plans to move forward with IDinsight to improve our understanding of cataract surgery as an intervention.

Background

Our impression is that surgical interventions are intuitively attractive to many donors because they seem to offer concrete, low-cost, and life-changing impacts. For example, some organizations performing cataract surgeries claim that each surgery costs approximately $25 (e.g. 1, 2). Fistula surgery organizations, which repair obstetric fistulas, cite costs around $450-$600 per surgery (e.g. 1, 2). These figures would likely be competitive with our cost-effectiveness estimates for our current recommended charities, a critical factor in GiveWell deciding to recommend them. This suggests that we should look into recommending organizations working on these interventions.

We briefly looked at developing-world corrective surgery as a potential priority program in 2010. However, we identified two major challenges to finding top charities working in this space:

1. Room for more funding. We were unsure whether directing additional funding to a charity would cause more surgeries to happen, or if something other than funding—such as the availability of surgeons—was the bottleneck to further surgeries being performed.
2. High-quality monitoring. Many charities don’t conduct the type of high-quality monitoring that we’d like to see, including surgery charities. We’re particularly interested in closely monitoring surgical outcomes due to our view that surgical interventions are complex relative to, for example, the distribution of a mass commodity like a deworming pill or cash. Performing surgery requires skill and we think it’s likely that the quality of surgeries varies. In addition, some surgeries may require longer-term follow-up care, and we’re unsure what the impact is for patients who do not receive this care. Monitoring information thus feels particularly important in our analysis of whether to recommend charities working on surgery.

We deprioritized additional work on corrective surgeries at the time that report was published, although we maintained our interest in potentially recommending charities working in this area. In 2016, we completed an evidence review of cataract surgery and classified it as one of our “priority programs.”

Cataract surgery as an intervention

We believe there is evidence that cataract surgeries substantially improve vision. Very roughly, we estimate that the cost-effectiveness of cataract surgery is ~$1,000 per severe visual impairment reversed.[1]

However, we have not completed an in-depth cost-effectiveness analysis of cataract surgery. We remain highly uncertain about the full costs involved because our current cost estimates are based on literature on the costs of performing surgeries. In our experience, charities’ own budgets (rather than academic literature) have given us the best information about how much an intervention costs.

In addition, we also currently have limited information about the preoperative visual acuity of cataract surgery patients.[2] And we do not have a good understanding of the progression of cataract to blindness—we have looked for this information but have not found it—and so we do not currently incorporate an estimate of the benefits of preventing future blindness in our cost-effectiveness estimate.

Why we don’t focus on trachoma

Another sight-related intervention is surgery to treat trachoma, a bacterial infection commonly transmitted by flies that can result in low vision and eventually blindness from scarring due to eyelashes rubbing on the cornea.
Our impression is that the evidence base around trachoma progression is weaker than cataract surgery, such that we have open questions around the likelihood of the infection progressing from its earlier stages (trachomatous trichiasis, where the eyelashes are rubbing against the cornea) to blindness, and the average age of onset of each. In addition, we have concerns about trachoma recurrence; because trachoma can recur, surgery to repair vision loss from trachoma may be less cost-effective than cataract surgery in the long run. Cataracts do not recur because the surgery replaces the natural lens, although individuals who have had cataract surgery may still experience vision loss due to other causes.

Organizations implementing cataract surgery programs

The organizations we spoke with as part of our investigation into cataract surgery run a variety of programs. Many of them were not directly implementing additional surgeries, but rather were conducting activities such as supporting trainings for surgeons, providing general support to hospitals for eye care interventions, or encouraging more people to access available health services. We have not yet seen from organizations compelling monitoring and evaluation to demonstrate their impact.

Our plans with IDinsight

We’re working closely with IDinsight as part of GiveWell’s Incubation Grants program to grow the pipeline of potential future top charities. IDinsight conducts impact evaluations with the goal of informing decisionmakers, such as governments or NGOs. (More on why we’re partnering with IDinsight in this post.)

We partnered with IDinsight to find a cataract surgery organization it can work with on monitoring and evaluation, as that remains one of the biggest obstacles we’re aware of to GiveWell recommending cataract surgery organizations. (As indicated above, we have spoken with a number of cataract surgery organizations but do not believe any have sufficient monitoring and evaluation information available to inform a GiveWell recommendation.) We expect IDinsight to consider a number of organizations by holding initial scoping calls, and to ultimately focus on working with a single organization that appears most likely to become a GiveWell top charity, although that organization may still not become a top charity. We expect IDinsight to focus on:

• Key monitoring questions, such as measuring pre-operative visual acuity as well as post-operative visual acuity.
• Conducting an evaluation of programs’ causal impact. Does the organization cause more surgeries to happen?

We’re also planning to ask other cataract surgery organizations for more detailed information about the costs of their programs. We hope that between IDinsight’s work and receiving additional cost information, we will better be able to assess whether cataract surgery should continue to be a GiveWell priority program.

Notes

[1] This estimate is on the higher end of the range we calculated, because it assumes additional costs due to demand generation activities, or identifying patients who would not otherwise have known about surgery. We use this figure because we expect that GiveWell is more likely to recommend an organization that can demonstrate, through its demand generation activities, that it is causing additional surgeries to happen. The $1,000 figure also reflects our sense that cost-effectiveness in general tends to worsen (become more expensive) as we spend more time building our model of any intervention. Finally, it is a round figure that communicates our uncertainty about this estimate overall.

[2] Visual acuity is reported as the ratio of the distance at which someone can distinguish a fixed detail relative to a person with “normal” vision. A ratio of 6/6 refers to “normal” vision; a ratio of 6/60 means that someone with impaired vision sees at 6 meters what someone with “normal” vision sees at 60 meters. The World Health Organization (WHO) defines binocular blindness as visual acuity worse than 3/60 in both eyes.

Visual acuity thresholds for surgical eligibility vary. Our understanding is that some portion of cataract surgery is done on individuals whose visual acuity is worse than 6/6 vision, but better than 3/60, and that this proportion likely varies by program and context. For the purposes of our cost-effectiveness estimate, we’ve assumed treatment of patients whose visual acuity is 6/60 or worse.