A few links from the past month that we aren’t doing full posts on, but might be of interest for people interested in effective giving:
- Via Chris Blattman, a randomized controlled trial “find[s] no evidence that teacher incentives increase student performance, attendance, or graduation, nor … any evidence that the incentives change student or teacher behavior” (note this study took place in the U.S.) This can be added to a long list of high-quality studies with disappointing outcomes in the area of improving education in the U.S. (For a relatively recent review of these studies, see page 2 of a book chapter linked from the same post.) We think it’s important to look for rigorous evidence of effectiveness when giving to improve education, rather than assuming that logical-seeming programs will be effective.
- A recent study (PDF) argues that KIPP, one of our top U.S. charities, relies on high attrition (i.e., many of its students leave over time) and high funding to achieve its results. KIPP responds with objections to the study’s methodology. I haven’t thoroughly reviewed the study or the response.
My first impression is that the study is indeed seriously flawed, as KIPP suggests, but it’s also important to note that our endorsement of KIPP does not depend on the idea that it can serve all students or can operate on the same terms as public schools. For us, it’s rare enough – and valuable enough – to find a charity that can reliably improve education outcomes and narrow the achievement gap.
- Felix Salmon refers to Michael Bloomberg’s promotion of giving to Japan as “philanthropy theater”: “The point is really to be seen to be doing good, to feel as though you’re making a difference.” I agree with this characterization of such giving, and I think it’s a good example of giving that’s all about the giver, not the recipient.
- Sean Stannard-Stockton discusses his “four approaches to philanthropy” and makes a point that I think is important: “effective giving” is often conflated with what he calls “Strategic Philanthropy,” in which the funder diagnoses social problems and designs solutions themselves, seeing nonprofits as mere contractors. We find this a particularly challenging and un-promising form of giving, and we prefer what Sean calls “charitable giving”: giving to nonprofits that have found their own solutions, already know what works, and are able to help more people with more dollars.
- Aid Watch publishes unsatisfying responses from World Vision to its questions about giving away shirts. The exchange is reminiscent of many of our past exchanges with charities.
Comments
I have to say that finding an absence of evidence that teacher incentives increase student performance has got to be about the least surprising thing one could imagine. It is far more surprising that there were any previous circumstances in which such incentives actually produced a performance improvement.
The notion that financial incentives for two years could motivate teachers to improve their performance, and thereby improve student performance, presumes a classical theoretical economic model of purely rational human behavior. (And even at that, it does a poor job of addressing that classic model of behavior, since apparently almost all of the schools rewarded all teachers equally, if a school’s performance improved.)
There is a large and compelling body of evidence, however, that human behavior not only doesn’t match the classical model, but in fact, that incremental financial incentives can have precisely the opposite effect on human behavior – what Daniel Pink, in _Drive_, for example, calls a reverse Tom Sawyer effect: turning play into work. Like many jobs, being effective as a teacher requires skill, passion and effort – and normally its passion that drives the investment of effort. There’s significant evidence that creating a financial bonus system tied directly to performance results undermines passion, and creates a perception of the work as a chore because it is done for financial reward, rather than personal interest. (Accord to Pink, these kind of incentives are only effective for fairly unskilled, uncreative jobs, that involve a high level of routine and a low level of autonomy. In other words, if the job involves tasks that are already viewed as a chore, then immediate financial incentives may motivate better performance. But for creative jobs, such financial incentives reduce the job to a chore, and have a detrimental effect.)
Furthermore, as the author of the study notes, the kind of incentive system tested presumes that the essence of the problem is that teachers don’t work hard enough to achieve results, and therefore, that motivating them to work harder (assuming the kind of financial incentives used actually had that effect) will improve performance.
In fact, I’d argue that the problems are much, much broader than that…but the full litany of problems is beyond the scope for this comment – it’s already going to be way too long!
To the degree that financial incentives are actually an issue, however, the kind of financial incentives that I think one might reasonably expect to be effective are much longer-term incentives. For example, raising base compensation to a level sufficient to interest more people in being teachers, and to make teachers comfortable that they are adequately compensated. (FYI, IMO, this is generally not a problem here in NY, though it has certainly been an issue in other places I’ve lived, such as NC.) Coupled with demanding performance expectations (and a probably a higher degree of autonomy than is currently allowed today), you probably have much greater success in attracting great candidates to pursue teaching, and in ensuring that only high performers remain as teachers.
To clarify the point, I’d draw an analogy to professional sports. It’s quite common today for star athletes to have contracts that award them a bonus if they meet certain performance goals. But does anybody seriously believe that those bonuses motivate greater effort and performance on the part of those star athletes? Would anybody expect a poorly performing team to suddenly improve it’s performance because the athletes were all given performance incentives? (Or for that matter, simply by increasing their salaries?) Anybody that does believe such, is just nuts!
(The fact is, those bonuses are really just a mechanism for resolving a disagreement about what the value of an athlete is to a team. The team is comfortable that the athlete is worth X. The athlete thinks he is worth X+Y. So they agree that he will be paid X, but if he actually does perform at the outstanding level he anticipates, then he’ll be paid the additional Y, as a bonus. It has nothing to do with motivating him to perform. His passion had better be his motivation, or he’s almost certain to fail.)
Teams don’t get better by applying incentives, or giving higher compensation generally, to the existing athletes. Rather, the general level of performance has increased (at least in major sports) over time because the salaries for professional athletes are high enough to encourage lots of people to pursue those sports and invest in developing effort. And by having many people competing for professional sports opportunities, you end up with a much more robust supply of talent than if few people were pursuing such.
Bottom line being, I find it rather strange if anybody believes – or if any experiment suggested – that merely creating a system of performance bonuses would produce significant performance improvements, over a short period of time.
Comments are closed.