A Primer on the Nashville Incentive Pay Experiment, Part 4

Part 4: What Can We Learn?


Previous posts:
Part 1: Background Info
Part 2: What to Look For
Part 3: Why it Matters


Despite what Rick Hess says, we won't learn "nothing" from the results of the study (but do read his post, as most of his points are good ones).  So what, exactly, will we learn from the results of the study?  It's hard to say exactly, but here are some things that we can and cannot learn from the study:

We can learn how individual teachers respond to a financial incentive offered for individual results.
We cannot learn how individual teachers, groups of teachers, or entire schools respond to financial or other types of incentives offered to groups of teachers or schools.

We can learn whether teachers will change their teaching in ways that will raise student test scores in response to an individual financial incentive.
We cannot learn whether teachers will change their teaching in ways that will increase student engagement, critical thinking, creativity or a myriad of other factors in response to an individual financial incentive.

We can learn whether middle school math teachers in Nashville were more likely to switch schools/districts or leave the profession if they were in the control or treatment group.
We cannot learn whether talented people across the country are more likely to become teachers, and subsequently remain in teaching, if there are performance bonuses in place.

We can learn whether, under this particular system, teachers who are "better" as measured by standardized tests across all three years tend to be rewarded on a year-to-year basis.
We cannot learn whether, under performance pay systems, better teachers (as measured in any number of ways) tend to be paid more.

In short, yes, there are rather severe limitations on what we can learn from this one study.  But, at the same time, I'd argue that we can learn more from this study than from most others.  Despite what Rick Hess, we will not learn "nothing of value," in part because different people value different things.  Hess might think that the teacher recruitment/retention aspect of performance pay is the most important, but plenty of others think the incentivizing of effort is the most important.

What does this mean for the interested observer watching from afar?  It means that your ears should perk up if when you hear strongly worded statements from both sides of the debate.  This study is one piece in the puzzle -- and an important piece, at that.  It's neither the end all and be all of research into performance pay nor an utterly useless waste of time that fails to inform the debate in the least.