How Good are Teach for America Teachers?

This was the question asked by a team of researchers at the Urban Institute. I happened to see the article presented at AERA, and have since read the full version.

A number of previous studies have found that TFA members were somewhere between just as good and slightly better at teaching math and between just as good and slightly worse at teaching English as regular teachers (based on test scores). One of the differences between this paper and others was that it included only high schoolers. To make a long story short, the authors found that students taught by TFA teachers slightly outperformed students taught by regular teachers -- even experienced ones. I previously alluded to coverage of this paper here.

I debated whether to discuss this further, but I've now seen discussion of it here, here, and here.

Everyody except for eduwonkette seems to just accept the findings and move on. The study was well done in a lot of ways, so I have no desire to trash it, but I've found the coverage really lacking. As with any study, there were significant weaknesses. Among these:

-there was no definitive way to match students to teachers -- they were pretty sure they had it right for 84% of the students and threw out the rest
-they only had one test per subject for each student, meaning they couldn't measure their growth over a year
-they compare TFA teachers to all teachers despite the fact that they teach very different students in very different schools -- I'd argue that they're doing jobs that aren't really comparable
-the number of TFA teachers was very small -- a total of 69 over the years, meaning that it was probably about 30-50 actual people

So what does this prove? It seems like a pretty good bet that TFA teachers are outperforming other high school teachers in North Carolina, but it's not a sure thing. Even less of a sure thing is if this is true in other states, grades, and subjects. Conclusion #1: saying that this study found that TFA teachers are better than others and leaving it at that is misleading at best. Conclusion #2: it almost always takes a number of similar studies to prove anything.

update: more write-ups here and here, and eduwonkette is still the only one to point out any limitations of the study