Reaction to the CMO Study

I don't disagree with the larger point behind Andrew Rotherham's argument regarding the recent study on Charter Management Organizations (CMOs) that we need to "live with some lemons" in order to see other schools succeed (and subsequently learn from those success stories).  I certainly wouldn't want my kid to attend one of the lemon schools (the educational equivalent of NIMBYism), but the larger argument that we sometimes have to take a step backward to take two steps forward is reasonable.

I, do, however disagree with two things he writes in the column:

1.) He writes that the study disproves charter critics' argument that "the good ones can’t be replicated to serve enough kids to really make a difference" by making it "clear that it is indeed possible to build a lot of schools that are game-changers for a lot of students".  He mentions that KIPP has 100 or so schools now and continues to perform well.  That's good.  That's a lot of schools.  What KIPP has done is really impressive.

But when charter critics say that high-flying charters can't be scaled up, they don't mean that we can't create 100 good charters.  They mean that we can't create 10,000 good schools run exactly the way KIPP and other schools are currently run.  In particular, KIPP and other top-performing schools tend to rely heavily on 20-something TFA members and TFA alums who are high-achieving, idealistic, often single, and willing to work long hours for relatively little pay for a few years.  There's nothing wrong with doing that.  If I opened a charter school tomorrow, I'd probably do the same thing.  But it is something that can't be replicated in every high-poverty school across the country.  The fact that KIPP has opened 100 schools absolutely does not change the fact that there aren't enough teachers who fit this mold to run all schools like this.

That's not to say that we can't copy many other innovations and features of the KIPPs of the world, just that building 100 schools run one way doesn't mean we can run every other school in the country (or even just all the high-poverty schools) the exact same way.  The "enough to make a difference" description of the argument is arguably a straw man since even one school "makes a difference" -- Rotherham should instead focus on implications for systemic reform rather than beating up straw men.

2.) I also find it really odd that the study itself and some of the articles on the study seem to emphasize that the differences in achievement gains between CMO-run charters and traditional public schools were not statistically significant, while Rotherham writes that "The study found that, in general, students at charter-network schools outperform similar students at traditional public schools, although sometimes not by very much" (and he's not the only one who said something like this).

I understand that the way these results are interpreted are different in academia and policy.  Of the eight point estimates they calculate, one is borderline significantly positive, three others seem like they could maybe be positive (point estimate larger than the standard error), three are trivial, through technically positive, and one is trivially negative.  So, I guess that's kinda (sorta) positive.  On the other hand, of the 22 CMOs they examine, 11 have positive coefficients in both math and reading, 2 are neutral in both, and 9 are negative in both.

In academia, that means that there were no significant differences -- I can't imagine trying to construe that as a positive world.  In the policy world, it could mean that "there are likely to be very small, positive differences," but to say they "outperform" other schools "in general" seems rather strong to me -- especially since he added "sometimes not by very much".  There were no areas in which the average CMO school outperformed the average non-charter by a lot, so that seems like an odd thing to say.


Of course, I hesitate to even point out these two things (they just struck me as odd while I was reading his column) because they threaten to distract from what I think should be the main discussion surrounding this study and other research on charters.  The argument about whether or not charters are better or not is, quite frankly, silly.  Some are.  Some aren't.  The end.  I want to know why some are better and what we can learn from those that are better.  And, similarly, why some are worse and what we can learn from them.  As long as we regard "charter" as a word that magically makes a school either evil or, well, magical, we won't really be learning all we can to help educate our students.