Earlier, I discussed G.R. Kearney's book about Cristo Rey Jesuit High School in Chicago. I sent along some questions about the book and Kearney was kind enough to answer them:
1.) How different is Cristo Rey now than when it opened in 1996? It's obviously bigger, in newer buildings, more selective, and you seem to say that it offers a more traditional curriculum. But if the founders walked into the school today, would they still recognize it?
Much has changed, no doubt. As you said, the buildings are new. The classrooms are nicer. There’s a beautiful gym. The student body has more than quadrupled. There’s even an endowment. The curriculum today is far less progressive and experimental. All that said, the founders would certainly recognize it. And I think most would see their fingerprints all over the place. The students are still earning their tuition and the school is still very much student centered. It’s hard to speak of Cristo Rey’s founders as a monolithic group. There were multiple founders, and not surprisingly, multiple viewpoints. There were Jesuits who’d spent decades teaching in very traditional but indisputably high quality high schools. There were progressive educators who insisted that traditional model wouldn’t work in Pilsen, the neighborhood where the original school opened. There were astute businesspeople who argued that efficiency and frugality needed to be top priorities, not experimentation. All the founders, though, were united in their desire to provide a better high school to a segment of the population that desperately needed one. That’s still the focus. And though the classes are taking place in more spacious rooms with carpeting and gadgets, the fundamental mission of the place hasn’t changed.
2.) Did you read David Whitman's description of Cristo Rey in his book Sweating the Small Stuff? If so, what did you think of it? It seems quite different from, if not diametrically opposed to, your description.
I have to admit that I have not read the book… I can’t afford it. A good friend has it and I’m next in line to go through it. I’ve read a good deal about it and have discussed it with many folks. I don’t, based on what I do know of Mr. Whitman’s writing, think his portrayal and mine are diametrically opposed. In what you’ve written about the book, you seem to focus on the following things as the primary differences between the Cristo Presented by Mr. Whitman and the Cristo Rey presented in MTAD: 1.) progressive “student centered” curriculum in place at the school vs. no-nonsense approach to education; 2.) lack of discipline and academic performance from some students in MTAD vs. Mr. Whitman’s presentation of a very disciplined school where punishment is meted out for dress code violations, gum chewing, and not handing in homework.
I’ll try to address both. But I must begin by saying that much of MTAD is set in the school’s early years. I don’t know when exactly Mr. Whitman’s study of the school took place, but Cristo Rey had changed a great deal in the intervening years, which I suspect is one of the main reasons that the presentations of the school feel so different. The development of the curriculum and the resulting ideological tug of war was very much a part of Cristo Rey’s early years. Starting in 2002-2003, the school shifted away from much of its experimentation (which had included custom designed classes, extensive cross curricular teaching, block scheduling and more) and had adopted a more traditional curriculum. I sincerely hope, though, that I did not give readers of MTAD the impression that the curricular innovations taking place at the school resulted in a lowering of disciplinary or academic expectations for students. While Cristo Rey’s teachers were developing innovative classes, they were also pushing their students through both classwork and homework to think at a high level and to deliver high quality work products. And there were certainly consequences for students who didn’t perform academically. From the earliest days, students who fell behind academically, oftentimes simply by failing to hand in assignment or two, were required to attend an after school silent study hall moderated by a Jesuit brother who also served as the school’s maintenance man and could be quite stern when he needed to be.
Regarding discipline, again, I don’t think the differences are as wide as they may initially seem; those that do exist probably again stem from the timing differences. For those who didn’t read MTAD, it’s worth pointing out that Cristo Rey struggled mightily to attract students (the founders mistakenly assumed they’d be overrun by students wanting to attend). The school opened in the fall of 1996. By the beginning of summer of 1996, only a handful of students had registered. The founding faculty literally attended street fairs in the neighborhood to try and convince young people to attend. As a result, CR ended up with a slightly different student mix than the founders had anticipated. Many of the students they were able to find had, for one reason or another, left their previous schools. Many of these kids lacked the intrinsic motivation Cristo Rey’s founders assumed their students would have. Discipline was also more of an issue in the early years when the school was still finding its way. Cristo Rey’s founding faculty started the first year without a dean of discipline. They believed, erroneously, that the students would be so smitten with the school that discipline wouldn’t be an issue. They were wrong. Within the year (I think) they’d appointed a dean and developed a demerit / detention system. At points, your post seems to suggest that innovation resulted in a lowering of expectations or standards. With the exception of the period during which there wasn’t a formal discipline system, I don’t think this could be farther from the truth. Expectations for students were always high and students were held to very high standards, albeit standards that would have been different than those in place at many other college prep schools.
Lastly, I think it’s important to point out that a central tenet of Mr. Whitman’s argument in favor of paternalism is that the new paternalism is also compassionate. Your post seems to highlight the disciplinary / no nonsense approach of paternalism and overlook the softer side. By its very nature, the new paternalism is compassionate and caring. I think this was in the early days, and remains today, a focus at Cristo Rey. I also don’t believe it to be in any way inconsistent with Mr. Whitman’s presentation of a school with high standards and a “no nonsense” approach.
coreynote: I didn't mean to imply that Cristo Rey was overrun with discipline problems, only that Whitman seemed to describe it as a place where discipline problems almost magically ceased to exist while Kearney provides a slightly different picture.
3.) You mention that one of the largest concerns in the early years were the difficulties that Cristo Rey students encountered once they enrolled in college. Is the school any closer to solving this problem?
This is a concern at Cristo Rey and at all of the other Cristo Rey model schools. Unfortunately, the problem is not limited to CR schools. The Cristo Rey Network is working hard to collect better data about student performance and is undertaking a broad longitudinal study. In addition, they’ve partnered with over twenty universities (including Georgetown, Harvard, Marquette, DePaul, Bentley, and many others) to develop a variety of different programs and initiatives aimed at better preparing Cristo Rey’s students for college and increasing their retention rate in college. More information about these post-secondary initiatives is available at the Cristo Rey Network website (www.cristoreynetwork.org).
4.) Almost half of the student who enroll in Cristo Rey fail to graduate from Cristo Rey. You and Whitman both cite this as a major focus for the Cristo Rey network. But is this necessarily a bad thing? While it means that Cristo Rey fails to reach every student in need, it also means that students at Cristo Rey have fewer negative peer influences.
Cristo Rey has a fairly rigorous application process, though there is no entrance exam. The school goes to great pains to ensure that the students selected to attend are capable of graduating and attending college. In theory, those students who would be true negative influences are screened out in the application process. As a result, Cristo Rey views the departure of any student who is accepted into the school as a failure. It also means that a seat in the school (as you mentioned in your post, the school has become selective and now turns away more students than it can admit) goes unused during some or all of a school year. At Cristo Rey, these departures are also problematic in that each one represents a decrease in revenue. Every student works to earn money. If 20% of the student body departs, revenue falls by 20%. All of the schools exist in a tight economic space. Maintaining full enrollment is key to maintaining their financial health.
5.) You mention that a number of teachers were troubled by the question that some students raised as to why they have to work to earn their schooling but students at schools like St. Ignatius to not. Do you think this is a sign that we continue to treat poor students differently in our educational system, or do the positives of the work program outweigh any negative implications it might have?
I don’t know that the teachers you’re referring to were as troubled by the question as they were by the reluctance of the administration to let them address it with their students. That makes it sound like I’m trying to sidestep the question. I’m not. My answers to the subsequent questions are yes and yes. Yes, I think students with limited economic means are treated differently, though oftentimes for good/sensible/apparently sensible reasons. I’m not sure it’s productive, though, to compare St. Ignatius to Cristo Rey. St. Ignatius is a private college prep school. The families of the students who attend pay more than $10,000 per year in tuition. That sum represents approximately 30% of the average family income at Cristo Rey. The simple fact is that the students who attend Cristo Rey cannot afford to attend St. Ignatius (they also can’t qualify academically… if they’re accepted to St. Ignatius, they cannot attend Cristo Rey). This is, simply put, economic reality. I think a better question is why some students in greater Chicago can attend fantastic public schools, but the young people in Pilsen have historically been limited to a school that, despite often heroic efforts, has struggled to perform at a consistently high level. To try and conclude this rambling answer, I don’t feel the fact that the students ask questions about why they “have to” work when other students don’t is necessarily a negative thing. I think, oftentimes, we attempt to shield students (in most any high school) from some of the economic realities they’ll encounter as they try to finance their college education or when they begin working (hopefully after graduating). I think the fact that Cristo Rey’s students ask these questions is profoundly positive. It suggests to me that they’re conscious of what they’re doing, that they’ve made a choice to work and go to school. As an aside, I think the work program is a tremendous resource to the students. As I mentioned above, it was conceived largely in response to the economic reality that the school simply would not be feasible as a tuition free institution giving the rising cost of labor. Sending the students to work could help bridge the gap. It has become, though, a phenomenal compliment to what goes on in the classroom.
Thanks again to G.R. Kearney for providing thorough responses. As a side note, for anybody else waiting for a turn to read Whitman's book, Fordham has made a PDF version available here.
Wednesday, September 30, 2009
Book Discussion: More than a Dream: How One School's Vision is Changing the World (The Cristo Rey Story)
After reading David Whitman's book, Sweating the Small Stuff: Inner-City Schools and the new Paternalism last August, I got a copy of a book on one of the schools profiled in the book: Cristo Rey Jesuit High School in Chicago. Unfortunately, I let G.R. Kearney's book, More than a Dream, sit on my bookshelf for over a year before I picked it up this week.
The book recounts the story of the founding of Cristo Rey, from idea through year seven. Though Kearney worked at the school for two years, the book is mostly dispassionate -- and certainly more dispassionate than expected given the title. Kearney did a good job of writing the school profile that he promised and avoiding the sappy memoir that it could have turned into.
Cristo Rey was initially conceived in 1992 by Father Brad Schaeffer, at that time the Provincial for the Chicago region, who respected the elite Jesuit schools in the city and around the country (he'd earlier been principal of one), but felt that Jesuits were called to help those with the most need. He eventually settled on opening a high school in the poor and heavily-Hispanic Pilsen/Little Village neighborhoods. For a while, various people puzzled over how to create a school that would be affordable to the neediest residents of the school. They eventually settled on a plan for students to work at firms around town one day a week (later adjusted to one day/week + an additional day/month), with the firms paying the school directly (well, actually a separately incorporated non-profit org connected to the school) in order to fund the students' education. After opening in 1996, the school got off to a bit of a rocky start, but eventually was able to attract millions of dollars in grants and now routinely sends 90+% of its students to college -- eventually leading people around the country to ask how they could replicate the school. The Cristo Rey Network now consists of 24 schools around the country.
The book alternates between the school's story and the story of a number of Cristo Rey students. Though too many details, and far too many names, were included, the story generally flows well. The readers are initially hit with a number of Jesuit abbreviations that would confuse many, but Kearney does take a little time to explain Jesuit history and get readers up to speed. Personally, I would have appreciated a glossary of Jesuit terms and abbreviations in the front or back of the book.
Nonetheless, the book definitely provides useful knowledge for anybody interested in urban education. It's no substitute for seeing schools in person, but it's the next best thing.
While the most distinctive feature of Cristo Rey is its jobs programs, the book spends more time discussing the day-to-day workings of the school. And the Cristo Rey that comes into focus as the book progresses is one that engages in fairly liberal pedagogy . . . teachers believe in creating courses that will engage students rather than simply teaching the basics. The principals of the school describe the school as "student-centered" again and again. In the first few years, the students took a course their first year designed to "equip them with the ability to constantly learn what they felt they needed to know" rather than "teaching knowledge for the sake of knowledge" (pp. 200-201). Students took part in an "Active Learners" capstone project, explained thusly by one teacher: "Traditionally, we think of students as absorbing the knowledge of the teachers. We wanted to teach the students to actually create knowledge, to become producers of knowledge." (p. 209)
Teachers spent far more time planning curricula than at most schools because they seemed to be obsessed with finding a better way to do things. Kearney writes that many teachers routinely stayed until at least 8pm and arrived around the crack of dawn (which might have been one of the reasons that so few of the founding teachers were still around by the end of the book). The book clearly depicts a school in which teachers see their job as more of a calling than anything else; indeed, the school hires more and more volunteer teachers as the school grows.
What really jumped out at me, though, was that the portrait Kearney painted of Cristo Rey was almost entirely different than the one painted by Whitman. In Whitman's book, he extols the virtues of Cristo Rey's paternalistic, no-nonsense approach to education. In Kearney's book, he depicts a notably progressive faculty experimenting with new ways of helping students -- though the school does seem to become more traditional as the years pass by. Whitman writes that Cristo Rey's jobs training program "provides a dose of cultural imperialism" (p. 131), while Kearney seems to emphasize that the jobs training program evolved to be less didactic and more interactive over time. Kearney describes the angst over expelling students, particularly ones that had been fired from their jobs. Eventually the faculty decides to create a second chance program for these students. Whitman quotes the principal as saying "If a student loses a job a second time, we would ask them to leave. You can't be dubbed unemployable and be a student here" (p. 133). Kearney describes one student (who graduated) who "quickly became a discipline problem," who, "by the end of his first semester, was handing in less than 25% of his homework assignments" (pp. 245-246). Whitman writes of a school where no nonsense is tolerated and a student complains of getting detention for chewing gum, wearing tight pants, missing two homework assignments, coming to school or class late, or talking to friends (p. 133).
I'm not sure if these versions of Cristo Rey are incompatible, or if the authors just see the school in a different light. But it's very clear that Whitman views the school's no-nonsense, paternalistic approach to education as its defining characteristic while Kearney mentions very little that would seem to indicate this is one of the school's notable features. Given that Whitman observed the school a few years after Kearney did it may also be the case that the school evolved from a progressive approach to a more zero tolerance approach over the years.
While it seems clear from both accounts that Cristo Rey is doing a remarkable and laudable job, and I very much appreciate both their innovative approach to education and their dedication to an important goal, the policy person in me can't help but worry about four possible shortcomings:
1.) The student body changed dramatically over the course of the first five years or so. In the beginning, the school would accept nearly everybody that applied. Not only was the school in one of the worst neighborhoods in Chicago, it seemed to be enrolling some of the worst students in that neighborhood. But as the years went by, the school gained prestige and acclaim . . . and admissions became more and more selective. The school still focused exclusively on children from low-income families, but the student body more and more was composed of the cream of the crop from the surrounding areas. I certainly can't blame the school for wanting to enroll better students (not to mention expel troublemakers), but I share the same concern expressed by many of the teachers in the book: what happens to the types of students who initially enrolled -- the students who haven't had much success, but are willing to try? Who's going to educate them?
2.) As both authors acknowledge, the attrition rate is very high in the school. Kearney pegs the graduation rate at all Cristo Rey schools as just under 60% (p. 374) and both authors note that raising this rate is a prime goal for the schools' leaders. While each of the schools may do a phenomenal job with the students that remain in them, I can't help but wonder what happens to the other students. And I again pose the question: who's going to educate them?
3.) While the phenomenally fast replication of Cristo Rey is quite remarkable, the model can only be replicated so many times. It's good to have schools providing the poorest kids with a fantastic education, but there simply aren't enough firms with job openings to replicate this model more than maybe a few times in one city -- clearly all 100+ high schools in NYC can't operate this way. So even if this model proves to be the savior for a number of kids, which is both impressive and important, we still have a long way to go to solve urban education in this country.
4.) Last but not least, I share the same concern raised by some of the students and faculty at Cristo Rey: why do the poorest kids have to spend a day at work each week instead of at school in order to get a first-rate education? The students at the other Jesuit schools attend five days each week. I'm torn on this. On the one hand, it seems exceedingly likely that the experience at work is a formative and valuable one for the students. On the other hand, it troubles me that we seem to have no qualms creating different kinds of schools for our poorest children.
Despite those qualms, I have little doubt that the people running Cristo Rey are remarkable and deserving of every bit of attention they have received. Kearney's book is an excellent, and easily readable, description of how the school came about and how it overcame numerous obstacles. It doesn't have the heft of a research volume, but it's equally illuminating.
The book recounts the story of the founding of Cristo Rey, from idea through year seven. Though Kearney worked at the school for two years, the book is mostly dispassionate -- and certainly more dispassionate than expected given the title. Kearney did a good job of writing the school profile that he promised and avoiding the sappy memoir that it could have turned into.
Cristo Rey was initially conceived in 1992 by Father Brad Schaeffer, at that time the Provincial for the Chicago region, who respected the elite Jesuit schools in the city and around the country (he'd earlier been principal of one), but felt that Jesuits were called to help those with the most need. He eventually settled on opening a high school in the poor and heavily-Hispanic Pilsen/Little Village neighborhoods. For a while, various people puzzled over how to create a school that would be affordable to the neediest residents of the school. They eventually settled on a plan for students to work at firms around town one day a week (later adjusted to one day/week + an additional day/month), with the firms paying the school directly (well, actually a separately incorporated non-profit org connected to the school) in order to fund the students' education. After opening in 1996, the school got off to a bit of a rocky start, but eventually was able to attract millions of dollars in grants and now routinely sends 90+% of its students to college -- eventually leading people around the country to ask how they could replicate the school. The Cristo Rey Network now consists of 24 schools around the country.
The book alternates between the school's story and the story of a number of Cristo Rey students. Though too many details, and far too many names, were included, the story generally flows well. The readers are initially hit with a number of Jesuit abbreviations that would confuse many, but Kearney does take a little time to explain Jesuit history and get readers up to speed. Personally, I would have appreciated a glossary of Jesuit terms and abbreviations in the front or back of the book.
Nonetheless, the book definitely provides useful knowledge for anybody interested in urban education. It's no substitute for seeing schools in person, but it's the next best thing.
While the most distinctive feature of Cristo Rey is its jobs programs, the book spends more time discussing the day-to-day workings of the school. And the Cristo Rey that comes into focus as the book progresses is one that engages in fairly liberal pedagogy . . . teachers believe in creating courses that will engage students rather than simply teaching the basics. The principals of the school describe the school as "student-centered" again and again. In the first few years, the students took a course their first year designed to "equip them with the ability to constantly learn what they felt they needed to know" rather than "teaching knowledge for the sake of knowledge" (pp. 200-201). Students took part in an "Active Learners" capstone project, explained thusly by one teacher: "Traditionally, we think of students as absorbing the knowledge of the teachers. We wanted to teach the students to actually create knowledge, to become producers of knowledge." (p. 209)
Teachers spent far more time planning curricula than at most schools because they seemed to be obsessed with finding a better way to do things. Kearney writes that many teachers routinely stayed until at least 8pm and arrived around the crack of dawn (which might have been one of the reasons that so few of the founding teachers were still around by the end of the book). The book clearly depicts a school in which teachers see their job as more of a calling than anything else; indeed, the school hires more and more volunteer teachers as the school grows.
What really jumped out at me, though, was that the portrait Kearney painted of Cristo Rey was almost entirely different than the one painted by Whitman. In Whitman's book, he extols the virtues of Cristo Rey's paternalistic, no-nonsense approach to education. In Kearney's book, he depicts a notably progressive faculty experimenting with new ways of helping students -- though the school does seem to become more traditional as the years pass by. Whitman writes that Cristo Rey's jobs training program "provides a dose of cultural imperialism" (p. 131), while Kearney seems to emphasize that the jobs training program evolved to be less didactic and more interactive over time. Kearney describes the angst over expelling students, particularly ones that had been fired from their jobs. Eventually the faculty decides to create a second chance program for these students. Whitman quotes the principal as saying "If a student loses a job a second time, we would ask them to leave. You can't be dubbed unemployable and be a student here" (p. 133). Kearney describes one student (who graduated) who "quickly became a discipline problem," who, "by the end of his first semester, was handing in less than 25% of his homework assignments" (pp. 245-246). Whitman writes of a school where no nonsense is tolerated and a student complains of getting detention for chewing gum, wearing tight pants, missing two homework assignments, coming to school or class late, or talking to friends (p. 133).
I'm not sure if these versions of Cristo Rey are incompatible, or if the authors just see the school in a different light. But it's very clear that Whitman views the school's no-nonsense, paternalistic approach to education as its defining characteristic while Kearney mentions very little that would seem to indicate this is one of the school's notable features. Given that Whitman observed the school a few years after Kearney did it may also be the case that the school evolved from a progressive approach to a more zero tolerance approach over the years.
While it seems clear from both accounts that Cristo Rey is doing a remarkable and laudable job, and I very much appreciate both their innovative approach to education and their dedication to an important goal, the policy person in me can't help but worry about four possible shortcomings:
1.) The student body changed dramatically over the course of the first five years or so. In the beginning, the school would accept nearly everybody that applied. Not only was the school in one of the worst neighborhoods in Chicago, it seemed to be enrolling some of the worst students in that neighborhood. But as the years went by, the school gained prestige and acclaim . . . and admissions became more and more selective. The school still focused exclusively on children from low-income families, but the student body more and more was composed of the cream of the crop from the surrounding areas. I certainly can't blame the school for wanting to enroll better students (not to mention expel troublemakers), but I share the same concern expressed by many of the teachers in the book: what happens to the types of students who initially enrolled -- the students who haven't had much success, but are willing to try? Who's going to educate them?
2.) As both authors acknowledge, the attrition rate is very high in the school. Kearney pegs the graduation rate at all Cristo Rey schools as just under 60% (p. 374) and both authors note that raising this rate is a prime goal for the schools' leaders. While each of the schools may do a phenomenal job with the students that remain in them, I can't help but wonder what happens to the other students. And I again pose the question: who's going to educate them?
3.) While the phenomenally fast replication of Cristo Rey is quite remarkable, the model can only be replicated so many times. It's good to have schools providing the poorest kids with a fantastic education, but there simply aren't enough firms with job openings to replicate this model more than maybe a few times in one city -- clearly all 100+ high schools in NYC can't operate this way. So even if this model proves to be the savior for a number of kids, which is both impressive and important, we still have a long way to go to solve urban education in this country.
4.) Last but not least, I share the same concern raised by some of the students and faculty at Cristo Rey: why do the poorest kids have to spend a day at work each week instead of at school in order to get a first-rate education? The students at the other Jesuit schools attend five days each week. I'm torn on this. On the one hand, it seems exceedingly likely that the experience at work is a formative and valuable one for the students. On the other hand, it troubles me that we seem to have no qualms creating different kinds of schools for our poorest children.
Despite those qualms, I have little doubt that the people running Cristo Rey are remarkable and deserving of every bit of attention they have received. Kearney's book is an excellent, and easily readable, description of how the school came about and how it overcame numerous obstacles. It doesn't have the heft of a research volume, but it's equally illuminating.
Tuesday, September 29, 2009
Apples to Apples? Not Necessarily
Mike Petrilli has a good post over at Flypaper on the Hoxby et. al. charter school study that was recently released. It's good to see I'm not the only one pointing out that the Wall St. Journal and others shouldn't be writing NYC charters don't cream -- because they probably do.
But I have to take issue with one thing that Petrilli writes about the study: "because it’s a gold-standard random-assignment study, we can be sure that it’s an apples to apples comparison". Not so fast Mike . . . the random assignment part might well be true, but there are plenty of reasons to think that this isn't quite an apples-to-apples comparison.
First, in regards to the random assignment* designation: it may not actually be random assignment. I have yet to hear back regarding the number of charter schools each kid applied to, but it's certainly possible that those who won a charter school lottery applied to more charters, on average, than those who lost a charter school lottery. In other words, it's possible that the winners were more motivated than the losers to begin with. On the other hand, it's also possible that losers were more likely to apply to charters with long waiting lists -- and it might be true that those with longer waiting lists are superior schools that attract superior students.
In addition to these possibilities, I can think of at least one other way that the study design could result in a comparison that is not quite apples-to-apples (note that the authors did address some of these in the paper):
Attrition may not be the same for charter enrollees as it is for the others. 22% of the lottery-losing students transferred to a school outside of NYC or a private school, while 14% of the charter students transferred to a traditional NYC public school. While it's likely that both groups leave their current school due to some level of dissatisfaction, it may be the case that those who transfer from charter schools to another NYC school do so because they're struggling at their school and/or are encouraged to leave due to discipline or other problems. Meanwhile, it may be the case that those who transfer from traditional public schools to non-NYC schools do so because they view their schools as failing them and think they can do better elsewhere. In other words, it's plausible that the most motivated non-charter students transfer out while the least-motivated charter students transfer out. The authors do take a look at this and conclude that there's no difference in the relationship between test scores and likelihood of leaving between the two groups, but there could be no difference in test scores but a large difference in motivation, attitude, family support, or a myriad of other factors that are more likely to lead to growth in test scores (which, remember, is the outcome variable in the study).
*For those of you without a research background, a random assignment study is just what it sounds like -- a study in which those being studied were randomly assigned to a control group or treatment group (for example, if we flipped a quarter for every kid who walked in the door and those who got tails were assigned to Mrs. Johnson's class and those who got heads were assigned to Mr. Smith's class).
But I have to take issue with one thing that Petrilli writes about the study: "because it’s a gold-standard random-assignment study, we can be sure that it’s an apples to apples comparison". Not so fast Mike . . . the random assignment part might well be true, but there are plenty of reasons to think that this isn't quite an apples-to-apples comparison.
First, in regards to the random assignment* designation: it may not actually be random assignment. I have yet to hear back regarding the number of charter schools each kid applied to, but it's certainly possible that those who won a charter school lottery applied to more charters, on average, than those who lost a charter school lottery. In other words, it's possible that the winners were more motivated than the losers to begin with. On the other hand, it's also possible that losers were more likely to apply to charters with long waiting lists -- and it might be true that those with longer waiting lists are superior schools that attract superior students.
In addition to these possibilities, I can think of at least one other way that the study design could result in a comparison that is not quite apples-to-apples (note that the authors did address some of these in the paper):
Attrition may not be the same for charter enrollees as it is for the others. 22% of the lottery-losing students transferred to a school outside of NYC or a private school, while 14% of the charter students transferred to a traditional NYC public school. While it's likely that both groups leave their current school due to some level of dissatisfaction, it may be the case that those who transfer from charter schools to another NYC school do so because they're struggling at their school and/or are encouraged to leave due to discipline or other problems. Meanwhile, it may be the case that those who transfer from traditional public schools to non-NYC schools do so because they view their schools as failing them and think they can do better elsewhere. In other words, it's plausible that the most motivated non-charter students transfer out while the least-motivated charter students transfer out. The authors do take a look at this and conclude that there's no difference in the relationship between test scores and likelihood of leaving between the two groups, but there could be no difference in test scores but a large difference in motivation, attitude, family support, or a myriad of other factors that are more likely to lead to growth in test scores (which, remember, is the outcome variable in the study).
*For those of you without a research background, a random assignment study is just what it sounds like -- a study in which those being studied were randomly assigned to a control group or treatment group (for example, if we flipped a quarter for every kid who walked in the door and those who got tails were assigned to Mrs. Johnson's class and those who got heads were assigned to Mr. Smith's class).
Monday, September 28, 2009
(Why) Are NYC Charters Different?
Now that the Washington Post has also taken the findings of the latest charter school report as gospel, I think it's important that we examine some of the issues it raises more in-depth.
As Diane Ravitch points out, there a number of other studies finding blase or discouraging results for charters both nationally and in other cities. So if we assume that the report's findings are correct (while I think the report skims across a lot of issues, it does seem fairly likely that charter schools are outperforming traditional public schools in NYC), the first question we should ask is why charters in NYC are different. I can think of at least four reasons why NYC charters may be outperforming charters elsewhere when compared to other local schools:
1.) NYC schools are uniquely bad. If the public schools in NYC are incredibly atrocious and dysfunctional, then it shouldn't be that hard to create a school that outperforms most local schools. From experience, I can tell you that there are certainly a number of schools in NYC that are incredibly atrocious and poorly run. While teaching there I regularly heard horror stories from other teachers (not to mention the things I experienced in my school) that should make anybody cringe. I really can't say, though, that these horror stories make NYC schools any worse than those in Philly, Chicago, L.A., etc. -- I'm sure you can find more than any school's fair share of horror stories coming from other big cities as well.
2.) NYC charters more easily attract talent and funding. NYC is, to many people, the most important city in the world. There's certainly more talent and money floating around the city than there is in most other American cities. I have no evidence either way on this, but it's certainly plausible that NYC charter schools both find it easier to attract both talent (both in terms of management and employees) and outside funding -- both when compared to charters in other cities and when compared to other NYC schools. Especially since there are only around 80 charter schools in the city compared to around 1,000 or so traditional public schools.
3.) The NY Charter Review Board outperforms others. It's certainly possible that the board that approves the creation of charter schools in NYC does a better job of screening and monitoring than do other boards. One of our teachers during our pre-teaching summer training session was in the process of creating a charter school in Harlem. From what he said, it sounded like an awfully involved process involving a heck of a lot of thought and work. And maybe this has resulted in better planned and more successful charter schools.
4.) The other studies are all flawed. In other words, maybe Hoxby et. al. are right and everybody else is wrong. I'm sure all the other studies are flawed, but so is this one -- so I don't really find this idea convincing. It's not out of the realm of possibility that charters are generally outperforming other schools and we just haven't proved it yet, but I find it more plausible that charters are performing differently in different places.
As Diane Ravitch points out, there a number of other studies finding blase or discouraging results for charters both nationally and in other cities. So if we assume that the report's findings are correct (while I think the report skims across a lot of issues, it does seem fairly likely that charter schools are outperforming traditional public schools in NYC), the first question we should ask is why charters in NYC are different. I can think of at least four reasons why NYC charters may be outperforming charters elsewhere when compared to other local schools:
1.) NYC schools are uniquely bad. If the public schools in NYC are incredibly atrocious and dysfunctional, then it shouldn't be that hard to create a school that outperforms most local schools. From experience, I can tell you that there are certainly a number of schools in NYC that are incredibly atrocious and poorly run. While teaching there I regularly heard horror stories from other teachers (not to mention the things I experienced in my school) that should make anybody cringe. I really can't say, though, that these horror stories make NYC schools any worse than those in Philly, Chicago, L.A., etc. -- I'm sure you can find more than any school's fair share of horror stories coming from other big cities as well.
2.) NYC charters more easily attract talent and funding. NYC is, to many people, the most important city in the world. There's certainly more talent and money floating around the city than there is in most other American cities. I have no evidence either way on this, but it's certainly plausible that NYC charter schools both find it easier to attract both talent (both in terms of management and employees) and outside funding -- both when compared to charters in other cities and when compared to other NYC schools. Especially since there are only around 80 charter schools in the city compared to around 1,000 or so traditional public schools.
3.) The NY Charter Review Board outperforms others. It's certainly possible that the board that approves the creation of charter schools in NYC does a better job of screening and monitoring than do other boards. One of our teachers during our pre-teaching summer training session was in the process of creating a charter school in Harlem. From what he said, it sounded like an awfully involved process involving a heck of a lot of thought and work. And maybe this has resulted in better planned and more successful charter schools.
4.) The other studies are all flawed. In other words, maybe Hoxby et. al. are right and everybody else is wrong. I'm sure all the other studies are flawed, but so is this one -- so I don't really find this idea convincing. It's not out of the realm of possibility that charters are generally outperforming other schools and we just haven't proved it yet, but I find it more plausible that charters are performing differently in different places.
Sunday Commentary: Do NYC Charter Schools "Cream?"
Probably. At least that seems to be the most sensible interpretation of the recently released report on NYC charter schools. And it's important to note that the Wall St. Journal gets this dead wrong in their editorial on the report. The main conclusion of the report was that students who enrolled in NYC charter schools performed much better than did students who tried, but failed, to enroll in these charter schools. So I found it more than a bit odd that the Wall St. Journal's headline trumpeted something else. Why lie when there's plenty of truth to extol?
Nowhere in the report is the claim made that charter schools do not "cream" (select only the top students, like one would cream the fat off of milk, for those unfamiliar with the lingo). The claim that charter schools don't cream seems to be based on the fact that charter applicants are more likely to be black or poor than the average NYC student. This is an important, and entirely unsurprising, piece of information.
But here's what they missed: the authors go out of their way to stress that they cannot accurately compare the prior test scores of charter school applicants to non-applicants, writing that "it is not possible to draw conclusions about how charter school applicants' achievement compares to that of students in New York City's traditional public schools" (p. 28).
Nonetheless, they display a chart on the same page with calculations that the average test score of charter school applicants is no different than the average score of non-applicants. This despite the fact that applicants were much more likely to be poor or Black and much less likely to be White or Asian. And any cursory glance at test scores in NYC will tell you that the average poor Black student scores far below the average non-poor White or Asian kid. Which means that we should see average scores considerably below the citywide average from charter school applicants. The fact that there is no difference would indicate, to me at least, that charter school applicants are out-performing other demographically similar students.
Similarly, the authors write on page 71 that students who are accepted into, but do not enroll in, a charter school have prior test scores no different from those who do enroll. But, at the same time, admitted students who choose not to enroll are much more likely to be White or Asian. Which raises two questions: 1.) If the authors had previously argued that there wasn't test data on enough students to make any claims about test scores, why are they now making claims about test scores? and 2.) If the non-enrolling admitted students are more likely to be White or Asian, would we not expect them to have higher previous test scores than do enrolling admitted students?
Besides, comparing test scores is, to some extent, beside the point. It might very well matter more how motivated a student and their family is, what kind of support they are receiving at home, etc. If you were running a school and your goal was for students to make as much progress as possible, which would be more important qualification for students: previous success on tests or the willingness to work hard?
In the end, we need to see both disaggregated data (e.g. comparing Black enrollees to Black non-enrollees) and comparisons of test scores and other variables controlling for various demographic variables before we can make any firm claim about NYC charter schools creaming students. But the authors' position is firm: they can't tell. And if you ask me, all signs indicate that they likely do. Either way, the Wall St. Journal still needs run a correction.
Last, but certainly not least, should we care if charter schools cream? If charter schools enroll more motivated or more capable students, might that not be a good thing (especially since they're clearly attracting students from the most at-risk demographics)? The answer, of course, is that it depends. It's hard to imagine that enrolling more motivated and higher achieving kids in a school wouldn't be a good thing for those who attend that school. At the same time, there's a possibility that it might negatively influence kids in the surrounding schools that are losing the cream of the crop. But it's hard to say to what extent either of these scenarios are playing out. At this point, whether or not creaming makes charter schools bad is largely in the eyes of the beholder.
But, more importantly, it affects the way that charter schools should be analyzed -- because, ultimately, charter schools are supposed to serve as incubators of ideas that spread to other schools. Which means that we should make an effort to figure out which of these ideas are worth replicating. And if charter schools cream a little or a lot and we don't take this into account, it skews both the analysis of their performance and the analysis of which ideas should be replicated. Which has the possibility of harming the students we ultimately want to help.
Nowhere in the report is the claim made that charter schools do not "cream" (select only the top students, like one would cream the fat off of milk, for those unfamiliar with the lingo). The claim that charter schools don't cream seems to be based on the fact that charter applicants are more likely to be black or poor than the average NYC student. This is an important, and entirely unsurprising, piece of information.
But here's what they missed: the authors go out of their way to stress that they cannot accurately compare the prior test scores of charter school applicants to non-applicants, writing that "it is not possible to draw conclusions about how charter school applicants' achievement compares to that of students in New York City's traditional public schools" (p. 28).
Nonetheless, they display a chart on the same page with calculations that the average test score of charter school applicants is no different than the average score of non-applicants. This despite the fact that applicants were much more likely to be poor or Black and much less likely to be White or Asian. And any cursory glance at test scores in NYC will tell you that the average poor Black student scores far below the average non-poor White or Asian kid. Which means that we should see average scores considerably below the citywide average from charter school applicants. The fact that there is no difference would indicate, to me at least, that charter school applicants are out-performing other demographically similar students.
Similarly, the authors write on page 71 that students who are accepted into, but do not enroll in, a charter school have prior test scores no different from those who do enroll. But, at the same time, admitted students who choose not to enroll are much more likely to be White or Asian. Which raises two questions: 1.) If the authors had previously argued that there wasn't test data on enough students to make any claims about test scores, why are they now making claims about test scores? and 2.) If the non-enrolling admitted students are more likely to be White or Asian, would we not expect them to have higher previous test scores than do enrolling admitted students?
Besides, comparing test scores is, to some extent, beside the point. It might very well matter more how motivated a student and their family is, what kind of support they are receiving at home, etc. If you were running a school and your goal was for students to make as much progress as possible, which would be more important qualification for students: previous success on tests or the willingness to work hard?
In the end, we need to see both disaggregated data (e.g. comparing Black enrollees to Black non-enrollees) and comparisons of test scores and other variables controlling for various demographic variables before we can make any firm claim about NYC charter schools creaming students. But the authors' position is firm: they can't tell. And if you ask me, all signs indicate that they likely do. Either way, the Wall St. Journal still needs run a correction.
Last, but certainly not least, should we care if charter schools cream? If charter schools enroll more motivated or more capable students, might that not be a good thing (especially since they're clearly attracting students from the most at-risk demographics)? The answer, of course, is that it depends. It's hard to imagine that enrolling more motivated and higher achieving kids in a school wouldn't be a good thing for those who attend that school. At the same time, there's a possibility that it might negatively influence kids in the surrounding schools that are losing the cream of the crop. But it's hard to say to what extent either of these scenarios are playing out. At this point, whether or not creaming makes charter schools bad is largely in the eyes of the beholder.
But, more importantly, it affects the way that charter schools should be analyzed -- because, ultimately, charter schools are supposed to serve as incubators of ideas that spread to other schools. Which means that we should make an effort to figure out which of these ideas are worth replicating. And if charter schools cream a little or a lot and we don't take this into account, it skews both the analysis of their performance and the analysis of which ideas should be replicated. Which has the possibility of harming the students we ultimately want to help.
Wednesday, September 23, 2009
A Few Thoughts on the NYC Charter Schools Evaluation Project
The NY Times profiled a report on charter schools that was released today. The group conducting the investigation was led by Caroline Hoxby, an economist based at Stanford's Hoover Institute who has published a number of articles in prestigious journals that find positive returns to school choice. The report is written in plain English and is very easily readable even for those who have little background knowledge or statistical training (which was quite a surprise to me given the highly technical nature of most of her work).
The bottom line of the report is that students who enroll in charter schools do much better than those who enter lotteries to enroll in charter schools, but don't have their number picked out of the hat. The methodology of comparing "lottery winners" to "lottery losers" is probably the best way to go when evaluating charter schools because you should end up with comparison and treatment groups that are demographically similar. Though, of course, that doesn't mean it's perfect. A few thoughts:
-It's unclear whether students who are lottery winners apply to more charter schools than those who are lottery losers (one could imagine that a more motivated parent would apply to 10 charters and be more likely to end up with their child in one of them than the parent who applies to 1).
-It's also somewhat unclear how they treat a student who leaves or enrolls in a charter school in a subsequent year. They mention that about 14% of students leave charter schools (p. 71), but I didn't notice a part where they addressed the latter. (correction: the 14% figure is the percentage of treatment group students that have transferred to traditional public schools over the life of the study, at least 17% have transferred to other schools and 24% have dropped out of the study for various reasons (p. 72))
-The study points out that lottery losers are 32% more likely to transfer to a private school or a school outside of NYC than are lottery winners (p. 72), but I'm not sure they fully investigate the effects of this. It sounds as though lottery-losing leavers have test scores that are no different than lottery-winning leavers, but this isn't really the full story. Current test score is only one predictor of future test score, and since they're measuring growth in test scores, I'd have to imagine that motivation is a better predictor of growth in test scores. And as far as I can tell, it's entirely feasible that more motivated students are more likely to leave the system if they're not enrolled in a charter school.
-They estimate the average growth in test scores for charter school students to be .12 (math) and .09 (reading) standard deviations above and beyond those of lottery losers each year (p. 42). For one year, that's not particularly impressive, but they calculate that these effects add up over time to create quite a powerful effect.
-Perhaps more interesting are the effects they calculate for individual schools. Charter schools are supposed to vary widely, so it shouldn't really be possible to generalize to all charter schools. Here are how the effects of charter schools broke down in each subject (p. 57-59)
Assuming that these measurements are accurate and that higher test scores reflect more meaningful growth as well (neither of which should be taken for granted), I'd argue that schools over .2 might be worth getting excited about and schools between .1 - .2 are probably doing something right. Making those two dangerous assumptions, we might say that a majority of schools seem to be doing something right. It also becomes evident why, as I've argued in the past, charter school advocates should advocate closing more charters. Think how much higher the estimates would be if the bottom 20% or so that are getting returns that are negative or no different from zero were closed.
-Why are charter schools doing better (same two assumptions apply)? The authors measure the effects of a wide variety of variables on page 64. The most notable number is that adding 10 days to the school year boosts achievement .15 SD. This is the only number that is both meaningful to me and statistically significant at traditional levels. The average charter school has about 10 more days of school than the average traditional public school. Meanwhile, the average charter school advantage is about .10-.11 SD. Which raises the possibility that charters are doing better mostly because they have longer school years -- a reform which could very easily be replicated in all schools.
-You should ignore all the calculations regarding the benefits of attending a charter high school. In the last year of data, there are four charter high schools in NYC.
I'll e-mail the team about the first two bullet points, hopefully I'll hear back -- I'll post the answers if and when I do.
The bottom line of the report is that students who enroll in charter schools do much better than those who enter lotteries to enroll in charter schools, but don't have their number picked out of the hat. The methodology of comparing "lottery winners" to "lottery losers" is probably the best way to go when evaluating charter schools because you should end up with comparison and treatment groups that are demographically similar. Though, of course, that doesn't mean it's perfect. A few thoughts:
-It's unclear whether students who are lottery winners apply to more charter schools than those who are lottery losers (one could imagine that a more motivated parent would apply to 10 charters and be more likely to end up with their child in one of them than the parent who applies to 1).
-It's also somewhat unclear how they treat a student who leaves or enrolls in a charter school in a subsequent year. They mention that about 14% of students leave charter schools (p. 71), but I didn't notice a part where they addressed the latter. (correction: the 14% figure is the percentage of treatment group students that have transferred to traditional public schools over the life of the study, at least 17% have transferred to other schools and 24% have dropped out of the study for various reasons (p. 72))
-The study points out that lottery losers are 32% more likely to transfer to a private school or a school outside of NYC than are lottery winners (p. 72), but I'm not sure they fully investigate the effects of this. It sounds as though lottery-losing leavers have test scores that are no different than lottery-winning leavers, but this isn't really the full story. Current test score is only one predictor of future test score, and since they're measuring growth in test scores, I'd have to imagine that motivation is a better predictor of growth in test scores. And as far as I can tell, it's entirely feasible that more motivated students are more likely to leave the system if they're not enrolled in a charter school.
-They estimate the average growth in test scores for charter school students to be .12 (math) and .09 (reading) standard deviations above and beyond those of lottery losers each year (p. 42). For one year, that's not particularly impressive, but they calculate that these effects add up over time to create quite a powerful effect.
-Perhaps more interesting are the effects they calculate for individual schools. Charter schools are supposed to vary widely, so it shouldn't really be possible to generalize to all charter schools. Here are how the effects of charter schools broke down in each subject (p. 57-59)
| reading | math | |
| negative | 8% | 14% |
| 0 - .1 SD | 16% | 17% |
| .1 - .2 SD | 45% | 59% |
| > .2 SD | 31% | 10% |
Assuming that these measurements are accurate and that higher test scores reflect more meaningful growth as well (neither of which should be taken for granted), I'd argue that schools over .2 might be worth getting excited about and schools between .1 - .2 are probably doing something right. Making those two dangerous assumptions, we might say that a majority of schools seem to be doing something right. It also becomes evident why, as I've argued in the past, charter school advocates should advocate closing more charters. Think how much higher the estimates would be if the bottom 20% or so that are getting returns that are negative or no different from zero were closed.
-Why are charter schools doing better (same two assumptions apply)? The authors measure the effects of a wide variety of variables on page 64. The most notable number is that adding 10 days to the school year boosts achievement .15 SD. This is the only number that is both meaningful to me and statistically significant at traditional levels. The average charter school has about 10 more days of school than the average traditional public school. Meanwhile, the average charter school advantage is about .10-.11 SD. Which raises the possibility that charters are doing better mostly because they have longer school years -- a reform which could very easily be replicated in all schools.
-You should ignore all the calculations regarding the benefits of attending a charter high school. In the last year of data, there are four charter high schools in NYC.
I'll e-mail the team about the first two bullet points, hopefully I'll hear back -- I'll post the answers if and when I do.
Wednesday, September 9, 2009
Today's Random Thoughts
-When looking at the browser requirements for filling out the FAFSA, I noticed that anybody who regularly updates their computer is unable to fill one out online. The FAFSA website can only handle up to Internet Explorer 7 and Firefox 2.0 -- they're up to versions 8 and 3.5 now, respectively. This is something the Department of Education should've fixed months ago. While I'm on the subject of the federal govt. updating their websites, why can't they combine the login systems for e-filing tax returns and applying for financial aid? Why must one type in all their same tax info all over again?
-I continue to see little discussion of the fact that about 2/3 of teachers don't teach tested subjects when discussing merit pay. Indeed, NYC recently passed out 12,000 teacher evaluations based on state test scores . . . there are 87,000 teachers in NYC. Assuming these evaluations were accurate, how would we evaluate the other 75,000 teachers?
-Speaking of merit pay and evaluating teachers using standardized test scores . . . many object to the latter on the grounds that we might fire a good teacher based on faulty measures. What if, instead, we reward mediocre teachers for high performance based on test scores -- breaking the bank and not helping our schools at the same time? And, it's clear that faulty measures are being used (check out the last graph in this piece if you believe at all that NYC's school grading system is statistically valid), leading to 97% of NYC schools receiving a grade of A or B. New York's faulty tests also resulted in a doubling of the budget for teacher bonuses this year. Merit pay is supposed to be a fiscally efficient reform, but it's not under these types of conditions.
-The AP has a story on private investment in charter schools. I wonder how this will play out if charter schools continue to expand. If private monies dry up, we might not hear much more about it. But if private monies start going disproportionately to charter schools, it might be hard for public schools to compete. On the other hand, it might make charter schools even better, which would probably be a good thing for those attending charters.
-I wrote before that we seem to be seeing a charterization of urban public districts. Not only are charter schools spreading, but charter-like public schools (read: small, specialized schools often not for just one neighborhood) are as well. The NY Post has a little blurb on the vast array of specialties that this year's crop of new schools offer. If this continues I wonder what the ramifications of having community schools in the suburbs and beyond and specialty schools in the inner-city will be.
-I continue to see little discussion of the fact that about 2/3 of teachers don't teach tested subjects when discussing merit pay. Indeed, NYC recently passed out 12,000 teacher evaluations based on state test scores . . . there are 87,000 teachers in NYC. Assuming these evaluations were accurate, how would we evaluate the other 75,000 teachers?
-Speaking of merit pay and evaluating teachers using standardized test scores . . . many object to the latter on the grounds that we might fire a good teacher based on faulty measures. What if, instead, we reward mediocre teachers for high performance based on test scores -- breaking the bank and not helping our schools at the same time? And, it's clear that faulty measures are being used (check out the last graph in this piece if you believe at all that NYC's school grading system is statistically valid), leading to 97% of NYC schools receiving a grade of A or B. New York's faulty tests also resulted in a doubling of the budget for teacher bonuses this year. Merit pay is supposed to be a fiscally efficient reform, but it's not under these types of conditions.
-The AP has a story on private investment in charter schools. I wonder how this will play out if charter schools continue to expand. If private monies dry up, we might not hear much more about it. But if private monies start going disproportionately to charter schools, it might be hard for public schools to compete. On the other hand, it might make charter schools even better, which would probably be a good thing for those attending charters.
-I wrote before that we seem to be seeing a charterization of urban public districts. Not only are charter schools spreading, but charter-like public schools (read: small, specialized schools often not for just one neighborhood) are as well. The NY Post has a little blurb on the vast array of specialties that this year's crop of new schools offer. If this continues I wonder what the ramifications of having community schools in the suburbs and beyond and specialty schools in the inner-city will be.
Friday, September 4, 2009
Steven Brill Wrote a Really Bad Article
I tried to think of a cutesy title for a while, perhaps involving a pun, but I decided to just say what I thought.
On Sunday, I wrote that it's probably not as hard to get rid of "bad teachers" as many might think. That was prompted largely by this piece in the New Yorker by Steven Brill about NYC's so-called "rubber room." Just so we're all on the same page here, the rubber room (actually a variety of locations throughout the city) are where teachers who are accused of misdeeds or rated unsatisfactory by their principals are sent to twiddle their thumbs until they get a hearing. Teachers often report there for years, getting paid to do little besides gossip and catch up on their crosswords.
When I started seeing reaction to Brill's piece (before reading it), I thought it was going to be an illuminating piece on life inside the rubber room. So I read it. Turns out it was a slanted hit job on teachers and their unions instead. Maybe it's not the New Yorker's style, but there was never any pretense that this would be an objective piece or attempt to answer honest questions. Instead, he went searching for proof that teachers' unions are bad and that Joel Klein is right.
He manages to uncover some anecdotes that should thoroughly embarrass the union and highlight some rules and procedures that need to be altered. But he does so with a chip on his shoulder and a disjointed view of reality. Here are his seven deadly sins:
1.) He seems to assume that all teachers in the rubber room are bad teachers and/or bad people (the subtitle is "the battle over New York City's worst teachers"). He snidely notes that a teacher who has been in the rubber room for two years and claims to deserve every penny of her salary got into teaching because she "loves children." I have little doubt that a good number of the people in the rubber room shouldn't be teaching. But I'm also quite sure that more than a couple never should've been assigned there in the first place. So to suddenly decide that all 600 or so teachers are among the worst human beings alive seems rather extreme. Especially since he reports only being granted access to the records of one of the teachers. In the future, he should learn more about people before condemning them.
2.) He cites numbers as gospel that don't mean nearly what he says they do. He says that in 2002 97% of teachers earn tenure, that 99% are rated satisfactory, and that almost none were fired. But he fails to take into account that at least half of all teachers in NYC quit before they're ever eligible for tenure. Or that a heck of a lot of teachers leave their school and/or the profession without being rated unsatisfactory or being fired. Besides, those stats could just as easily mean that there are a lot of good teachers -- or, if that's not the case, that principals are failing to report problems in their schools. A more critical examination of the numbers was in order.
3.) He cites the 2006 Brookings report that reported back of the envelope calculations concluding that having the best teachers for four consecutive years can close the achievement gap. Despite the fact that this haphazard calculation has been widely discredited by an array of scholars. There was never any evidence that this statement might be true, it was simply an ill-informed guess. But it's awfully convenient to cite when cherry-picking evidence to use against your enemies.
4.) His comments on the Absent Teacher Reserve are overly simplistic. It's likely that a good number of ATR teachers who remain without jobs for a long period of time are either not ideal candidates or not trying too hard to find a job. But there's also evidence that principals shy away from hiring experienced ATR teachers in order to hire younger, cheaper teachers. This could be corrected by changing the way such hires affect a school's budget. Brill fails to address this. Or the fact that teachers are in the ATR through no fault or shortcoming of their own. Nor does he take any time to investigate whether the NYC Dept. of Education -- which, usually, is responsible for teachers being placed in the ATR -- is making a good faith attempt to help these teachers find jobs. Or at least providing them with the necessary information regarding job openings in a practical manner.
5.) He presents an uncritical look at value-added test scores -- assuming that anybody who says using such measures is a good thing is smart and anybody who says we should be careful when using it is obstinate. These measures are simply not ready for prime time. Period. NYC tried using invalid measures to grade schools, and what was the result? 97% of all schools in one of the worst districts in the country were awarded an A or a B. Wouldn't it be ironic if the use of test scores for tenure decisions was adopted in order to make it easier to fire teachers and, instead, it made it harder? Given how much easier this year's state test was than the previous year's, that's likely what would have happened -- gains in scores would indicate that almost all teachers did a phenomenal job, making it harder to deny tenure to those with questionable records. Let's think through the possible ramifications of using value-added scores before deciding to do so.
6.) Brill reports that 1.8% of NYC's 87,300 teachers received unsatisfactory ratings last year, but that only 50 people total are in the rubber room while "the rest still teach." Given that most people remain in the rubber room for 2-3 years, this would mean that only about 20 of the 1570 or so teachers rated unsatisfactory each year are removed from the classroom, while the other 1550 remain. And that's simply not true. It would mean that only 20 per year are removed from the classroom and choose to fight it. I'm almost positive that more than 1% of the teachers rated unsatisfactory are not allowed to remain in the classroom the following year. But even if I'm wrong about that, he neither includes those who are asked to leave and do so quietly nor the umpteen teachers who self-select out or are asked to leave without a corresponding U rating.
7.) Worst of all, he ruined what could've been a fantastic article. He found some legitimate problems that need to be dealt with in the next round of contract negotiations. He has some fascinating stories to tell from his experiences in the mysterious rubber room. I find it especially interesting that every teacher he mentions seems so adamant that they've been persecuted by a Bloomberg/Klein regime to which they seem militantly opposed. I would really like to know more about why this is. Instead, he mockingly dismisses their claims and wonders why it's so hard to fire "freaks" and "crazies." And this is typical of the rest of the article. He could've reported facts, provided illuminating stories, and begun a sensible discussion on what to do about all this. Instead, he provides a slanted analysis of a variety of misleading facts peppered with snide remarks. And that's too bad.
On Sunday, I wrote that it's probably not as hard to get rid of "bad teachers" as many might think. That was prompted largely by this piece in the New Yorker by Steven Brill about NYC's so-called "rubber room." Just so we're all on the same page here, the rubber room (actually a variety of locations throughout the city) are where teachers who are accused of misdeeds or rated unsatisfactory by their principals are sent to twiddle their thumbs until they get a hearing. Teachers often report there for years, getting paid to do little besides gossip and catch up on their crosswords.
When I started seeing reaction to Brill's piece (before reading it), I thought it was going to be an illuminating piece on life inside the rubber room. So I read it. Turns out it was a slanted hit job on teachers and their unions instead. Maybe it's not the New Yorker's style, but there was never any pretense that this would be an objective piece or attempt to answer honest questions. Instead, he went searching for proof that teachers' unions are bad and that Joel Klein is right.
He manages to uncover some anecdotes that should thoroughly embarrass the union and highlight some rules and procedures that need to be altered. But he does so with a chip on his shoulder and a disjointed view of reality. Here are his seven deadly sins:
1.) He seems to assume that all teachers in the rubber room are bad teachers and/or bad people (the subtitle is "the battle over New York City's worst teachers"). He snidely notes that a teacher who has been in the rubber room for two years and claims to deserve every penny of her salary got into teaching because she "loves children." I have little doubt that a good number of the people in the rubber room shouldn't be teaching. But I'm also quite sure that more than a couple never should've been assigned there in the first place. So to suddenly decide that all 600 or so teachers are among the worst human beings alive seems rather extreme. Especially since he reports only being granted access to the records of one of the teachers. In the future, he should learn more about people before condemning them.
2.) He cites numbers as gospel that don't mean nearly what he says they do. He says that in 2002 97% of teachers earn tenure, that 99% are rated satisfactory, and that almost none were fired. But he fails to take into account that at least half of all teachers in NYC quit before they're ever eligible for tenure. Or that a heck of a lot of teachers leave their school and/or the profession without being rated unsatisfactory or being fired. Besides, those stats could just as easily mean that there are a lot of good teachers -- or, if that's not the case, that principals are failing to report problems in their schools. A more critical examination of the numbers was in order.
3.) He cites the 2006 Brookings report that reported back of the envelope calculations concluding that having the best teachers for four consecutive years can close the achievement gap. Despite the fact that this haphazard calculation has been widely discredited by an array of scholars. There was never any evidence that this statement might be true, it was simply an ill-informed guess. But it's awfully convenient to cite when cherry-picking evidence to use against your enemies.
4.) His comments on the Absent Teacher Reserve are overly simplistic. It's likely that a good number of ATR teachers who remain without jobs for a long period of time are either not ideal candidates or not trying too hard to find a job. But there's also evidence that principals shy away from hiring experienced ATR teachers in order to hire younger, cheaper teachers. This could be corrected by changing the way such hires affect a school's budget. Brill fails to address this. Or the fact that teachers are in the ATR through no fault or shortcoming of their own. Nor does he take any time to investigate whether the NYC Dept. of Education -- which, usually, is responsible for teachers being placed in the ATR -- is making a good faith attempt to help these teachers find jobs. Or at least providing them with the necessary information regarding job openings in a practical manner.
5.) He presents an uncritical look at value-added test scores -- assuming that anybody who says using such measures is a good thing is smart and anybody who says we should be careful when using it is obstinate. These measures are simply not ready for prime time. Period. NYC tried using invalid measures to grade schools, and what was the result? 97% of all schools in one of the worst districts in the country were awarded an A or a B. Wouldn't it be ironic if the use of test scores for tenure decisions was adopted in order to make it easier to fire teachers and, instead, it made it harder? Given how much easier this year's state test was than the previous year's, that's likely what would have happened -- gains in scores would indicate that almost all teachers did a phenomenal job, making it harder to deny tenure to those with questionable records. Let's think through the possible ramifications of using value-added scores before deciding to do so.
6.) Brill reports that 1.8% of NYC's 87,300 teachers received unsatisfactory ratings last year, but that only 50 people total are in the rubber room while "the rest still teach." Given that most people remain in the rubber room for 2-3 years, this would mean that only about 20 of the 1570 or so teachers rated unsatisfactory each year are removed from the classroom, while the other 1550 remain. And that's simply not true. It would mean that only 20 per year are removed from the classroom and choose to fight it. I'm almost positive that more than 1% of the teachers rated unsatisfactory are not allowed to remain in the classroom the following year. But even if I'm wrong about that, he neither includes those who are asked to leave and do so quietly nor the umpteen teachers who self-select out or are asked to leave without a corresponding U rating.
7.) Worst of all, he ruined what could've been a fantastic article. He found some legitimate problems that need to be dealt with in the next round of contract negotiations. He has some fascinating stories to tell from his experiences in the mysterious rubber room. I find it especially interesting that every teacher he mentions seems so adamant that they've been persecuted by a Bloomberg/Klein regime to which they seem militantly opposed. I would really like to know more about why this is. Instead, he mockingly dismisses their claims and wonders why it's so hard to fire "freaks" and "crazies." And this is typical of the rest of the article. He could've reported facts, provided illuminating stories, and begun a sensible discussion on what to do about all this. Instead, he provides a slanted analysis of a variety of misleading facts peppered with snide remarks. And that's too bad.