I'm in the middle of finals week, so I only have a couple minutes, but I think this is worth looking at for a second.
A grand headline in the NY Times today reads "Study Suggests Math Teachers Scrap Balls and Slices." The article summarizes a study being published today that found people learned math better when taught abstractly (think: formulas) than when taught using manipulatives (the balls and slices to which the headline refers).
The finding is extremely interesting. And I saw it echoed and trumpeted all over the place. And then I read the article. It was a short study on one kind of problem done with college kids. Meaning that we have virtually no idea, based on this study, whether this is true for elementary school kids who learn this way for 6 years.
This is no critique of the authors -- there's value to their study -- but, for the life of me, I can't figure out why everybody seems to think this means that we should stop using manipulatives forevermore. This sheds little, if any, light on that issue.
I'm disappointed in the NY Times, and I'm disappointed in everybody else who parroted these results without explaining what they really meant.
(disclaimer: finals are making me a bit grumpier than usual)
Update: Thanks, Dr. Dorn
Friday, April 25, 2008
Wednesday, April 23, 2008
What's Wrong with Ed Policy
Most of the time I can't imagine focusing on anything but education policy for the rest of my life. But, at certain times, I wonder if it's worth it. Petty arguments and ideological venom seem to be poisoning the field, and I'm not sure if I can take it. Here's the latest example of what's wrong with education policy. We'll follow it step by step.
Step 1. In the wake of the latest evidence on vouchers in Milwaukee, Sol Stern (a noted supporter of vouchers for many years) writes that "markets in education may not be a panacea," that there has been no "Milwaukee Miracle," and that we should consider other options.
Step 2. Jason Riley writes a piece in the Wall St. Journal in defense of vouchers and choice, arguing that "seeking panaceas and miracles is setting the bar for success unreasonably high."
Step 3. Letters to the editor pour in (here and here), including some from teachers who argue that the biggest problem is that their kids come to school far behind.
Step 4. George Clowes writes in to call these letters condescending and suggests that we should blame teachers instead of students or families ("Instead of blaming social conditions for student failure, let's just ask teachers to do their job: Educate students").
Step 5. Greg Forster revels in this "smackdown" on Jay Greene's blog and compares Clowes to Mr. T.
Notice that the level of discourse (not to mention civility) devolves with each step. By the end we're left with one side rooting against the other. How is this productive? It's not.
If the field of education policy becomes nothing but half the participants fighting to defeat the other half of the participants, then I have no interest in participating. Being childish and immature can be fun, but it has no place in education policy. If it's not possible to make an honest attempt to find solutions to problems without a large group making knee-jerk judgments regardless of what you find, then count me out.
I see no utility in rooting for one solution or blaming one group instead of another. In other words, can't we focus on attacking the problem instead of each other?
Step 1. In the wake of the latest evidence on vouchers in Milwaukee, Sol Stern (a noted supporter of vouchers for many years) writes that "markets in education may not be a panacea," that there has been no "Milwaukee Miracle," and that we should consider other options.
Step 2. Jason Riley writes a piece in the Wall St. Journal in defense of vouchers and choice, arguing that "seeking panaceas and miracles is setting the bar for success unreasonably high."
Step 3. Letters to the editor pour in (here and here), including some from teachers who argue that the biggest problem is that their kids come to school far behind.
Step 4. George Clowes writes in to call these letters condescending and suggests that we should blame teachers instead of students or families ("Instead of blaming social conditions for student failure, let's just ask teachers to do their job: Educate students").
Step 5. Greg Forster revels in this "smackdown" on Jay Greene's blog and compares Clowes to Mr. T.
Notice that the level of discourse (not to mention civility) devolves with each step. By the end we're left with one side rooting against the other. How is this productive? It's not.
If the field of education policy becomes nothing but half the participants fighting to defeat the other half of the participants, then I have no interest in participating. Being childish and immature can be fun, but it has no place in education policy. If it's not possible to make an honest attempt to find solutions to problems without a large group making knee-jerk judgments regardless of what you find, then count me out.
I see no utility in rooting for one solution or blaming one group instead of another. In other words, can't we focus on attacking the problem instead of each other?
Tuesday, April 22, 2008
Is America's Entire Education System Failing?
Bashing our education system has been a popular sport since well before I was born. Indeed, if I didn't think there were serious problems I wouldn't be studying education policy. Nonetheless, I sometimes wonder if people go too far. After reading Bob Herbert's op-ed in the Times today I decided I should take a little time to wonder if our education system is really that bad.Though standardized tests have their shortcomings, they're usually the best way to compare the performance of different countries' school systems. Pasted above is a chart I made last semester of the reading ("Literacy") scores from just the G-8 countries (basically the 8 wealthiest countries in the world) so that we're comparing apples to apples. The scores are broken down by percentile for each country and then ranked.
The U.S. ranks 5th out of the 8 countries for median score, substantiating the feeling that we're falling behind and that reform is needed. But, you can see a large difference in performance for the top scorers vs. the bottom scorers in our country. Only two of the countries have top scorers that do better than the top scorers (90th and 95th percentiles) in the United States. Meanwhile, the bottom scorers (5th and 10th percentiles) in the U.S. only outperform two of the other seven countries.
So maybe the problem isn't really the entire system. Maybe our system does fine for some students while it fails others. You hear a lot in the news about impoverished schools in the inner-city or rural America, but you also hear a lot about the proliferation of AP tests, and the increasing competitiveness of admissions to the top colleges.
So maybe it's not our system that's broken but, rather, part of our system. We have both a lot of success stories and a lot of failures. Maybe we should be trying to fix the system that exists for the least fortunate students rather than berating the entire system.
A Solution to the Shortage of Science Students?
Ever since the Soviets launched Sputnik we've been worrying about how well our schools are doing at teaching science. More recently, people have worried that not enough college students are choosing science majors. In the long-run we worry that we won't have enough people to drive a high-tech economy. Well, last week's edition of The Economist has a solution: let in more foreigners with science degrees.
The Economist, of course, will almost always think something is a good idea when it involves fewer restrictions on the market -- but they include some interesting statistics in defense of their position.
-"Some 70 or so of the 300 Americans who have won Nobel prizes since 1901 were immigrants"
-"Some 40% of American PhDs in science and engineering go to immigrants"
They write that there are 85,000 H1B visas (for highly educated immigrants) allotted each year and that hundreds of thousands of people each year are unsuccessful in obtaining one when they enter their names in the visa lottery. They also claim that both Bill Gates and respected economists agree that "every foreigner who is given an H1B visa creates jobs for five regular Americans."
All of this adds up to a shortcut solution for fixing our shortage of science students -- let in more science grads from abroad. An imperfect solution to be sure, but it's probably better than simply letting our economy fall into oblivion over the next few decades. And, heck, it has to be cheaper to grant visas than provide two decades of education.
The Economist, of course, will almost always think something is a good idea when it involves fewer restrictions on the market -- but they include some interesting statistics in defense of their position.
-"Some 70 or so of the 300 Americans who have won Nobel prizes since 1901 were immigrants"
-"Some 40% of American PhDs in science and engineering go to immigrants"
They write that there are 85,000 H1B visas (for highly educated immigrants) allotted each year and that hundreds of thousands of people each year are unsuccessful in obtaining one when they enter their names in the visa lottery. They also claim that both Bill Gates and respected economists agree that "every foreigner who is given an H1B visa creates jobs for five regular Americans."
All of this adds up to a shortcut solution for fixing our shortage of science students -- let in more science grads from abroad. An imperfect solution to be sure, but it's probably better than simply letting our economy fall into oblivion over the next few decades. And, heck, it has to be cheaper to grant visas than provide two decades of education.
Sunday, April 20, 2008
Could Judging Teachers by Test Scores do Harm?
I'm sick of the discussion over the NY tenure law but, for some reason, I can't resist chipping in another two cents worth of thoughts. I've already written twice (here and here) that I can't see much utility to using the test scores, in their current form, to decide on teacher tenure in NY -- so I won't harp on that again.
NYCEducator argues that the scores are not needed to tell which teachers should be let go, while Kevin Carey fails to see the harm in including the scores in tenure decisions.
If it turns out that the scores provide even a snippet of useful data (which is debatable) then I'd agree with Carey that it also wouldn't do much harm to include these scores in decision-making about teacher ratings (and, hence, firings). But lets turn to a different side of this that I haven't seen anybody address. I've already argued that not using them does no harm, and I think Carey's arguing that using them would also do no harm. We can argue all day over whether the test scores will provide any useful knowledge, but let's think about whether using them could do any harm. Since I'm in the middle of finals week, I'll choose the most intellectually flabby way possible to examine this question -- a game of what if.
Imagine that schools in NY started using the test data (which is questionable in accuracy and only covers 3rd-8th grade Math and English):
What if this were used as the primary way to evaluate teachers instead of a supplemental one?
What if Math and English teachers were scrutinized more closely than teachers in all other subjects?
What if upper-elementary and middle school teachers were scrutinized more closely than other teachers?
What if a good teacher had artificially low scores?
What if a bad teacher had artificially high scores?
What if a teacher had high test scores but abused students?
What if we let gym teachers do whatever they want while we closely monitor other teachers?
What if that gym teachers (not to pick on gym teachers), while we're focusing on other teachers, takes advantage of a student?
What if we forget to monitor serious breaches of ethics because we're so focused on test scores?
What if a teacher cheats on the tests but cleverly enough so as to avoid detection?
What would happen if the hadn't passed is really just speculation, but it's entirely possible that using these test scores to evaluate teachers could have unintended consequences -- and severely hurt students as result.
NYCEducator argues that the scores are not needed to tell which teachers should be let go, while Kevin Carey fails to see the harm in including the scores in tenure decisions.
If it turns out that the scores provide even a snippet of useful data (which is debatable) then I'd agree with Carey that it also wouldn't do much harm to include these scores in decision-making about teacher ratings (and, hence, firings). But lets turn to a different side of this that I haven't seen anybody address. I've already argued that not using them does no harm, and I think Carey's arguing that using them would also do no harm. We can argue all day over whether the test scores will provide any useful knowledge, but let's think about whether using them could do any harm. Since I'm in the middle of finals week, I'll choose the most intellectually flabby way possible to examine this question -- a game of what if.
Imagine that schools in NY started using the test data (which is questionable in accuracy and only covers 3rd-8th grade Math and English):
What if this were used as the primary way to evaluate teachers instead of a supplemental one?
What if Math and English teachers were scrutinized more closely than teachers in all other subjects?
What if upper-elementary and middle school teachers were scrutinized more closely than other teachers?
What if a good teacher had artificially low scores?
What if a bad teacher had artificially high scores?
What if a teacher had high test scores but abused students?
What if we let gym teachers do whatever they want while we closely monitor other teachers?
What if that gym teachers (not to pick on gym teachers), while we're focusing on other teachers, takes advantage of a student?
What if we forget to monitor serious breaches of ethics because we're so focused on test scores?
What if a teacher cheats on the tests but cleverly enough so as to avoid detection?
What would happen if the hadn't passed is really just speculation, but it's entirely possible that using these test scores to evaluate teachers could have unintended consequences -- and severely hurt students as result.
Saturday, April 19, 2008
More on School Expenditures Over Time
This is a follow-up to my last post where I posted a graph that surprised me. As a percentage of GDP, per-pupil expenditures have remained flat over time -- other than during WWII a little under a dollar per billion dollars of GDP per student. The first comment (by the author of the post that prompted my post) asked why we should expect to spending on education to rise over time compared to GDP. I gave a terse response in my follow-up comment, but I'd like to take more time to flesh that out.
To oversimplify economic theory, many goods can be counted as either necessities or luxuries and people (and countries) are expected to increase their spending on luxury items as income increases while simultaneously decreasing their spending on necessities relative to their income. So, for example, somebody making a million dollars per year is expected to spend a smaller portion of their budget on food and shelter than is somebody making twenty thousand dollars per year. At the same time, the wealthier person is expected to spend a greater share of their budget on wine or boats or vacation homes. They're able to do this because they have more "discretionary income" -- money beyond what's needed to pay for the bare necessities of life.
Countries work in a similar fashion. There are certain things that even the poorest country must spend money on (e.g. a basic system of defense or a basic network of roads) and some things that wealthy countries can afford to devote a larger share of income towards as their discretionary income increases (e.g. science research or museums).
So, whether you believe that education spending should increase, decrease, or remain the same relative to GDP depends on how you view education spending. If you think that it's a bare necessity, then you would expect it to decrease over time as we gain more money. If you view it as more of a luxury good, then you'd expect spending to increase over time as discretionary income increases. If you view it as somewhere in between, then you might expect spending to stay relatively flat over time.
I said at the beginning of the post that I was surprised when I saw that it was flat. I was surprised because I was under the distinct impression that countries tended to spend a larger share of GDP on education as they become wealthier. I guess further investigation is needed to find out whether I was wrong or if the US is an outlier.
Update: Actually, when I corrected the graph, it appears as though spending per pupil compared to GDP per capita has been falling rather than flat
To oversimplify economic theory, many goods can be counted as either necessities or luxuries and people (and countries) are expected to increase their spending on luxury items as income increases while simultaneously decreasing their spending on necessities relative to their income. So, for example, somebody making a million dollars per year is expected to spend a smaller portion of their budget on food and shelter than is somebody making twenty thousand dollars per year. At the same time, the wealthier person is expected to spend a greater share of their budget on wine or boats or vacation homes. They're able to do this because they have more "discretionary income" -- money beyond what's needed to pay for the bare necessities of life.
Countries work in a similar fashion. There are certain things that even the poorest country must spend money on (e.g. a basic system of defense or a basic network of roads) and some things that wealthy countries can afford to devote a larger share of income towards as their discretionary income increases (e.g. science research or museums).
So, whether you believe that education spending should increase, decrease, or remain the same relative to GDP depends on how you view education spending. If you think that it's a bare necessity, then you would expect it to decrease over time as we gain more money. If you view it as more of a luxury good, then you'd expect spending to increase over time as discretionary income increases. If you view it as somewhere in between, then you might expect spending to stay relatively flat over time.
I said at the beginning of the post that I was surprised when I saw that it was flat. I was surprised because I was under the distinct impression that countries tended to spend a larger share of GDP on education as they become wealthier. I guess further investigation is needed to find out whether I was wrong or if the US is an outlier.
Update: Actually, when I corrected the graph, it appears as though spending per pupil compared to GDP per capita has been falling rather than flat
Has Education Spending Really Skyrocketed?
I just read a post at D-Ed Reckoning about the astronomical increases in per-pupil education spending over the past century. Though expenditures rose in terms of real dollars, I wondered how they compared to the GDP of the U.S. So I found data on the GDP of the U.S. since 1940 (here) and combined the two. The results are above. As a percentage of GDP, per-pupil expenditures (other than a drop during WWII) have stayed almost exactly flat over the past 65 years. The chart is a bit small, but if you click on it then you can see a larger version.
The question remains over what the most relevant comparison is over time. The fact that per-pupil spending relative to GDP has remained the same over time indicates that we're putting forth about the same financial effort relative to our capacity to spend, but if our capacity to spend has increased then we might still expect larger gains in achievement over that time. On the other hand, increased spending capacity should also increase the country's discretionary income -- apparently we've chosen to spend that on items other than education.
Update: More on this in a follow-up post
Later update: Corrected version of the graph (oops) here
Friday, April 18, 2008
How Should We Get Rid of Bad Teachers?
My my last post had far too much vitriol for my liking, and I feel the need to cleanse my system with a less argumentative and more thoughtful post.
One problem in schools is that there are bad teachers who continue to harm students year after year. What do we do about these teachers? Well, fire them of course -- that's the easy answer.
Unfortunately, the solution is far from simple. Furthermore, the extent of the problem is maybe just as complex.
Let's start with the scope of the problem. How many bad teachers are there? What qualifies a teacher as "bad?" Are there more bad teachers than there are bad lawyers or accountants? Are teachers bad because they have no talent, put forth no effort, or because they attempt to harm students? I'm not sure that we have answers to any of these questions.
Assuming that there are a lot of bad teachers, the next question is why. Why are there so many bad teachers? Are the unions protecting them? Are principals failing to evaluate thoroughly? Do teachers tend to burn out? Do bad people try to be put in a position of authority? Are there no good teachers to replace them? There might be merit to theories that accompany all of these questions.
So, given that we (or at least I) don't really know how bad the problem is, how should we deal with the problem? Create better evaluation systems? Outlaw unions? Put more pressure on principals to evaluate teachers? Put more pressure on bad teachers? Give bad teachers more training? Fire them all?
I'm of the distinct opinion that there's no easy solution. That, of course, doesn't mean that we shouldn't try or should indefinitely delay action (any teacher that abuses a student, for example, should spend another second in the classroom). But I'll bet we'd be closer to an answer if people tried to answer all of these questions rather than just pushing a pet solution. In other words, it's likely that the solution is no simpler than the problem.
More questions inspired by comments: Would "bad" teachers be more effective in a different environment? When should "bad" teachers be filtered out?
One problem in schools is that there are bad teachers who continue to harm students year after year. What do we do about these teachers? Well, fire them of course -- that's the easy answer.
Unfortunately, the solution is far from simple. Furthermore, the extent of the problem is maybe just as complex.
Let's start with the scope of the problem. How many bad teachers are there? What qualifies a teacher as "bad?" Are there more bad teachers than there are bad lawyers or accountants? Are teachers bad because they have no talent, put forth no effort, or because they attempt to harm students? I'm not sure that we have answers to any of these questions.
Assuming that there are a lot of bad teachers, the next question is why. Why are there so many bad teachers? Are the unions protecting them? Are principals failing to evaluate thoroughly? Do teachers tend to burn out? Do bad people try to be put in a position of authority? Are there no good teachers to replace them? There might be merit to theories that accompany all of these questions.
So, given that we (or at least I) don't really know how bad the problem is, how should we deal with the problem? Create better evaluation systems? Outlaw unions? Put more pressure on principals to evaluate teachers? Put more pressure on bad teachers? Give bad teachers more training? Fire them all?
I'm of the distinct opinion that there's no easy solution. That, of course, doesn't mean that we shouldn't try or should indefinitely delay action (any teacher that abuses a student, for example, should spend another second in the classroom). But I'll bet we'd be closer to an answer if people tried to answer all of these questions rather than just pushing a pet solution. In other words, it's likely that the solution is no simpler than the problem.
More questions inspired by comments: Would "bad" teachers be more effective in a different environment? When should "bad" teachers be filtered out?
More on the NY Tenure Law
I wrote before on the tenure law that was passed in NY and had no intention of writing about it again, in part because I think the end result is essentially inconsequential. But the more I read about it the more annoyed I become with critics of the law. Not because it's not a bad law -- it is -- but because the effects of the law are constantly overstated.
The latest offender is Kevin Carey over at The Quick and the Ed. I hesitate to pick on him b/c I've read more ridiculous assertions than his, but his just happened to be the straw that broke the camel's back. Before I begin let me say that he's written a lot of other things that I thought were very good.
Though not the main focus of his post, Carey implies at the end that using standardized tests to make tenure decisions would eliminate a lot of bad teachers in NY. That theory, of course, makes sense on paper -- but it plays out differently in reality.
The tests are ill-suited for use as a measure of the amount that students learned in one year for three major reasons: 1.) They're given well before the end of the year, 2.) They're not designed to be value-added tests -- they're designed to determine proficiency in each separate grade, and 3.) The majority of teachers do not teach tested subjects.
Carey writes that academics are too cautious about endorsing programs, and he may have a point, but this one case where the caution is merited. It's not the case that the information gathered from these tests would be only a little helpful rather than a lot, it's the case that the information gathered from the tests would be so inaccurate as to be useless.
He writes that there are a number of bad teachers in NYC (and references this as proof). Of course there are a number of bad teachers -- anybody who disputes this is dead wrong -- but would the information gained from these tests really solve the problem?
First of all, a number of the bad teachers referenced in that blog post do not teach a tested subject (3rd-8th grade Math or English) and would, therefore, be completely unaffected by the use of the test scores. This means that a 4th grade English teacher, say, with slightly sub-par scores might face some heat while the Spanish teacher who takes his shirt off in class will not. Secondly, all of the offenses mentioned in the post are observable by peers and supervisors -- a test will not inform a principal that a teacher had an affair with a student. Thirdly, because the test results would be somewhat random it very likely that a good teacher could get bad results and a bad teacher good results -- completely defeating the purpose of using the tests in the first place. In short, using test results would likely hurt at least as many people as it hurt.
It is clear, then, that the test results should not be a deciding factor when evaluating teachers. The argument that NY should hold off for a couple years until they develop a better value-added testing system is correct.
As I previously wrote, it is a heavy-handed law and was passed in a devious manner -- which are both worthy of negative press attention -- but it is quite clear that the rule regarding the use of test results will not, in and of itself, do substantial harm over the next two years. I find any argument that it will to be disingenuous.
Let me end with two clarifications. 1.) I do not object to change. Not in the least. Bad teachers should be removed from schools. I will never, ever, argue this point. The tenure law, however, does not prevent bad teachers from being removed from schools. 2.) I bear no personal ill-will toward Kevin Carey. If I've been unfair to him then I assume either he or somebody else will let me know.
Update: I've been informed that I misspelled Kevin Carey's name 4 out of 5 times in the previous edition of this post. My apologies -- it's been corrected.
The latest offender is Kevin Carey over at The Quick and the Ed. I hesitate to pick on him b/c I've read more ridiculous assertions than his, but his just happened to be the straw that broke the camel's back. Before I begin let me say that he's written a lot of other things that I thought were very good.
Though not the main focus of his post, Carey implies at the end that using standardized tests to make tenure decisions would eliminate a lot of bad teachers in NY. That theory, of course, makes sense on paper -- but it plays out differently in reality.
The tests are ill-suited for use as a measure of the amount that students learned in one year for three major reasons: 1.) They're given well before the end of the year, 2.) They're not designed to be value-added tests -- they're designed to determine proficiency in each separate grade, and 3.) The majority of teachers do not teach tested subjects.
Carey writes that academics are too cautious about endorsing programs, and he may have a point, but this one case where the caution is merited. It's not the case that the information gathered from these tests would be only a little helpful rather than a lot, it's the case that the information gathered from the tests would be so inaccurate as to be useless.
He writes that there are a number of bad teachers in NYC (and references this as proof). Of course there are a number of bad teachers -- anybody who disputes this is dead wrong -- but would the information gained from these tests really solve the problem?
First of all, a number of the bad teachers referenced in that blog post do not teach a tested subject (3rd-8th grade Math or English) and would, therefore, be completely unaffected by the use of the test scores. This means that a 4th grade English teacher, say, with slightly sub-par scores might face some heat while the Spanish teacher who takes his shirt off in class will not. Secondly, all of the offenses mentioned in the post are observable by peers and supervisors -- a test will not inform a principal that a teacher had an affair with a student. Thirdly, because the test results would be somewhat random it very likely that a good teacher could get bad results and a bad teacher good results -- completely defeating the purpose of using the tests in the first place. In short, using test results would likely hurt at least as many people as it hurt.
It is clear, then, that the test results should not be a deciding factor when evaluating teachers. The argument that NY should hold off for a couple years until they develop a better value-added testing system is correct.
As I previously wrote, it is a heavy-handed law and was passed in a devious manner -- which are both worthy of negative press attention -- but it is quite clear that the rule regarding the use of test results will not, in and of itself, do substantial harm over the next two years. I find any argument that it will to be disingenuous.
Let me end with two clarifications. 1.) I do not object to change. Not in the least. Bad teachers should be removed from schools. I will never, ever, argue this point. The tenure law, however, does not prevent bad teachers from being removed from schools. 2.) I bear no personal ill-will toward Kevin Carey. If I've been unfair to him then I assume either he or somebody else will let me know.
Update: I've been informed that I misspelled Kevin Carey's name 4 out of 5 times in the previous edition of this post. My apologies -- it's been corrected.
Random Thought on Generalizing
Before entering academia I was always told not to make generalizations about people. In academia, however, the main goal is to find results that you can generalize to the largest number of people. There is, of course, a reason for this -- but I still find it odd when I think about it.
Thursday, April 17, 2008
Don't Interpret an Article's Findings based its Title
I've mentioned a few times that research is nowhere near as definitive as I imagined it was before I started grad school. Every paper has flaws, limitations, and shortcomings -- no matter how the prestigious the journal in which it was published. Peer review isn't perfect. Sometimes people are so focused on examining an issue in one manner that they forget to look at it in other ways.
I found one such example yesterday. In an influential piece in the American Economic Review (a very prestigious journal) written by Caroline Hoxby* in 2000 ("Does Competition among Public Schools Benefit Students and Taxpayers?") found that schools with more competition were "more productive" (higher scores with less money spent). In the paper, competition is determined by the number of school districts within a metropolitan area and the number of streams (rivers/creeks, not any fancy academic term) in that same area. The theory is that more streams led to the creation of more towns (and, hence, more school districts) because they provided geographic boundaries. Those areas where more streams led to the creation of more districts were said to have more choice b/c people could choose to move to more districts closer to home and, therefore, the districts had to compete more not to lose students. The statistics are really sophisticated, and we devoted two full periods of class to examining the underlying theory, the statistical analysis, and later re-analysis (which got quite heated, and in which I have no desire of involving myself at this point in time), but that is the general gist.
Anyway, we were focused on understanding how more competition and choice led to more productive schools when a thought occurred to me. Metropolitan areas that have more streams and, therefore, smaller towns and smaller school districts, don't just have more school districts nearby -- they also have smaller towns and smaller districts. I wondered aloud whether it was possible that the effects of smaller districts (perhaps a tighter-knit community, more parental involvement, less bureaucracy, etc.) might explain the positive effects in addition to, or instead of, more choice.
I'm still somewhat in doubt that the author and numerous reviewers missed that point, but I don't see any indication that district size was controlled for and can find no other method of counting this out. So, for now, my conclusion is that people get stuck thinking about something one way and forget to think about it in others.
*I often refrain from naming names. I mention it in this case only b/c understanding my thought is wholly dependent upon knowing to which article I am referring. I have absolutely no desire to start a fight with Caroline Hoxby (who has proven many times over that she is quite smart -- I particularly like her outside-the-box idea of using streams as exogenous predictor of # of districts); I simply wanted to share an interesting thought on education policy.
I found one such example yesterday. In an influential piece in the American Economic Review (a very prestigious journal) written by Caroline Hoxby* in 2000 ("Does Competition among Public Schools Benefit Students and Taxpayers?") found that schools with more competition were "more productive" (higher scores with less money spent). In the paper, competition is determined by the number of school districts within a metropolitan area and the number of streams (rivers/creeks, not any fancy academic term) in that same area. The theory is that more streams led to the creation of more towns (and, hence, more school districts) because they provided geographic boundaries. Those areas where more streams led to the creation of more districts were said to have more choice b/c people could choose to move to more districts closer to home and, therefore, the districts had to compete more not to lose students. The statistics are really sophisticated, and we devoted two full periods of class to examining the underlying theory, the statistical analysis, and later re-analysis (which got quite heated, and in which I have no desire of involving myself at this point in time), but that is the general gist.
Anyway, we were focused on understanding how more competition and choice led to more productive schools when a thought occurred to me. Metropolitan areas that have more streams and, therefore, smaller towns and smaller school districts, don't just have more school districts nearby -- they also have smaller towns and smaller districts. I wondered aloud whether it was possible that the effects of smaller districts (perhaps a tighter-knit community, more parental involvement, less bureaucracy, etc.) might explain the positive effects in addition to, or instead of, more choice.
I'm still somewhat in doubt that the author and numerous reviewers missed that point, but I don't see any indication that district size was controlled for and can find no other method of counting this out. So, for now, my conclusion is that people get stuck thinking about something one way and forget to think about it in others.
*I often refrain from naming names. I mention it in this case only b/c understanding my thought is wholly dependent upon knowing to which article I am referring. I have absolutely no desire to start a fight with Caroline Hoxby (who has proven many times over that she is quite smart -- I particularly like her outside-the-box idea of using streams as exogenous predictor of # of districts); I simply wanted to share an interesting thought on education policy.
Wednesday, April 16, 2008
Debate is Good . . . No it's not
The easiest thing about education research is finding things that don't work (or, at least, things that have only a very small effect). When I was teaching I saw a lot of things that didn't work. Part of this is due to the fact that a lot of bad ideas get implemented, part is due to the fact that a lot of good ideas get implemented badly, and part is due to the fact that there is no simple way to "fix" education. I'm not sure that one can analyze education policy without turning cynical.
An article in EdWeek yesterday, however, reminded me of one thing that does work. Debate. About the only good experience I had at my school was coaching the debate team. Now, granted, the team was made up of kids who volunteered to spend their time after-school and on Saturdays hanging out with me, and I had the power to remove anybody who didn't fully engage during that time period, but, nonetheless, it was clear that the kids I worked with gained something from it.
I saw very few examples of kids being excited to learn in my school, but these kids were. I saw very few examples of kids understanding complex issues in my school, but these kids did. Not because of anything that I did, but because debates are fun and because the only way to win a debate is to deeply understand all the issues surrounding a topic.
I have no idea if these kids made huge gains on their test results and, quite frankly, I don't care. I don't care what the regression analysis or t-tests show. I saw it in their eyes. I watched their arguments develop. I felt their excitement. I ceded when they begged me to make practices longer and more frequent. They gained knowledge about important topics. They gained confidence in themselves. They gained the ability to construct and defend an argument (and poke holes in those that others make). Whether or not they bubbled in more correct answers on the state test, they were better off as a result of participating in debate.
I don't know if every student would benefit from debating. I don't know that debate would be particularly easy to integrate in the classroom. I'm not going to argue that debate will magically transform our schools. But I know that some students can benefit from it. And that's enough for me.
Sorry for getting all sappy and advocate-y, but occasionally it's nice to take a break from reading about what doesn't work.
If you happen to teach in NYC, or know someone who does, you can check out the NY Urban Debate League run by the IMPACT Coalition.
An article in EdWeek yesterday, however, reminded me of one thing that does work. Debate. About the only good experience I had at my school was coaching the debate team. Now, granted, the team was made up of kids who volunteered to spend their time after-school and on Saturdays hanging out with me, and I had the power to remove anybody who didn't fully engage during that time period, but, nonetheless, it was clear that the kids I worked with gained something from it.
I saw very few examples of kids being excited to learn in my school, but these kids were. I saw very few examples of kids understanding complex issues in my school, but these kids did. Not because of anything that I did, but because debates are fun and because the only way to win a debate is to deeply understand all the issues surrounding a topic.
I have no idea if these kids made huge gains on their test results and, quite frankly, I don't care. I don't care what the regression analysis or t-tests show. I saw it in their eyes. I watched their arguments develop. I felt their excitement. I ceded when they begged me to make practices longer and more frequent. They gained knowledge about important topics. They gained confidence in themselves. They gained the ability to construct and defend an argument (and poke holes in those that others make). Whether or not they bubbled in more correct answers on the state test, they were better off as a result of participating in debate.
I don't know if every student would benefit from debating. I don't know that debate would be particularly easy to integrate in the classroom. I'm not going to argue that debate will magically transform our schools. But I know that some students can benefit from it. And that's enough for me.
Sorry for getting all sappy and advocate-y, but occasionally it's nice to take a break from reading about what doesn't work.
If you happen to teach in NYC, or know someone who does, you can check out the NY Urban Debate League run by the IMPACT Coalition.
Saturday, April 12, 2008
Standardized Tests and Tenure
Earlier this week, New York passed a law banning the use of standardized test results when making decisions on teacher tenure. The little afterthought of a law that was stuck in a big budget bill has set the education blogosphere aflame. It contains a piece of almost all of the major issues of the day: unions, standardized tests, teacher quality, and the intervention of politicians.
Here's a recap of how various people reacted to the decision:
Joel Klein: The sky is falling, and it's all the union's fault
Randi Weingarten: We don't need no stinkin' standardized tests
The NY Times: It's not a good idea, and it wasted valuable time
The Quick and The Ed: It's bad . . . and anyone who argues it's not is stupid
Education Notes: It's good . . . and nobody is honestly refuting my argument
The Socratic Method: It's both unnecessary and stupid
Eduwonkette: People are overreacting
Sherman Dorn: Let's not forget that it's a moratorium, not a ban
My guess is that the way people view this law is largely the same as the way the view unions. If they don't like unions, they don't like the law; and vice-versa. And that's really a shame, because it's the sort of polarizing pseudo-argument that overly-partisan politicians in Washington use to paralyze our country.
The reality, in this situation, (as it always seems to be) is somewhere in between. The idea that unions are either purely good or purely bad is pure nonsense. In this case, unions went out of their way to push a law that may or may not have been unnecessary, but probably won't really hurt anyone in the short-run, though it could prove harmful if it became permanent. In the meantime, it's possible that the state government would have been more productive if they'd been doing something else. What? Exactly. Let's break this down:
The Means: Unions clearly used the back door route to pressure politicians to tuck what they wanted in an unrelated large bill. I can't fault anybody who doesn't like the way they did it -- they're not exactly setting a stellar example here.
The Ends: Districts aren't allowed to use data from state tests to make decisions on teacher tenure for the next two years. This might be a bit extreme, but I see little evidence that the data would have radically transformed decision-making. The last time I checked, the state English test was given in Mid-January and the state Math test was given in Mid-March (approximately 55% and 75% of the way through the school year). Furthermore, I'm not sure that the tests are designed to be used as value-added assessments (meaning that you can compare the results from one year to those from the previous year to see how much a student learned) as they are in TN and some other states. Throw in the fact that the majority of teachers do not teach 3rd-8th grade Math or English, and you have a set of information that is far from perfect. Maybe two years from now they'll have a better system, but right now I don't see any compelling reason to believe that a district or principal gains much from using the data or loses much from not using the data. Meanwhile, the possibility that inaccurate data is used to decide whether a teacher stays or goes is eliminated.
The Context: How much of an effect the law has over the next two years largely depends on how many principals and districts were planning on using the data from state tests to make decisions on tenure over the next two years. If nobody was planning on doing it, then the law was a gigantic waste of time for everybody involved. If a lot of people were moving in that direction, then the state just intervened in a heavy-handed way that might result in fairer tenure decisions over the next two years.
So, in the end, these are really the questions we should be answering:
Here's a recap of how various people reacted to the decision:
Joel Klein: The sky is falling, and it's all the union's fault
Randi Weingarten: We don't need no stinkin' standardized tests
The NY Times: It's not a good idea, and it wasted valuable time
The Quick and The Ed: It's bad . . . and anyone who argues it's not is stupid
Education Notes: It's good . . . and nobody is honestly refuting my argument
The Socratic Method: It's both unnecessary and stupid
Eduwonkette: People are overreacting
Sherman Dorn: Let's not forget that it's a moratorium, not a ban
My guess is that the way people view this law is largely the same as the way the view unions. If they don't like unions, they don't like the law; and vice-versa. And that's really a shame, because it's the sort of polarizing pseudo-argument that overly-partisan politicians in Washington use to paralyze our country.
The reality, in this situation, (as it always seems to be) is somewhere in between. The idea that unions are either purely good or purely bad is pure nonsense. In this case, unions went out of their way to push a law that may or may not have been unnecessary, but probably won't really hurt anyone in the short-run, though it could prove harmful if it became permanent. In the meantime, it's possible that the state government would have been more productive if they'd been doing something else. What? Exactly. Let's break this down:
The Means: Unions clearly used the back door route to pressure politicians to tuck what they wanted in an unrelated large bill. I can't fault anybody who doesn't like the way they did it -- they're not exactly setting a stellar example here.
The Ends: Districts aren't allowed to use data from state tests to make decisions on teacher tenure for the next two years. This might be a bit extreme, but I see little evidence that the data would have radically transformed decision-making. The last time I checked, the state English test was given in Mid-January and the state Math test was given in Mid-March (approximately 55% and 75% of the way through the school year). Furthermore, I'm not sure that the tests are designed to be used as value-added assessments (meaning that you can compare the results from one year to those from the previous year to see how much a student learned) as they are in TN and some other states. Throw in the fact that the majority of teachers do not teach 3rd-8th grade Math or English, and you have a set of information that is far from perfect. Maybe two years from now they'll have a better system, but right now I don't see any compelling reason to believe that a district or principal gains much from using the data or loses much from not using the data. Meanwhile, the possibility that inaccurate data is used to decide whether a teacher stays or goes is eliminated.
The Context: How much of an effect the law has over the next two years largely depends on how many principals and districts were planning on using the data from state tests to make decisions on tenure over the next two years. If nobody was planning on doing it, then the law was a gigantic waste of time for everybody involved. If a lot of people were moving in that direction, then the state just intervened in a heavy-handed way that might result in fairer tenure decisions over the next two years.
So, in the end, these are really the questions we should be answering:
- Did the ends justify the means?
- How can we obtain better information on teacher performance than is currently available?
- Is it possible for all of us to say the word "union" and remain rational?
Buying Curricula
Ever thought that colleges were missing a fundamental text or course when you looked at what they taught? Ever tried to convince them to add your pet project? Well, here's some advice: donate a million dollars and try again.
This piece (hat tip: Paul Krugman) discusses the recent actions of a charitable arm of BB&T banking corp. I don't know the full story, but is sounds like they essentially exchanged considerable amounts of money for promises from various colleges to teach Ayn Rand books. Maybe I have too much faith in humanity, but I have a hard time believing that it could be this simple. How could any college possibly justify such an outlandishly egregious breach of ethics?
The article says that colleges have started tailoring grant requests to the organization by mentioning Ayn Rand, so it hardly seems that this is one evil corporation trying to influence some financially-strapped colleges. Is this really where the fundraising race will lead -- colleges putting their curricula up for sale?
This piece (hat tip: Paul Krugman) discusses the recent actions of a charitable arm of BB&T banking corp. I don't know the full story, but is sounds like they essentially exchanged considerable amounts of money for promises from various colleges to teach Ayn Rand books. Maybe I have too much faith in humanity, but I have a hard time believing that it could be this simple. How could any college possibly justify such an outlandishly egregious breach of ethics?
The article says that colleges have started tailoring grant requests to the organization by mentioning Ayn Rand, so it hardly seems that this is one evil corporation trying to influence some financially-strapped colleges. Is this really where the fundraising race will lead -- colleges putting their curricula up for sale?
Friday, April 11, 2008
A Day Back in the Trenches
Well, ok, it was really only half a day. I haven't been inside schools and classrooms nearly enough the past couple years, so when an acquaintance offered to bring me to their semi-dysfunctional urban high school, I jumped (ok, dragged myself out of bed) at the chance. Here are some things I noticed:
The day started with the chair of the department circulating the school to find volunteers. An e-mail had just been sent out notifying them that 2 teachers from each school must attend a district-wide PD session on Monday.
First period was spent signing in late kids. They all calmly walked over, got their detention slip, and moved on. Some grunted when spoken to; most just ignored the adult that was trying to speak to them. Those that had a note from a parent or were seen getting out of a car were excused from detention. A steady stream of students poured in all period, virtually none of whom seemed to give a damn about detention.
About a third of the students in the first class failed to bring a pencil and about half made no attempt at doing any work. Students were hostile and rude when approached, but were also surprisingly calm. The parade of uninterested and unprepared zombie-kids continued for most of the morning.
The final period I witnessed had some kids who were a bit more lively. One student walked in and sat down without his book or folder. The teacher asked him nicely to get his book. He let loose with a stream of profanities and marched into the hallway. He did not come back. 15 minutes into the period a security guard guided a girl into the room. She proceeded to sit down and scream profanities at other students, who returned the favor. The teacher asked her to get a book. She marched angrily into the hall. Five minutes later she returned. This time the teacher has a book and folder out and ready for her and attempts to keep her away from the other volatile students in the class. More screaming ensued. I baby-sat her for the rest of period.
It reminded me a lot of my school, except that the kids were much calmer. But most refused to engage in any sort of meaningful discourse with an adult, and clearly resented it when an adult tried to strike up a conversation with them (yet alone ask them to do something). The kids walking in the door, walking through the hallway, and walking into the classroom all seemed to share the same expression; a mixture of indifference and hostility. They didn't care and didn't want to be bothered. I witnessed an experienced, intelligent, engaging teacher go out of their way to do whatever they could for the kids. Some responded. Most were uninterested in responding (again, unless asked directly to do something -- then they were no longer so calm). The classroom I was in, and others, were not spiraling out of control as kids went wild (as they did at my school) but were, instead, filled with students who fairly calmly refused to engage or try. Most classes were down from about 35 at the start of the year to 20 or even fewer now (which was probably part of the reason everybody was so calm) as a result of the high drop-out rate. I ate in a student-run cafeteria -- one of the few times I saw students willingly engage in their surroundings. It was a good refresher course in the real world -- we tend to overlook a lot of these things up in the ivory tower.
The day started with the chair of the department circulating the school to find volunteers. An e-mail had just been sent out notifying them that 2 teachers from each school must attend a district-wide PD session on Monday.
First period was spent signing in late kids. They all calmly walked over, got their detention slip, and moved on. Some grunted when spoken to; most just ignored the adult that was trying to speak to them. Those that had a note from a parent or were seen getting out of a car were excused from detention. A steady stream of students poured in all period, virtually none of whom seemed to give a damn about detention.
About a third of the students in the first class failed to bring a pencil and about half made no attempt at doing any work. Students were hostile and rude when approached, but were also surprisingly calm. The parade of uninterested and unprepared zombie-kids continued for most of the morning.
The final period I witnessed had some kids who were a bit more lively. One student walked in and sat down without his book or folder. The teacher asked him nicely to get his book. He let loose with a stream of profanities and marched into the hallway. He did not come back. 15 minutes into the period a security guard guided a girl into the room. She proceeded to sit down and scream profanities at other students, who returned the favor. The teacher asked her to get a book. She marched angrily into the hall. Five minutes later she returned. This time the teacher has a book and folder out and ready for her and attempts to keep her away from the other volatile students in the class. More screaming ensued. I baby-sat her for the rest of period.
It reminded me a lot of my school, except that the kids were much calmer. But most refused to engage in any sort of meaningful discourse with an adult, and clearly resented it when an adult tried to strike up a conversation with them (yet alone ask them to do something). The kids walking in the door, walking through the hallway, and walking into the classroom all seemed to share the same expression; a mixture of indifference and hostility. They didn't care and didn't want to be bothered. I witnessed an experienced, intelligent, engaging teacher go out of their way to do whatever they could for the kids. Some responded. Most were uninterested in responding (again, unless asked directly to do something -- then they were no longer so calm). The classroom I was in, and others, were not spiraling out of control as kids went wild (as they did at my school) but were, instead, filled with students who fairly calmly refused to engage or try. Most classes were down from about 35 at the start of the year to 20 or even fewer now (which was probably part of the reason everybody was so calm) as a result of the high drop-out rate. I ate in a student-run cafeteria -- one of the few times I saw students willingly engage in their surroundings. It was a good refresher course in the real world -- we tend to overlook a lot of these things up in the ivory tower.
Thursday, April 10, 2008
Even More on Teachers and Professionalism
When I posted a quick thought on teachers as professionals a few days ago I was planning on moving on to something else the next day. Then the comments started pouring (ok, trickling) in, and I decided to post a follow-up comment. But the comments kept coming. And now Marc Dean Millot has responded in his blog rather than adding to the pile of comments. I feel compelled to respond to his response and the other comments I've received.
Truthfully, the title "Are Teachers Professionals?" was a rather flippant and overly-general question for the top of a post in which I actually asked whether any workers in any prestigious field receive as many directives from above as teachers. I still haven't answered that question, but I'm leaning toward "no" right now.
Since I threw out the question initially, I might as well take a stab at answering it. Are teachers professionals? I don't know. On the one hand, they work in a "profession" that requires a fairly high level of training but, on the other, I don't think they're really treated like professionals by their superiors (for now I will skip the questions of why teachers are treated as they are or whether teachers act like professionals).
First, I'm not sure that there's really an accepted definition of "profession" or that, to some extent, any label matters all that much. Millot defines professionals as those who "owe a special duty of care" to their clients (by law), have the autonomy to provide it, and regulate their field. So, I agree, by his definition teachers wouldn't be professionals. But I'm not really sure that any of those three items define what it means to be a professional. Those seem like things that professionals often do rather than things that make one a professional. If we go with the dictionary definitions, professionals are simply people who have the specialized knowledge necessary to work in a field.
Furthermore, I'm not convinced that these are necessarily possible or feasible for teachers. How would one judge whether a teacher is meeting their professional obligations to students? An engineer fails if a bridge collapses. A doctor fails if they prescribe the wrong drugs or operate on the wrong foot. An accountant fails if they take an illegal deduction. When does a teacher fail? When a child fails to learn? What if the child refuses to try? Is the teacher negligent in that case? I'm not saying it's impossible to define, but it's a heck of a lot tougher than in other fields. Furthermore, I don't hear anybody arguing that professors aren't professionals and I don't see them carrying malpractice insurance or being sued for negligence. Should teachers leave the field if they neglect their job responsibilities? Yes. Would codifying their job responsibilities make them more professional? I remain unconvinced.
I'll offer less disagreement on the points about autonomy and self-policing. Unless somebody can convince me otherwise, I'd have to say that teachers do not have the autonomy of people in more prestigious fields. I'm not sure autonomy makes somebody a professional, but it's certainly a sign that somebody is one (e.g. "professional latitude"). And I'd say the same thing about policing the field. Obviously teachers don't bear much responsibility for determining who is allowed to enter and forced to leave their field. I'm not sure that this makes them unprofessional, but the "professions" do tend to police their fields (faculty are mostly in charge of hiring and tenure decisions, for example).
I'll also largely agree that there is a trade-off between accountability and autonomy. But it's not absolute (again, other than tenure, how accountable are professors for what they do?), and I'm not convinced that teachers are as unaccountable as one might think. It doesn't make the news when a lawyer posts risque photos online, but you'll hear about it if a teacher does. You won't hear about it if a doctor curses out a patient, but you will if a teacher curses out a student. You won't hear about it if a lawyer tells their client to pray with them, but you will if a teacher tells their class to pray with them. Despite the fact that teachers aren't treated like professionals in a lot of ways, they sure seem to be held to higher standards of conduct than maybe any other profession or field in this country.
Truthfully, the title "Are Teachers Professionals?" was a rather flippant and overly-general question for the top of a post in which I actually asked whether any workers in any prestigious field receive as many directives from above as teachers. I still haven't answered that question, but I'm leaning toward "no" right now.
Since I threw out the question initially, I might as well take a stab at answering it. Are teachers professionals? I don't know. On the one hand, they work in a "profession" that requires a fairly high level of training but, on the other, I don't think they're really treated like professionals by their superiors (for now I will skip the questions of why teachers are treated as they are or whether teachers act like professionals).
First, I'm not sure that there's really an accepted definition of "profession" or that, to some extent, any label matters all that much. Millot defines professionals as those who "owe a special duty of care" to their clients (by law), have the autonomy to provide it, and regulate their field. So, I agree, by his definition teachers wouldn't be professionals. But I'm not really sure that any of those three items define what it means to be a professional. Those seem like things that professionals often do rather than things that make one a professional. If we go with the dictionary definitions, professionals are simply people who have the specialized knowledge necessary to work in a field.
Furthermore, I'm not convinced that these are necessarily possible or feasible for teachers. How would one judge whether a teacher is meeting their professional obligations to students? An engineer fails if a bridge collapses. A doctor fails if they prescribe the wrong drugs or operate on the wrong foot. An accountant fails if they take an illegal deduction. When does a teacher fail? When a child fails to learn? What if the child refuses to try? Is the teacher negligent in that case? I'm not saying it's impossible to define, but it's a heck of a lot tougher than in other fields. Furthermore, I don't hear anybody arguing that professors aren't professionals and I don't see them carrying malpractice insurance or being sued for negligence. Should teachers leave the field if they neglect their job responsibilities? Yes. Would codifying their job responsibilities make them more professional? I remain unconvinced.
I'll offer less disagreement on the points about autonomy and self-policing. Unless somebody can convince me otherwise, I'd have to say that teachers do not have the autonomy of people in more prestigious fields. I'm not sure autonomy makes somebody a professional, but it's certainly a sign that somebody is one (e.g. "professional latitude"). And I'd say the same thing about policing the field. Obviously teachers don't bear much responsibility for determining who is allowed to enter and forced to leave their field. I'm not sure that this makes them unprofessional, but the "professions" do tend to police their fields (faculty are mostly in charge of hiring and tenure decisions, for example).
I'll also largely agree that there is a trade-off between accountability and autonomy. But it's not absolute (again, other than tenure, how accountable are professors for what they do?), and I'm not convinced that teachers are as unaccountable as one might think. It doesn't make the news when a lawyer posts risque photos online, but you'll hear about it if a teacher does. You won't hear about it if a doctor curses out a patient, but you will if a teacher curses out a student. You won't hear about it if a lawyer tells their client to pray with them, but you will if a teacher tells their class to pray with them. Despite the fact that teachers aren't treated like professionals in a lot of ways, they sure seem to be held to higher standards of conduct than maybe any other profession or field in this country.
Tuesday, April 8, 2008
Which Matters More: Race or Wealth?
I've wondered about this question for a while. I was under the impression that a rich white kid outperforms a rich black kid, but that a rich black kid outperforms a poor white kid. And then I heard a throwaway line in a documentary I saw in class last semester that said something to the effect of "if we control for wealth (including home equity) instead of income, the black-white gap disappears."
What? Really? Since then I've been trying to figure out where that information came from and if it were true or not. Finally, yesterday, a friend who's been working on a huge project on the achievement gap forwarded me an article (abstract) from the latest issue of the journal Child Development. In it, the authors use a large dataset to see if wealth mediates the achievement gap. Before I get to the findings, let me stop for a second and answer a question that might be in your head. Why would wealth matter more than income? In this study, Whites earn about twice as much as Blacks, but have about five the times the amount of wealth, on average. Think about it: one family that makes x dollars/year but has no money in the bank and rents an apt. will not have the same lifestyle as another family that makes x dollars/year but owns their own house and has a large portfolio of stocks and bonds.
So, what's the answer? Which one is more important? Well, it looks like . . . neither.
The authors have data on a lot more than just wealth and race, and when they include all their other variables neither race nor wealth are statistically significant predictors of achievement in most of their models (they have both reading and math scores for 3-5 and 6-12 year-olds). The variables that are most frequently statistically significant are parent's occupation and mother's cognitive ability. Depending on the model, parental education, grandparent's education, debt, stock ownership, and a variety of home environment measures are also statistically significant.
So, according to this dataset, there is no difference in achievement between blacks and whites who have parents with the same job, cognitive ability, and some other factors. I e-mailed the author to make sure I had this right before writing about it (I did). I also asked if race were still a statistically significant predictor when only controlling for wealth (I didn't see a table with only these variables in it) and I was told that it was.
So, there appear to be two main lessons out of this paper (for me, anyway).
What? Really? Since then I've been trying to figure out where that information came from and if it were true or not. Finally, yesterday, a friend who's been working on a huge project on the achievement gap forwarded me an article (abstract) from the latest issue of the journal Child Development. In it, the authors use a large dataset to see if wealth mediates the achievement gap. Before I get to the findings, let me stop for a second and answer a question that might be in your head. Why would wealth matter more than income? In this study, Whites earn about twice as much as Blacks, but have about five the times the amount of wealth, on average. Think about it: one family that makes x dollars/year but has no money in the bank and rents an apt. will not have the same lifestyle as another family that makes x dollars/year but owns their own house and has a large portfolio of stocks and bonds.
So, what's the answer? Which one is more important? Well, it looks like . . . neither.
The authors have data on a lot more than just wealth and race, and when they include all their other variables neither race nor wealth are statistically significant predictors of achievement in most of their models (they have both reading and math scores for 3-5 and 6-12 year-olds). The variables that are most frequently statistically significant are parent's occupation and mother's cognitive ability. Depending on the model, parental education, grandparent's education, debt, stock ownership, and a variety of home environment measures are also statistically significant.
So, according to this dataset, there is no difference in achievement between blacks and whites who have parents with the same job, cognitive ability, and some other factors. I e-mailed the author to make sure I had this right before writing about it (I did). I also asked if race were still a statistically significant predictor when only controlling for wealth (I didn't see a table with only these variables in it) and I was told that it was.
So, there appear to be two main lessons out of this paper (for me, anyway).
- Race, in and of itself, doesn't seem to determine achievement level. Now, maybe it influences what kind of job somebody can get (for example), and that influences achievement level -- but if a parent is able to get that job then their kids should score about the same as other kids of parents with similar jobs regardless of their race.
- Though a better predictor of achievement than income, wealth does not fully mediate the effects of race on children. In other words, white children whose parents own a certain amount of property, stocks, etc. will out achieve (on average, of course) black children whose parents have the same level of wealth.
Monday, April 7, 2008
More on Teachers and Professionalism
Thought Experiment:
Envision the perfect school in your head.
What are the teachers like? How do they behave? What responsibilities do they have? How are they treated? Are teachers professionals in this school?
In my last post I asked whether any professionals in high-prestige jobs were subjected to as many directives and interference as are teachers. I still don't have an answer (surprise, not every question can be answered in 24 hours). But I do have more questions (courtesy of commenters).
Whether teachers are treated like people in other professions is, at best, half the story. You'd also need to know why they're treated the way they are and if they behave like people in other professions before you could make firm conclusion about the state of teaching. Of course, short blog posts will never answer complicated questions. So rather than trying to answer all those, I still seek the answer to this one: what is the most prestigious job where people are treated similarly to teachers?
Envision the perfect school in your head.
What are the teachers like? How do they behave? What responsibilities do they have? How are they treated? Are teachers professionals in this school?
In my last post I asked whether any professionals in high-prestige jobs were subjected to as many directives and interference as are teachers. I still don't have an answer (surprise, not every question can be answered in 24 hours). But I do have more questions (courtesy of commenters).
Whether teachers are treated like people in other professions is, at best, half the story. You'd also need to know why they're treated the way they are and if they behave like people in other professions before you could make firm conclusion about the state of teaching. Of course, short blog posts will never answer complicated questions. So rather than trying to answer all those, I still seek the answer to this one: what is the most prestigious job where people are treated similarly to teachers?
Sunday, April 6, 2008
Are Teachers Professionals?
I've heard a lot of talk about teachers as professionals, or at least the desire for teachers to be professionals. I'm not sure many people would disagree that, in an ideal system, teachers would be at least close to on-par with doctors, lawyers, engineers, etc.
Teachers are compared to people who work in these professions, as well as nurses, accountants, social workers, etc. somewhat frequently in research -- sometimes this is appropriate and sometimes it's not.
Regardless, here's my question of the day:
What is the most prestigious job in which people are subjected to as many directives from above as teachers?
I'm not sure what the answer is. I'm honestly not sure how much lawyers or accountants have bosses breathing down their necks (and I'm sure it varies widely), so I hesitate to even hazard a guess. It certainly varies widely in teaching, as some teachers shut their door and do their own thing while others have principals and superintendents constantly ordering them around. Ultimately, teachers are supposed to follow the directives of their supervisors (after all, they can be cited for insubordination). The curriculum they must teach might be determined by central office folk (in NYC a few years back seating and bulletin board arrangements were suggested by them as well). They are usually evaluated by supervisors on at least a yearly basis. They're limited as to how they can deal with disruptive students by school and district policy. I could go on, but I won't.
Are teachers treated less like professionals than people in equally or more prestigious jobs? Do people in other jobs have more autonomy or professional latitude than teachers? Does this matter?
Teachers are compared to people who work in these professions, as well as nurses, accountants, social workers, etc. somewhat frequently in research -- sometimes this is appropriate and sometimes it's not.
Regardless, here's my question of the day:
What is the most prestigious job in which people are subjected to as many directives from above as teachers?
I'm not sure what the answer is. I'm honestly not sure how much lawyers or accountants have bosses breathing down their necks (and I'm sure it varies widely), so I hesitate to even hazard a guess. It certainly varies widely in teaching, as some teachers shut their door and do their own thing while others have principals and superintendents constantly ordering them around. Ultimately, teachers are supposed to follow the directives of their supervisors (after all, they can be cited for insubordination). The curriculum they must teach might be determined by central office folk (in NYC a few years back seating and bulletin board arrangements were suggested by them as well). They are usually evaluated by supervisors on at least a yearly basis. They're limited as to how they can deal with disruptive students by school and district policy. I could go on, but I won't.
Are teachers treated less like professionals than people in equally or more prestigious jobs? Do people in other jobs have more autonomy or professional latitude than teachers? Does this matter?
Saturday, April 5, 2008
More on Self-Control
I was considering posting a follow-up to the previous entry, and Sherman Dorn has convinced me that it was necessary (don't you hate it when somebody who knows more than you do about a topic points out all the holes in your analysis?). I make a strenuous effort to keep these posts short so that they're easier to read for those of us with short attention spans and busy schedules, but that often means something gets left out. In this case a lot got left out. And, to be frank, a lot will still be left out after this post.
Anyway, back to self-control. I mentioned David Brooks' op-ed in the last post. In it he recounts Walter Mischel's marshmallow experiment from about 30 years ago. To summarize: 4-year olds were left in an empty room with a marshmallow and had two options: 1) ring the bell and eat the marshmallow, or 2) wait for the adult to come back (without being called) and give them a 2nd marshmallow. In short, those with more self-control were able to sweat it out and receive double the marshmallows while those with less simply succumbed to temptation and ate the one that was there. So, why does this matter? The kids from the two groups (those that waited and those that rang) were tracked into adulthood and the kids who were able to wait got higher test-scored, attended more prestigious colleges, earned more money, etc.
So now we're back to where we were in the last post. If self-control is so important, why don't schools teach it more? Well, like seemingly everything in schools, there is a delicate balancing act involved. Imagine if your kid learned nothing but self-control and discipline in school (no math, reading, etc.) -- you'd be furious, and rightfully so. On the other hand, I'm pretty convinced that the kids with less self-control learn a lot less than those with more, so it seems like teaching some self-control could lead to greater learning.
I've been told (by somebody who attended 1st grade in China) that the lesson on the first day of 1st grade in China is how to sit properly in a chair -- and that kids are expected to go home and practice. One can imagine what a dreadful education this would be if that was the only lesson for a year, but might there be some value in spending one day on it?
Anyway, the point is that students need to learn both self-control and academic topics and that focusing too much on one and not enough on the other could prove detrimental.
I think it has a lot of similarities to discipline in school; some level of order is necessary to facilitate learning, but focusing solely on order means that students never learn any subject matter. You could look at character education the same way.
Can we integrate lessons in self-control with lessons in reading, math, etc.? If so (and I think the answer is "yes"), then I think we should. While psychologists and others have studied self-control, I almost never heard it mentioned among teachers or in teacher education and I almost never hear it now in the news or in policy circles. I'll be the first to admit that there's no easy solution, but I think self-control is too important to ignore.
Anyway, back to self-control. I mentioned David Brooks' op-ed in the last post. In it he recounts Walter Mischel's marshmallow experiment from about 30 years ago. To summarize: 4-year olds were left in an empty room with a marshmallow and had two options: 1) ring the bell and eat the marshmallow, or 2) wait for the adult to come back (without being called) and give them a 2nd marshmallow. In short, those with more self-control were able to sweat it out and receive double the marshmallows while those with less simply succumbed to temptation and ate the one that was there. So, why does this matter? The kids from the two groups (those that waited and those that rang) were tracked into adulthood and the kids who were able to wait got higher test-scored, attended more prestigious colleges, earned more money, etc.
So now we're back to where we were in the last post. If self-control is so important, why don't schools teach it more? Well, like seemingly everything in schools, there is a delicate balancing act involved. Imagine if your kid learned nothing but self-control and discipline in school (no math, reading, etc.) -- you'd be furious, and rightfully so. On the other hand, I'm pretty convinced that the kids with less self-control learn a lot less than those with more, so it seems like teaching some self-control could lead to greater learning.
I've been told (by somebody who attended 1st grade in China) that the lesson on the first day of 1st grade in China is how to sit properly in a chair -- and that kids are expected to go home and practice. One can imagine what a dreadful education this would be if that was the only lesson for a year, but might there be some value in spending one day on it?
Anyway, the point is that students need to learn both self-control and academic topics and that focusing too much on one and not enough on the other could prove detrimental.
I think it has a lot of similarities to discipline in school; some level of order is necessary to facilitate learning, but focusing solely on order means that students never learn any subject matter. You could look at character education the same way.
Can we integrate lessons in self-control with lessons in reading, math, etc.? If so (and I think the answer is "yes"), then I think we should. While psychologists and others have studied self-control, I almost never heard it mentioned among teachers or in teacher education and I almost never hear it now in the news or in policy circles. I'll be the first to admit that there's no easy solution, but I think self-control is too important to ignore.
Resisting Impulses and Delaying Gratification
A couple days ago a pair of scientists wrote an op-ed in the NY Times that concluded "consistently doing any activity that requires self-control seems to increase willpower — and the ability to resist impulses and delay gratification is highly associated with success in life."
I occasionally like to step back and re-examine the current structure of schooling rather than just finding tweaks that would make it better, and I have to say that my reaction to the final sentence was "if this is highly associated with success in life, then is this what we should be teaching in school?"
Now, to be fair, I was already pre-disposed to answering "yes" to that question. Before starting teaching I was pretty sure I knew what was wrong with schools and how to fix them. I quickly discovered that I was wrong, and I discovered a few things that I never would have named as problems. One of these, and maybe the most important one, was self-control. From my perspective, there was no greater difference between the students that succeeded and those who floundered than the amount of self-control that they exhibited.
Near the end of my second year of teaching David Brooks wrote this op-ed in which he comes to a similar conclusion. Maybe I'm just looking the wrong places, but so far I can't disagree with his statement that self-control is "largely ignored by educators and policy makers." Indeed, I've only seen it mentioned in one study since that I can remember I started grad school.
I'm not really sure what the research base is on the linkage between self-control and "success" in life, but I see no reason not to believe people who know more than me when they say that there is a strong correlation. And I can't help but wonder if self-control is more important to teach than, say, an extra period of math.
I occasionally like to step back and re-examine the current structure of schooling rather than just finding tweaks that would make it better, and I have to say that my reaction to the final sentence was "if this is highly associated with success in life, then is this what we should be teaching in school?"
Now, to be fair, I was already pre-disposed to answering "yes" to that question. Before starting teaching I was pretty sure I knew what was wrong with schools and how to fix them. I quickly discovered that I was wrong, and I discovered a few things that I never would have named as problems. One of these, and maybe the most important one, was self-control. From my perspective, there was no greater difference between the students that succeeded and those who floundered than the amount of self-control that they exhibited.
Near the end of my second year of teaching David Brooks wrote this op-ed in which he comes to a similar conclusion. Maybe I'm just looking the wrong places, but so far I can't disagree with his statement that self-control is "largely ignored by educators and policy makers." Indeed, I've only seen it mentioned in one study since that I can remember I started grad school.
I'm not really sure what the research base is on the linkage between self-control and "success" in life, but I see no reason not to believe people who know more than me when they say that there is a strong correlation. And I can't help but wonder if self-control is more important to teach than, say, an extra period of math.
Friday, April 4, 2008
Where do we Draw the Line?
The Times has an interesting piece today about a school in upstate NY and their attempts to deal with discipline and teach kids responsibility. The school has, essentially, taken a no-nonsense approach: anybody who breaks any rule (or falters in any class) is barred from all activities and, in some cases, given detention or served lunch last. Students are assigned seats at lunch and not allowed to get up until given permission. Students have to walk to the right of the line in the middle of the hall (which the school where I taught used to do -- when I was there we just had the tape running down the middle of the hall and chaos all around it).
It would be easy to simply condemn this policy, but I hesitate for three reasons:
1. I read a short newspaper article about it; I don't really have a clear picture of what things are like on the ground
2. The new principal has coupled this no-nonsense approach with more fun activities in which the kids who aren't in trouble can participate
3. There has to be some structure in place to teach kids respect, responsibility, etc.
The strongest evidence against it is that about 1/4 of the students are currently on the no activities list. If that continues to be the case after a couple years of the policy then I think it would be fair to declare it a failure.
There's a fundamental question here that needs to be answered. How do we teach kids respect, responsibility, etc. without creating resentment and, at the same time, teaching self-control? Apparently lunchtime was a problem, so they clamped down. I'll assume that the students in the school are now more likely to view lunch as a controlled atmosphere rather than a time to goof off. But if they can't even get up when they want or sit with their friends, how are they supposed to learn to create such a controlled atmosphere on their own? (a conversation along similar lines has been taking place over at Bridging Differences).
In other words, it doesn't matter how much discipline can be imposed on the students in a school if they have no self-discipline outside of the system. But, at the same time, a system with no discipline does the kids a disservice. So, where do we draw the line? How do ensure the best of both worlds?
It would be easy to simply condemn this policy, but I hesitate for three reasons:
1. I read a short newspaper article about it; I don't really have a clear picture of what things are like on the ground
2. The new principal has coupled this no-nonsense approach with more fun activities in which the kids who aren't in trouble can participate
3. There has to be some structure in place to teach kids respect, responsibility, etc.
The strongest evidence against it is that about 1/4 of the students are currently on the no activities list. If that continues to be the case after a couple years of the policy then I think it would be fair to declare it a failure.
There's a fundamental question here that needs to be answered. How do we teach kids respect, responsibility, etc. without creating resentment and, at the same time, teaching self-control? Apparently lunchtime was a problem, so they clamped down. I'll assume that the students in the school are now more likely to view lunch as a controlled atmosphere rather than a time to goof off. But if they can't even get up when they want or sit with their friends, how are they supposed to learn to create such a controlled atmosphere on their own? (a conversation along similar lines has been taking place over at Bridging Differences).
In other words, it doesn't matter how much discipline can be imposed on the students in a school if they have no self-discipline outside of the system. But, at the same time, a system with no discipline does the kids a disservice. So, where do we draw the line? How do ensure the best of both worlds?
Wednesday, April 2, 2008
Who Should Evaluate Teachers?
A lot of ideas that make sense on paper (especially regarding teacher pay and teacher training) rest on being able to accurately (and fairly) evaluate teachers. This leads to two major questions: who should evaluate teachers and how should they do it?
The how is a more complicated question than I care to address right now, but I've frequently thought about the who last couple years, particularly given the problems I noticed with evaluations while I was teaching. The issue also came up in my last class -- where it was essentially batted around by a lot of former teachers. Based on previous thoughts, current practice, and recent discussion, these seem to be the options:
-An Administrative Supervisor (e.g. Principal or Asst. Principal)
-A Non-Administrative Supervisor (e.g. Dept. Chair)
-Other Teachers
-An Outside Expert (like in Martinique)
-A Mentor or Coach
-Some combination of the above
If you're a teacher (especially if there's a lot riding on the evaluation), you want to be evaluated by somebody who:
1.) Is an expert
2.) Knows you/your teaching well
3.) You can trust to be fair and impartial
The problem is that none of the options (save possibly the last one) meet all three of these criteria.
Administrative supervisors might meet all or none of these criteria depending on the situation. For example: a former English teacher who becomes a principal would not likely be an expert in teaching physics (and vice-versa). Whether your principal knows you well or is trustworthy depends on the circumstances.
I don't think there are many non-administrative supervisors. How many department chairs really count as supervisors? I'm sure exceptions exist, but the question then becomes what responsibilities these people have other than evaluating and if any of these are conflicts of interest.
I'm also not sure that I've ever heard of teachers bearing responsibility for evaluating other teachers (other than faculty making tenure decisions). I think there's some merit to this idea, but the largest problem is that it could negatively affect teacher communities, or at least make them artificial (e.g. people only say good things about themselves).
The outside expert could be both knowledgeable and impartial, but how well will they actually know the teacher?
The mentor or coach may best know your abilities, can they really be an effective source of guidance if they're also in charge of evaluating somebody (e.g. why would somebody come to them with their problems?).
Do we pick what we think is the best of these options, come up with something else, or combine these? Why not have an evaluation committee made up of some people from all walks of life? Imagine a committee made up of some teachers, administrators, and experts who were in charge of evaluating a number of teachers. Would this work? Would this be too cumbersome? Would it be worth it?
The how is a more complicated question than I care to address right now, but I've frequently thought about the who last couple years, particularly given the problems I noticed with evaluations while I was teaching. The issue also came up in my last class -- where it was essentially batted around by a lot of former teachers. Based on previous thoughts, current practice, and recent discussion, these seem to be the options:
-An Administrative Supervisor (e.g. Principal or Asst. Principal)
-A Non-Administrative Supervisor (e.g. Dept. Chair)
-Other Teachers
-An Outside Expert (like in Martinique)
-A Mentor or Coach
-Some combination of the above
If you're a teacher (especially if there's a lot riding on the evaluation), you want to be evaluated by somebody who:
1.) Is an expert
2.) Knows you/your teaching well
3.) You can trust to be fair and impartial
The problem is that none of the options (save possibly the last one) meet all three of these criteria.
Administrative supervisors might meet all or none of these criteria depending on the situation. For example: a former English teacher who becomes a principal would not likely be an expert in teaching physics (and vice-versa). Whether your principal knows you well or is trustworthy depends on the circumstances.
I don't think there are many non-administrative supervisors. How many department chairs really count as supervisors? I'm sure exceptions exist, but the question then becomes what responsibilities these people have other than evaluating and if any of these are conflicts of interest.
I'm also not sure that I've ever heard of teachers bearing responsibility for evaluating other teachers (other than faculty making tenure decisions). I think there's some merit to this idea, but the largest problem is that it could negatively affect teacher communities, or at least make them artificial (e.g. people only say good things about themselves).
The outside expert could be both knowledgeable and impartial, but how well will they actually know the teacher?
The mentor or coach may best know your abilities, can they really be an effective source of guidance if they're also in charge of evaluating somebody (e.g. why would somebody come to them with their problems?).
Do we pick what we think is the best of these options, come up with something else, or combine these? Why not have an evaluation committee made up of some people from all walks of life? Imagine a committee made up of some teachers, administrators, and experts who were in charge of evaluating a number of teachers. Would this work? Would this be too cumbersome? Would it be worth it?
Tuesday, April 1, 2008
How Good are Teach for America Teachers?
This was the question asked by a team of researchers at the Urban Institute. I happened to see the article presented at AERA, and have since read the full version.
A number of previous studies have found that TFA members were somewhere between just as good and slightly better at teaching math and between just as good and slightly worse at teaching English as regular teachers (based on test scores). One of the differences between this paper and others was that it included only high schoolers. To make a long story short, the authors found that students taught by TFA teachers slightly outperformed students taught by regular teachers -- even experienced ones. I previously alluded to coverage of this paper here.
I debated whether to discuss this further, but I've now seen discussion of it here, here, and here.
Everyody except for eduwonkette seems to just accept the findings and move on. The study was well done in a lot of ways, so I have no desire to trash it, but I've found the coverage really lacking. As with any study, there were significant weaknesses. Among these:
-there was no definitive way to match students to teachers -- they were pretty sure they had it right for 84% of the students and threw out the rest
-they only had one test per subject for each student, meaning they couldn't measure their growth over a year
-they compare TFA teachers to all teachers despite the fact that they teach very different students in very different schools -- I'd argue that they're doing jobs that aren't really comparable
-the number of TFA teachers was very small -- a total of 69 over the years, meaning that it was probably about 30-50 actual people
So what does this prove? It seems like a pretty good bet that TFA teachers are outperforming other high school teachers in North Carolina, but it's not a sure thing. Even less of a sure thing is if this is true in other states, grades, and subjects. Conclusion #1: saying that this study found that TFA teachers are better than others and leaving it at that is misleading at best. Conclusion #2: it almost always takes a number of similar studies to prove anything.
update: more write-ups here and here, and eduwonkette is still the only one to point out any limitations of the study
A number of previous studies have found that TFA members were somewhere between just as good and slightly better at teaching math and between just as good and slightly worse at teaching English as regular teachers (based on test scores). One of the differences between this paper and others was that it included only high schoolers. To make a long story short, the authors found that students taught by TFA teachers slightly outperformed students taught by regular teachers -- even experienced ones. I previously alluded to coverage of this paper here.
I debated whether to discuss this further, but I've now seen discussion of it here, here, and here.
Everyody except for eduwonkette seems to just accept the findings and move on. The study was well done in a lot of ways, so I have no desire to trash it, but I've found the coverage really lacking. As with any study, there were significant weaknesses. Among these:
-there was no definitive way to match students to teachers -- they were pretty sure they had it right for 84% of the students and threw out the rest
-they only had one test per subject for each student, meaning they couldn't measure their growth over a year
-they compare TFA teachers to all teachers despite the fact that they teach very different students in very different schools -- I'd argue that they're doing jobs that aren't really comparable
-the number of TFA teachers was very small -- a total of 69 over the years, meaning that it was probably about 30-50 actual people
So what does this prove? It seems like a pretty good bet that TFA teachers are outperforming other high school teachers in North Carolina, but it's not a sure thing. Even less of a sure thing is if this is true in other states, grades, and subjects. Conclusion #1: saying that this study found that TFA teachers are better than others and leaving it at that is misleading at best. Conclusion #2: it almost always takes a number of similar studies to prove anything.
update: more write-ups here and here, and eduwonkette is still the only one to point out any limitations of the study
Measuring Graduation Rates
High school graduation rates always come up in the news. People are worried about them. People decry how low they are. People celebrate when they improve. Short of test scores, graduation rates are probably the most used indicator of educational success/failure in this country.
It makes sense to worry about graduation rates, but they're far from perfect. Two of the biggest problems are that it's almost impossible to calculate an exact rate for a school and that everybody calculates graduation rates differently.
I first learned about this a couple years ago when some folks from RAND did a study for Pittsburgh on their graduation rates. All calculations of graduation rates start with the same statistic: the number of new freshman who enroll in a given year. But how do we determine how many of these people have graduated and what the graduation rate should be? Do we check on them after four years? 5? 6? What about students who move out of the district? What about students who move into the district or otherwise enroll after the start of 9th grade? What about people who just stop showing up -- how do we know if they dropped out, moved away, or something else? Can we tell which students are in the 9th grade for the first time and which ones are repeating?
A large part of the difficulty is that there is no national database of students. If a student leaves and enrolls in a different city or state, there's no firm way of knowing that they're still enrolled. Similarly, if a student simply stops showing up at school rather than declaring themselves a dropout it's hard to tell if they've dropped out or enrolled in another district.
Anyway, the point is that it's not possible to compute an exact graduation rate. And the fact that every state and district seem to use different formulas means that it's hard to compare these inexact numbers. But, apparently, the Dept. of Ed. is going to step in and standardize the way that graduation rates are calculated. An article in the NY Times today says that the details of the new formula have not been released, but that all states will be required to use the same one.
The technicalities of which formula a state uses seems so trivial and boring that most people probably don't really give it much thought. But the article details people reacting quite strongly to the announcement. This makes sense because although the change is small, the effects of the change might be huge. The federal govt. is ostensibly holding states accountable for graduation rates, but they all calculate them differently (a recent change in North Carolina's formula led the official rate to drop from 95% to 68%). Sometimes the smallest changes are the ones that matter most.
It makes sense to worry about graduation rates, but they're far from perfect. Two of the biggest problems are that it's almost impossible to calculate an exact rate for a school and that everybody calculates graduation rates differently.
I first learned about this a couple years ago when some folks from RAND did a study for Pittsburgh on their graduation rates. All calculations of graduation rates start with the same statistic: the number of new freshman who enroll in a given year. But how do we determine how many of these people have graduated and what the graduation rate should be? Do we check on them after four years? 5? 6? What about students who move out of the district? What about students who move into the district or otherwise enroll after the start of 9th grade? What about people who just stop showing up -- how do we know if they dropped out, moved away, or something else? Can we tell which students are in the 9th grade for the first time and which ones are repeating?
A large part of the difficulty is that there is no national database of students. If a student leaves and enrolls in a different city or state, there's no firm way of knowing that they're still enrolled. Similarly, if a student simply stops showing up at school rather than declaring themselves a dropout it's hard to tell if they've dropped out or enrolled in another district.
Anyway, the point is that it's not possible to compute an exact graduation rate. And the fact that every state and district seem to use different formulas means that it's hard to compare these inexact numbers. But, apparently, the Dept. of Ed. is going to step in and standardize the way that graduation rates are calculated. An article in the NY Times today says that the details of the new formula have not been released, but that all states will be required to use the same one.
The technicalities of which formula a state uses seems so trivial and boring that most people probably don't really give it much thought. But the article details people reacting quite strongly to the announcement. This makes sense because although the change is small, the effects of the change might be huge. The federal govt. is ostensibly holding states accountable for graduation rates, but they all calculate them differently (a recent change in North Carolina's formula led the official rate to drop from 95% to 68%). Sometimes the smallest changes are the ones that matter most.