What Should We Learn from Promise Academy? Part 1

Note: In lieu of a Sunday Commentary, I will be running a multi-part discussion of the recent controversy surrounding the Promise Academy. What follows in part 1.

Last week, I wrote that David Brooks' interpretation of the results reported from Harlem's Promise Academy was "flat-out irresponsible." But I didn't discuss much exactly what we should learn from the results of Dobbie and Fryer's working paper.

Before I begin, I want to reiterate what Aaron Pallas wrote last week -- that it's too early to tell whether results from Promise Academy truly do show some sort of "miracle." Though results on the NY state test seem fantastic for 8th grade students in math, they're significantly less so (though still quite good) in reading and in 7th grade math. And the results on the Iowa Test of Basic Skills are substantially less impressive still. Are good things happening at Promise Academy? Probably? Are miracles being worked? We don't know yet.

But let's assume for a second that something very good is, indeed, happening at Promise Academy. If this is the case, it stands to reason that we might want to learn from what they're doing and maybe even replicate these results. So let's take a look at a few aspects of Promise Academy as described in Paul Tough's book and how they relate to Dobbie and Fryer's analysis and see if we can figure out what might be leading to their success. Here


high rates of spending
By my calculation, the Promise Academy spent $18,073 per pupil in 2006-07 and $15,330 per pupil in 2007-08. I e-mailed the Harlem Children's Zone to see what all this entails, but have yet to receive a response. My best guess is that this does not include the after-school and Saturday tutoring programs and possibly some other programs as well. As Ken Hirsch reported, the school has raised about an additional $5,000 per student each of the past two years.

Dobbie and Fryer write that "The schools provide free medical, dental and mental-health services (students are screened upon entry and receive regular check-ups), student incentives for achievement (money, trips to France, e.g.), high-quality, nutritious, cafeteria meals, support for parents in the form of food baskets, meals, bus fare, and so forth, and less tangible benefits such as the support of a committed staff" (p. 7).


extended school day/year
Students start arriving at 7am and many don't leave until dinner time, not to mention that many come in on Saturdays as well. The school runs 11 months of the year. I've read different estimates that students spend between 50 and 100% more time in school than does the average student.


integration with the community
The Promise Academy is not only part of a designated 97 block area known as the Harlem Children's Zone, but Geoffrey Canada's stated goal is for all of the programs in the area to lead to a type of positive contamination across the neighborhood. In this way, he wants the neighborhood to influence the school and vice-versa.

Tough writes that "the motivational strategies used by schools like KIPP's, Canada knew, often had the effect of establishing a separation between the KIPP kids and the other kids in the low-income areas where they lived . . . KIPP students often became isolated from their community . . . And this situation -- a blighted neighborhood producing a select group of high achieving kids who manage to accomplish great things and succeed beyond their peers -- was exactly the one Canada was trying to avoid when he set up the Harlem Children's Zone. If Canada's model was one of contamination, in which positive ideas and practices spread within a family and throughout a neighborhood, the KIPP model sometimes seemed by contrast to be one of quarantine, walling off the most promising kids from a sick neighborhood's contagion" (p. 162-163).


small class sizes
According to the school's report card, class sizes in 8th grade have averaged around 18 students each of the last two years. The average class size appears to be about 27 students per class for 8th grade general education classes (here's all the data on class sizes).


focus on test-prep
Tough writes that "The students who were furthest behind had been assigned to remedial classes three mornings a week, from 7:00 to 7:45 A.M. Then, for more advanced students . . . there was a separate class on Saturday mornings, from 9:00 to 11:00 A.M." (p. 137).

This actually only describes the first year of operation -- test-prep became increasingly intense with each passing year. He writes that while test-prep was intense for a couple months in the spring the first year that it was running full-speed by mid-September in the second year.

expulsion
Some charter schools find ways to get rid of kids -- for example, by suggesting that they will fit in better elsewhere or threatening to hold them back if they remain. I think it's fair to ask about the expulsion tactics of any charter school before comparing its results to those of traditional public schools. In the case of Promise Academy, the record is decidedly mixed.

The book emphasizes time and time again that Canada very much wants to educate every kid in the neighborhood, especially those who are the most resistant or the lowest achieving -- indeed, he seems to be the only one at times who doesn't argue in favor of getting rid of a group of "bad apples" the principal thinks is ruining the school.

At the same time, one of these "bad apples" is expelled three weeks after the group receives a stern talking-to (p. 183). Perhaps more importantly, the notion that kids who don't want to behave can choose to go to their neighborhood instead if they don't want to put forth the effort, or will be expelled if they don't cooperate, is definitely pressed at points by the staff. At the beginning of the third year of school (fall 2006) the new principal tells the students on the first day of school that "if you find by the end of this week that you're not prepared to do what we're asking you to do, it's very important that you let someone know so we can make other arrangements for you, because we have a very long waiting list of students who would love to be here at the Promise Academy." And the new Dean follows that up by saying "If you would rather go to public school, that is really your option . . . if you decide to stay, you better recognize: This is not the place you once thought it was . . . we are going to the top this year, with or without you" (pp. 178-179). And perhaps most importantly, they essentially expel the entire 8th grade class in the spring of 2007.

While the cohort sizes dwindle noticeably (the first two go from about 100 in sixth grade to 68 and 81 by eigth grade), it's unclear how much of this attrition is voluntary. Dobbie and Fryer argue that student stability is about equal to what it is at other schools. Perhaps the bigger factor is that students who leave don't seem to be replaced by new students.

a no-nonense approach to schooling
Particularly with the installation of a new principal in year 3, the school seems to make a concerted effort to run a tight ship -- especially given that all involved seem to agree that discipline problems were the largest hindrance in the first two years. Tough describes some tough rhetoric from the Principal and Dean but doesn't go into detail regarding the extent to which this strategy permeates the daily workings of the school.

personal connections
It seems only natural that a school with dedicated staff working long hours with small groups of students could achieve deeper levels of personal connections than the average school. Consider this passage in Tough's book:

"'Why should I care about this test?' [the child] demanded. 'No one cares how I do on this test. I don't care, either.' 'But I care how you do,' [the teacher] replied. And with those words, tears sprang to the boy's eyes and started running down his face. 'Why do you care?' he asked. 'Because this is your future, and I care very deeply about you.' It was the X factor, the magic ingredient that could outweigh all the careful calculations behind Promise Academy's stratgey for success: on top of the hours and hours of cognitive training, what made the difference in many students' lives was a personal connection that was impossible to measure and difficult to replicate" (p. 186).


Coming later this week: the conclusion of this list, some analysis of the Dobbie/Fryer paper, discussion of what others have written on the topic, and some thoughts to tie everything together.