I'll Pass the AP Test . . . For the Right Price

The biggest education story in the press today (NY Times) the results of the second year of a pilot program that pays students in NYC up to $1,000 for passing an AP test. I'm not quite sure of the exact details of the scheme b/c different outlets are reporting different details. The other NY papers (Post, Daily News) are reporting $500 for a 3, $750 for a 4, and $1,000 for a 5, but the Times is reporting $1,000 for a 5 if students attend Saturday prep classes and $500 if they don't and then $750/$400 for a 4. The Times doesn't mention any reward for a 3 even though say it takes a 3 to pass. The Reach website confirms the amounts reported by the Times and reports that a 3 earns test takers $500/$300.

At any rate, more students both took and passed AP tests than last year -- when more students took, but fewer passed, the tests. The passing rate only budged from 32% to 33% (still down from 35% before the program started), so the main effect may have been encouraging more students to take tests. As far as I can tell, the number of exams taken and passed were:

2007: 4,275 taken/1,481 passed by ? students (34.6%) ? per student
2008: 4,620 taken/1,476 passed by 1,161 students (31.9%) 4.0 taken/1.27 passed per student
2009: 5,436 taken/1,774 passed by 1,240 students (32.5%) 4.39 taken/1.43 passed per student

(2007 data from this article)

In order to tell exactly what effect the program had we need to know, among other things, how many students were in these 31 schools in each of the three years, how many (if any) students re-took exams that they'd failed in a previous year, and whether there were any demographic changes among the school populations and test takers. Not to mention how many students took the tests in 2007, before the program started (I'll keep looking for that figure).

My first reaction to the news was "of course more students passed, I'd have taken and passed more AP tests in high school if I got $1,000 for each one," but then I read more of the details. Without knowing how many total students are in the schools, the results don't seem very impressive. I'd expect the possibility of earning thousands of dollars to yield more of a reaction from high school students. Also note that, nationwide, AP tests have about a 57% pass rate.

For a second, though, let's assume a best-case scenario for the program -- school populations shrunk, and the growth in test-taking was due to weaker and younger students signing up. In other words, let's assume that the numbers actually tell us that the rewards led more students to take and pass AP exams. Even then, I'm not really sure what to think.

The program is doling out nearly $1,000,000 in bonuses per year to students at only 31 schools. And I'm not sure exactly what it means to take or pass an AP test. I don't think most would recognize that as a goal within itself. I think we really need to know the secondary effects of the trial: do students view AP exams and/or school more positively? Do students study more for non-AP subjects? Are students more likely to enroll in college? Are students more likely to complete college now that they have a head start on earning credits? These seem like the real goals of the program.

In other words, the numbers tell us almost nothing other than that the program seems unlikely to have fomented a revolution within these schools. But to actually know what the results are we need to let it play out for a couple more years and, more importantly, answer the questions above.

update: I've been told three interesting pieces of information:

1.) The enrollment for the 31 schools is about 44,000 students

2.) Students were paid for passing the Spanish Literature AP exam in 2008, but only underclassmen were paid for doing so in 2009

3.) Virtually all of the increase in tests taken/passed is black/latino students