Learning First Alliance

Strengthening public schools for every child

What Really Happens When We Pay People for Test Scores?

vonzastrowc's picture

Paying anyone--students or teachers--for test scores might be a bad idea. That's one of the big lessons I draw from Roland Fryer's now famous study of programs that pay students for good behavior, hard work or test results.

In fact, I think the implications of Fryer's study reach farther than that. The study offers a glimpse of how dangerous it could be to attach any big consequences--good or bad--to test scores alone. Here are some of the things I took away from Fryer's report:

We ignore inputs at our peril. It has become received wisdom that outcomes--and that usually means test scores--are all that really matter in school reform. But Fryer's study suggests that people in schools who call for more attention to inputs and the processes of arriving at outcomes aren't just whiners after all. The study found that cash rewards for certain behaviors--like reading more books--were more effective than cash for test scores. In fact, cash for scores seemed to have no effect on student achievement. Why? Incentives to do the right things, the things that promote learning, might well work better than incentives to do well on a test.

Getting kids too focused on their test scores may do them little good--and may even harm them--in the long run. Fryer's team noted that students getting cash for scores naturally grasped at test-taking strategies rather than, say, better study skills or deeper engagement in class materials:

Students [who were asked what they could do to earn more money on the next test] stated [sic.] thinking about test-taking strategies rather than salient inputs into the education production function or improving their general understanding of a subject area.... Not a single student mentioned reading the textbook, studying harder, completing their homework, or asking teachers or other adults about confusing topics.

That lends some credence to those who call schools under NCLB "test prep factories." The students Fryer's team observed seem to be the natural products of a culture that focuses on "outcomes" at the expense of everything else.

Tying cash to test outcomes may blunt higher-order thinking skills. Fryer's team found evidence that students did not want tests to move them outside their comfort zones. The study describes some students' responses to New York's "incentive tests":

These exams can quickly move students outside their comfort zone and into material that was not covered in class-especially if they are answering questions correctly. The qualitative team noted several instances in which students complained to their teachers when they were taken aback by questions asked on the exams or surprised by their test results.

The students may have felt that higher-order thinking skills just weren't part of their bargain.

Forces outside of schools may drag down students' test results and challenge even the best incentive programs. Fryer's study notes that poorer students didn't get as much out of even the best programs. They got less bang for the school's buck. Fryer speculates about this finding:

The best evidence in favor of the importance of complements in production are the differences between students who were not eligible for free lunch (and likely have intact, more educated famliies who are more engaged in their schooling) and those who are eligible for free lunch, though the differences are not statistically significant. The social ills that are correlated with eligibility for free lunch may be important limitations in production of achievement.

These comments might send a chill down the spines of teachers whose evaluations and pay will hinge mostly on their students' test scores. Just one less reason to teach low-income children, I guess.

Tying teachers' pay to test scores might not be such a great idea after all. Fryer comes right out and says as much:

It might be less effective to give teachers incentive pay based on outputs (test scores of their students) relative to inputs (staying after school to tutor their students.)

Coercing or coaxing teachers to raise test scores may have little or no impact. It may even cause harm. But giving them more support to do the kinds of things that are likely to improve student learning might be a much better approach.

In its recent review of Fryer's study, Time magazine puts the issue this way: We tend to assume that kids (and adults) know how to achieve success. If they don't get there, it's for lack of effort--or talent. Sometimes that's true. But a lot of the time, people are just flying blind." Fryer suggests that pay tied to outcomes does little to remove the blindfold.


Despite what some have said about him, Fryer is no ideologue. He is cautious about his conclusions, and he would never present any of the speculations I've elaborated on as ironclad. The evidence is still thin. His interpretations need more research.

But some of the things he's hinting at are anathema to the hardest of hard-core people in the merit pay camp. As states like Florida prepare to bet the farm on teacher pay and evaluation programs tied mostly to test scores, they should pay close attention to what Fryer is writing between the lines.

(For another response to Fryer's study, see Larry Ferlazzo. Ferlazzo is very concerned about the long-term impact of cash rewards for students, and he offers a number of other ways he and his colleagues have been inspiring higher performance.)