What Really Happens When We Pay People for Test Scores?
Paying anyone--students or teachers--for test scores might be a bad idea. That's one of the big lessons I draw from Roland Fryer's now famous study of programs that pay students for good behavior, hard work or test results.
In fact, I think the implications of Fryer's study reach farther than that. The study offers a glimpse of how dangerous it could be to attach any big consequences--good or bad--to test scores alone. Here are some of the things I took away from Fryer's report:
We ignore inputs at our peril. It has become received wisdom that outcomes--and that usually means test scores--are all that really matter in school reform. But Fryer's study suggests that people in schools who call for more attention to inputs and the processes of arriving at outcomes aren't just whiners after all. The study found that cash rewards for certain behaviors--like reading more books--were more effective than cash for test scores. In fact, cash for scores seemed to have no effect on student achievement. Why? Incentives to do the right things, the things that promote learning, might well work better than incentives to do well on a test.
Getting kids too focused on their test scores may do them little good--and may even harm them--in the long run. Fryer's team noted that students getting cash for scores naturally grasped at test-taking strategies rather than, say, better study skills or deeper engagement in class materials:
Students [who were asked what they could do to earn more money on the next test] stated [sic.] thinking about test-taking strategies rather than salient inputs into the education production function or improving their general understanding of a subject area.... Not a single student mentioned reading the textbook, studying harder, completing their homework, or asking teachers or other adults about confusing topics.
That lends some credence to those who call schools under NCLB "test prep factories." The students Fryer's team observed seem to be the natural products of a culture that focuses on "outcomes" at the expense of everything else.
Tying cash to test outcomes may blunt higher-order thinking skills. Fryer's team found evidence that students did not want tests to move them outside their comfort zones. The study describes some students' responses to New York's "incentive tests":
These exams can quickly move students outside their comfort zone and into material that was not covered in class-especially if they are answering questions correctly. The qualitative team noted several instances in which students complained to their teachers when they were taken aback by questions asked on the exams or surprised by their test results.
The students may have felt that higher-order thinking skills just weren't part of their bargain.
Forces outside of schools may drag down students' test results and challenge even the best incentive programs. Fryer's study notes that poorer students didn't get as much out of even the best programs. They got less bang for the school's buck. Fryer speculates about this finding:
The best evidence in favor of the importance of complements in production are the differences between students who were not eligible for free lunch (and likely have intact, more educated famliies who are more engaged in their schooling) and those who are eligible for free lunch, though the differences are not statistically significant. The social ills that are correlated with eligibility for free lunch may be important limitations in production of achievement.
These comments might send a chill down the spines of teachers whose evaluations and pay will hinge mostly on their students' test scores. Just one less reason to teach low-income children, I guess.
Tying teachers' pay to test scores might not be such a great idea after all. Fryer comes right out and says as much:
It might be less effective to give teachers incentive pay based on outputs (test scores of their students) relative to inputs (staying after school to tutor their students.)
Coercing or coaxing teachers to raise test scores may have little or no impact. It may even cause harm. But giving them more support to do the kinds of things that are likely to improve student learning might be a much better approach.
In its recent review of Fryer's study, Time magazine puts the issue this way: We tend to assume that kids (and adults) know how to achieve success. If they don't get there, it's for lack of effort--or talent. Sometimes that's true. But a lot of the time, people are just flying blind." Fryer suggests that pay tied to outcomes does little to remove the blindfold.
Despite what some have said about him, Fryer is no ideologue. He is cautious about his conclusions, and he would never present any of the speculations I've elaborated on as ironclad. The evidence is still thin. His interpretations need more research.
But some of the things he's hinting at are anathema to the hardest of hard-core people in the merit pay camp. As states like Florida prepare to bet the farm on teacher pay and evaluation programs tied mostly to test scores, they should pay close attention to what Fryer is writing between the lines.
(For another response to Fryer's study, see Larry Ferlazzo. Ferlazzo is very concerned about the long-term impact of cash rewards for students, and he offers a number of other ways he and his colleagues have been inspiring higher performance.)
Click here to browse dozens of Public School Insights interviews with extraordinary education advocates, including:
- National PTA President Otha Thornton on the Common Core
- 2013 School Counselor of the Year Mindy Willard on the state of her profession
- Supervisor of Administration John Swang on saving money in energy costs
The views expressed in this website's interviews do not necessarily represent those of the Learning First Alliance or its members.
Excellence is the Standard
At Pierce County High School in rural southeast Georgia, the graduation rate has gone up 31% in seven years. Teachers describe their collaboration as the unifying factor that drives the school’s improvement. Learn more...
- AACTE's Ed Prep Matters
- ISTE Connects
- PTA's One Voice
- PDK Blog
- The EDifier
- School Board News Today
- Legal Clips
- Learning Forward’s PD Watch
- NAESP's Principals' Office
- NASSP's Principal's Policy Blog
- The Principal Difference
- ASCA Scene
- Always Something
- NSPRA: Social School Public Relations
- Transforming Learning
- AASA's The Leading Edge
- AASA Connects (formerly AASA's School Street)
- NEA Today
- Angles on Education
- Lily's Blackboard
What Else We're Reading
- DQC's The Flashlight
- Center for Teaching Quality
- The Answer Sheet
- Politics K-12
- U.S. Department of Education Blog
- John Wilson Unleashed
- The Core Knowledge Blog
- This Week in Education
- Inside School Research
- Teacher Leadership Today
- On the Shoulders of Giants
- Teacher in a Strange Land
- Teach Moore
- The Tempered Radical
- The Educated Reporter
- Taking Note
- Character Education Partnership Blog
- Why I Teach