Learning First Alliance

Strengthening public schools for every child

The Implications of New York's Testing Problems Reach Far Beyond the Empire State

vonzastrowc's picture

The big education story these days is the chilling effect of higher cut scores on New York State tests. The miracle in New York City seemed a bit less miraculous after after the state raised the bar. Most of the sniping among pundits and wonks has focused on the extent to which the new standard undermines the claims of New York City's school reformers. But I think the story raises even bigger questions. For example:

Where Have the Media Been for so Long?
Cut scores have by all accounts been low since 2006, but, as late as 2009, only a few newspapers had addressed that fact. Critics like Diane Ravitch had raised the issue for years. In August of 2009, teacher Diana Senechal showed that students could guess their way to a passing score. Only in September did the New York Times cover that story--and their story didn't mention Senechal.

By the time the Times ran the story, state board Chancellor Merryl Tisch was already on the case. She had the real courage to declare the cut scores bogus and call for a higher standard.

But in this case, the fourth estate lagged behind. Given how heated and political the school reform debate has become, and how ready parties on all sides are to make grand claims about success or failure, that's bad news.

Why Do We Have Such a High Tolerance for Data that Obscure as Much as they Reveal?
The answer to that question is easy: politics. When so much of the debate is driven by ideology, PR and even fear, you can't expect truth-tellers to get rewarded. Those whose jobs depend on the scores point out problems at their own peril. Those who stake their political and ideological fortunes on the scores have little incentive to peek behind the curtain.

The New York story should teach us to question claims that this or that strategy is closing achievement gaps. New York City's gaps widened dramatically after the cut score change. Why? Actual gaps in student learning can be much, much larger than gaps in the share of students meeting a cut score. With a higher cut score, struggling students who had just made it to the old standard now find themselves far below the new one. New York City

Officials are right to note that gaps between students' absolute performance on the state tests have narrowed somewhat since 2002. But the bigger message here is that we can't rely on the shares of students reaching a cut score to calculate achievement gaps. In the past, we've been all too willing to hoist every child over a low bar and then declare victory.*

In the end, the New York experience argues for much closer attention to the tests that drive major decisions about policy and practice. Easy tests favor struggling students and obscure achievement gaps. If we really want to close gaps, we had better start devoting far more time and many more resources to doing assessment right.*

Why Have New York City and DC Become the Sole Litmus Tests for School Reform?
The effect of New York's new cut scores is big news in large part because it fuels the battle over whether Joel Klein's reforms light the path forward for all schools. The reigning belief seems to be that Klein in NYC and Michelle Rhee in DC are waging a battle for the future of all urban school districts. For at least some of their supporters, Rhee and Klein are the only act in town.

But there are other big cities out there that warrant our attention. DC and NYC have indeed made some real strides in their NAEP scores (which most people see as more reliable than state test scores). But cities like Atlanta, Boston, LA and Houston boast gains that are as big or even bigger in some areas. Atlanta's gains since 2002 are among the largest in the nation, but we hear precious little about Atlanta superintendent Beverly Hall. Atlanta's long-running reforms don't feature all the street fighting that draws the media to NYC and DC. But Atlanta's reforms are worth a look.

Why Have Charter Schools Become the Sole Testing Ground for What Works?
The story about NYC's scores revealed some bright spots--schools that held up well even under the higher standard. Among them was PS 172, a regular public school where 80 percent of the student body receives free or reduced-price lunch and almost a third are still learning English. Almost all students at the school are proficient in math and English, so it's nice to see this school finally get its due. We know from our own research and books like It's Being Done that PS 172 is by no means alone. But you wouldn't know that from what you read in the papers these days.

Stellar traditional public schools seldom get much attention, while stellar charter schools like the Harlem Success and KIPP academies are becoming household names. Why is this a problem? In part, it's a question of fairness. The staff in all stellar schools deserve the recognition.

But more important, too many policy elites seem to believe that the "charterness" of a school is what really counts in the end. The best charter schools have become the poster children for all charter schools, while the worst regular public schools have come to represent all regular public schools. That can distort the nation's policy priorities and distract from the need for thoughtful charter policies. It can also blind us to lessons we should be learning from successful public schools that are "traditional" in name only.


When all is said and done, the lessons of the New York state testing kerfuffle spread far beyond the borders of the Empire State.

* Paragraphs edited for clarity, 8/2/2010