Join the conversation

...about what is working in our public schools.

Assessment

Blog Entries

The first paragraph of Education Next’s Grading Schools: Can Citizens Tell a Good School When They See One? discusses the widespread availability of school standardized test score data. Reading that, I thought I knew what the article would be about. Citizens judging schools based on test scores alone, rather than more meaningful measures. It resonated with me, because the same day I read the article, I had fallen prey to that trap. I was talking about a really great school...and talking only about its test scores. Someone called me on it. I could have mentioned the amazing parent engagement at the school. Or discussed how students at this school--over 90% of whom receive free or reduced price lunch--collected money to send to relief efforts in Haiti. In imparting such citizenship to its students, this school must be doing something right. I know all this, about this school and many others. But I still talk mainly about test scores. We do need to look beyond test scores in determining a school’s quality, but do most citizens actually do so?

Of course, by the end of the second paragraph I knew that was not what this article was about. Instead, it described a study that looked at whether citizens judge school quality based on performance data, or whether indicators such as the racial or class makeup of the school sway their perspective. An entirely different question, but also very interesting.

So I read the article. And while I am not sure I entirely trust their methodology, I am somewhat heartened to learn that citizens do judge the quality of their schools based on student proficiency rates in core academic subjects, not racial demographics. They do ...

The big education story these days is the chilling effect of higher cut scores on New York State tests. The miracle in New York City seemed a bit less miraculous after after the state raised the bar. Most of the sniping among pundits and wonks has focused on the extent to which the new standard undermines the claims of New York City's school reformers. But I think the story raises even bigger questions. For example:

Where Have the Media Been for so Long?
Cut scores have by all accounts been low since 2006, but, as late as 2009, only a few newspapers had addressed that fact. Critics like Diane Ravitch had raised the issue for years. In August of 2009, teacher Diana Senechal showed that students could guess their way to a passing score. Only in September did the New York Times cover that story--and their story didn't mention Senechal.

By the time the Times ran the story, state board Chancellor Merryl Tisch was already on the case. She had the real courage to declare the cut scores bogus and call for a higher standard.

But in this case, the fourth estate lagged behind. Given how heated and political the school reform debate has become, and how ready parties on all sides are to make grand claims about success or failure, that's bad news.

Why Do We Have Such a High Tolerance for Data that Obscure as Much as they Reveal?
The answer to that question is easy: politics. When so much of the debate is driven by ideology, PR and even fear, you can't expect truth-tellers to get rewarded. Those whose jobs depend on the scores point out problems at their own peril. Those who stake their political ...

New York state education officials recently learned that their standardized assessments were not properly measuring student proficiency. They recalibrated the way the tests were graded and, not surprisingly, the new (theoretically more accurate) scores are significantly lower than those previously reported.

In Florida, concern with the accuracy of test scores caused the state department of education to hire independent contractors to examine the results. Several districts believe that individual student gains fell in an unusual manner. The results are not yet in.

Two unrelated instances. But both illustrate the danger of relying heavily on standardized test scores in making high-stakes decisions for students and schools.

In New York, students appeared to have made more progress than they actually had. The schools looked good. But in New York, test scores are used to determine whether students must attend summer school and are promoted to the next grade level. Because of the grading problem, some students were denied services that could have helped them truly master the skills they need to ...

vonzastrowc's picture

Gaming the Tests

If there's a test, then there's a way to game it. It's crazy to think that we should therefore abandon standardized tests. But it also makes no sense to rely on test scores without looking for supporting or conflicting evidence elsewhere. Yesterday's New York Times piece on the City's gifted and talented Kindergartens drives this point home.

Two years ago, the score on a standard city-wide test became the sole basis for admission to those programs. Since then, the share of black and Hispanic children in those programs has plummeted. It appears that wealthy parents are buying pricey test-prep books and services for their children. Poor children are, of course, priced out of that market.

I don't know how healthy it is for wealthy four year olds to "turn to jelly on test day" because they've absorbed their parents' fears that a low score will blow their chances at Harvard. But I'm at least as worried about the fate of poor kids when the testing system gives rise to a market whose very premise is that money buys advantage.

As usual, the intentions behind the testing program were noble. Schools chancellor Joel Klein wanted an objective measure that put all children on an equal footing.

But I'm not sure the unintended outcome should really surprise us. We need look no further than the college admissions industry to see what can happen. Wealthy parents buy test prep services, and some even hire college consultants to help them craft the perfect ...

As more and more states agree to adopt the Common Core State Standards, critics of the effort have been quick to point out that high standards don't guarantee anything. They're right. But that doesn't mean we should back away from the Common Core initiative. High standards are a necessary but insufficient step towards better schools.

Those who raise questions about standards are doing us a service. As Linda Perlstein reminds us, the two states that won the Fordham Foundation's highest marks for their English standards--California and DC--hardly boast the best NAEP results. Folks at the Cato institute, who hate the Common Core effort, are quick to make a similar point. These skeptics offer a useful inoculation against media hype. (Perlstein always plays this role with grace and skill.) They also underscore the point that standards alone won't do wonders.

But the presence of high standards in states whose students don't perform all that well doesn't prove much of anything. Take DC, for example. Its standards are still quite new, and some have credited them with DC's recent rise in NAEP scores. And California's low per-pupil funding levels, together with a whole host of other things, might hold it back.

Clear, high standards won't have much of an impact if the tests are no good, the curriculum is weak, and schools have little or no support to make standards mean something in the classroom. In Massachusetts, whose standards earn high marks, students score on par with students in nations that regularly top the international charts. Some observers see the state's strong tests, staff development for teachers and other supports as reasons for the state's success.

I can already hear howls of protest. This is all mighty speculative, I know. The fact is that it's very ...

Stasia Honnold quit her job as a middle school English teacher. Why? She loved her kids, her colleagues and her principal. What she didn't love was the growing focus on multiple choice tests as a measure, often the sole measure, of her students' progress. When poorly applied, accountability regimes can drive good teachers like Honnold away from the profession.

We certainly need accountabiliy, but we have to pay close attention to how accountability--and the language of accountability--affects great teachers. As debates over school reform get nastier, teachers like Honnold can get labeled as obstructionists. What's worse, the very things that drew them to teaching can get lost in the din.

Let's not forget why great teachers are in the business in the first place. They love to teach, but they also love to learn. They're serving their students, but they're also serving their subject area. They see their subject's relevance to their students, but they also see its inherent worth.

In the end, great teachers want to uphold the value of intellectual work. They know that learning is exciting.

Those are the teachers we don't want to drive out of the profession. They're also the teachers who are most likely to decry the effects of standardized tests on schools. They don't want ...

Yesterday, the Learning First Alliance, which runs this website, released the following statement:

“The Learning First Alliance calls for the reauthorization of the Elementary and Secondary Education Act to promote improved assessment systems that better capture whether students are gaining the skills and knowledge they need to succeed in the 21st century. We believe reauthorization should support assessment systems that are designed to improve instruction and that:

  • Use multiple sources to show evidence of student learning. These sources should reflect both progress towards and mastery of the skills and knowledge students need to be successful in the 21st century. In addition to test scores, measures can include writing samples, performance assessments, portfolios and capstone projects.
  • Promote a broad curriculum that exposes students at all levels not only to reading and math but also to history, civics, science, the arts, world language, career and technical education and other important content areas.
  • Use universal design principles to ensure that all students, including English language learners and students with disabilities, are fully and appropriately included in the assessment system.
  • Include a formative assessment component and provide feedback on all assessments in a timely manner so educators may use results to inform instruction, guide and evaluate investments in professional development, and respond to students’ academic needs.
  • Ensure that educators have access to professional development that supports their understanding and use of data.” ...

The 2009 NAEP reading scores are out, and advocates of all stripes will be in a mad rush to interpret them in self-serving ways. That's bad news for NAEP.

People often use NAEP to support grand ideological claims. Just over a week ago, one blogger credited charter schools with DC's gains in NAEP scores from 1994 to 2007. "One blindingly obvious cause for the improvement: the 100 charter schools operating in the district educating over 30,000 children," he wrote.

Blindingly obvious? Perhaps not. The latest NAEP reading scores included bad news for charter schools in DC, whose 4th grade performance didn't budge, and whose 8th grade performance actually declined since 2007. In fact, the charters dragged down DC's overall numbers, which showed no improvement in eighth grade. Take charters out of the picture, and DC schools showed significant gains. 

So does this prove that charters are a flop? Of course not. But it does suggest that anyone who thinks the policy lessons of NAEP are "blindingly obvious" is prone to wishful thinking. There's been a lot of that going around.

It's always possible to find support for your favorite reform idea if you look hard enough at the NAEP results. Take mayoral control, for example. If you love it, ...

As the debate about school reforms heats up, it's getting tougher to have reasoned, thoughtful conversations about specific reform strategies. You're either a wild-eyed zealot pushing for scorched-earth change or a dour obstructionist doing all you can to defend the status quo. There is little room for doubt in this super-heated environment.

I see this dynamic at work in the growing crop of opinion pieces urging states to give no quarter on teacher evaluation and merit pay reforms. The standard for many pundits seems to be 50 percent. If you don't base at least half of a teacher's evaluation on test scores, you must be a weak-kneed servant of special interests. An editorial in yesterday's Washington Post offers just the latest example of this argument.

But aren't there some questions we should ask before we base most of our pay and evaluation decisions on test scores? Do we know how this will affect teacher morale? Do we know how it will influence teacher recruitment? Do we know how many teachers would stick around under the new regime? Are we sure ...

We're told we have to hold teachers and students to high expectations, but somehow it's OK to have low expectations of policy makers. Teachers and other school staff have learned by now that they must never say never. But policy makers? Don't expect too much from them.

Jay Mathews is just the latest to accept this double standard. "It is politically impossible to pass a plan that doesn't make teachers accountable for student performance," he writes.

We will never return to the good old days (in the minds of some) when we ignored that factor. I agree...that there are better measures of schools, but for the moment they are way too expensive (like regular inspections) and way too complicated for voters to understand and trust."

I admire Mathews very much, but he's way off base here. If we expect schools to move every mountain--"do whatever it takes"--to make every child successful, then why should we let policy makers off the hook in demanding--and paying for ...

Syndicate content