Scantron Gone Wrong

Robert Pondiscio unearthed this story about the U.K.'s plans to grade student essays by computer.
It turns out that the computer doesn't much like writers like Churchill and Hemingway. Hemingway's prose was too simplistic. Churchill's stirring call to "fight them on the beaches"? Too repetitive:
We shall fight on the beaches,
We shall fight on the landing grounds,
We shall fight in the fields and in the streets,
We shall fight in the hills;
We shall never surrender.
Churchill also loses points elsewhere for mentioning the "might of" the German army. "Might have" is apparently the appropriate phrase.
The lure of computer grading systems is growing, especially for those who balk at the cost of living, breathing essay graders. The British article notes that such systems are "already in use in America, where some children have learnt to write in a style which the computer appreciates, known as 'schmoozing the computer.'"
Defenders of the computer systems say they are every bit as accurate as humans. These days, that's a pretty low bar.
Be afraid. Be very afraid.
SIGN UP
Visionaries
Click here to browse dozens of Public School Insights interviews with extraordinary education advocates, including:
The views expressed in this website's interviews do not necessarily represent those of the Learning First Alliance or its members.
New Stories
Featured Story
The Long Turnaround
According to the numbers, Central Elementary School in Roundup, Montana, seems to fit the currently fashionable definition of a “turnaround” school. After many years of below-average test scores, the school has recently made double-digit gains in the number of its students meeting proficiency on the statewide assessment. In true turnaround fashion, that improvement appears to have happened in a very short period of time. As recently as the 2005–2006 school year, for example, Central’s math score was nearly 20 percentage points below the state average. In 2009 it was at the state average.
But ask current principal Vicki Begin about the school’s success and she’ll insist that it’s been anything but a quick turnaround. Begin, who is in her third year at Central, gives much of the credit not only to the school’s veteran teaching staff, which averages 23 years of experience, but also to her predecessor, Joe Ingalls, who guided the school from 1994–1995 to 2006–2007. Obviously, this is not a case of overnight success or of cleaning house and starting over. Read more...
School/District Characteristics
Hot Topics
Blog Roll
- Boardbuzz
- Edwize
- NSDC Reflections
- Advancing the Teaching Profession
- Principals' Office
- Principal's Policy Blog
- ASCA Scene
- The Leading Source
- PDK Blog
- ASBO Intl.'s Economic Recovery Blog
- Always Something
- AASA's The Leading Edge
- U.S. Department of Education Blog
- The Core Knowledge Blog
- This Week in Education
- PTA Blog
- Such a Smart Mom
- Eduwonkette
- Teacher Leadership Today
- On the Shoulders of Giants
- Relentless Pursuit of Acronyms
- Teacher in a Strange Land
- Teach Moore
- The Tempered Radical
- TLN Teacher Voices
- The Educated Reporter
- The Character Education Partnership Blog
- Center for Public Education
- Connect for Kids
- Once Upon a School



Hemingway ran on horribly in
Hemingway ran on horribly in his sentences. Horribly. And the quote you mentioned from Churchill was from a *speech,* which to my mind follows a different set of expectations from the audience.
Though we're agreed that computers are entirely too frequently and heavily relied upon. I myself have just received a proposed IEP for my non-verbal son which contained the name and diagnosis of another child. (Yes. It did.) Cut and paste, baybeee... and the nerve of it all was that the school wanted me to sign off on it. I only wish I were kidding!
I was surprised when we
I was surprised when we piloted one of these systems in the classroom with students -- a lot of them LOVED it. They got instant feedback on their writing instead of waiting a week or getting somewhat dubious editing from a peer. This is from ambitious urban high school students who will take all the advice and attention they can get, even if it is artificial, apparently.
Not that I'm necessarily a fan in principle, but sometimes technologies that seem completely bogus on the surface can have surprising application and resonance in the classroom. And vice versa!
I think the NY Times ran a
I think the NY Times ran a piece some time ago where they had someone analyze scores from the writing portion of the SAT. He could reliably (but not perfectly) call out a grade for a writing sample viewing it from across the room, because length was the main factor in the grading.
Will they get a computer like the HAL-9000 ("Hello, Dave") that really enjoys working with humans? Maybe then we could talk. What could go wrong?
Mrs. C-- It's true that a
Mrs. C--
It's true that a speech is distinct from a piece of expository writing. Still, I can see instances where repetition as a rhetorical figure could be very effective in an essay. My larger concern is that the most talented writers who go out on limbs could lose points under such a system. I'm more concerned by the computer's desire to replace "the might of the German army" with "the might have the German army." That's a pretty big limitation. Exceptional writers may be the ones who fall through the cracks.
Tom--You make a fair point. Clearly there was a teacher involved, however. A particularly good writer who feels cheated by the program could go to the teacher and get a human judgment. Does the same sort of thing happen in a more high-stakes environment? Thanks, by the way, for reminding me about the brighter side of this particular technology. There are great prospects for assessment systems--built in to serious games or other platforms--that collect rich data and make them immediately available to teachers and students.
David--Thanks for reminding me that it's not necessarily the computers here that are at fault--or the most limiting factor. (I wouldn't want to make Hal angry.)
Some dream that school will
Some dream that school will nurture creativity, while many others would happily settle for uniformity. Will there be a scantron for poetry one day? Pearson? Anyone?
anyone lived in a pretty how town
(with up so floating many bells down)
spring summer autumn winter
he sang his didn't he danced his did
Women and men (both little and small)
cared for anyone not at all
they sowed their isn't they reaped their same
sun moon stars rain
children guessed (but only a few)
and down they forgot as up they grew
cummings never did know how
cummings never did know how to capitalize or use punctuation so points off for him
can we dock him for the subversive message?
Post new comment