Join the conversation

...about what is working in our public schools.

Scantron Gone Wrong

vonzastrowc's picture

Robert Pondiscio unearthed this story about the U.K.'s plans to grade student essays by computer.

It turns out that the computer doesn't much like writers like Churchill and Hemingway. Hemingway's prose was too simplistic. Churchill's stirring call to "fight them on the beaches"? Too repetitive:

We shall fight on the beaches,
We shall fight on the landing grounds,
We shall fight in the fields and in the streets,
We shall fight in the hills;
We shall never surrender.

Churchill also loses points elsewhere for mentioning the "might of" the German army. "Might have" is apparently the appropriate phrase.

The lure of computer grading systems is growing, especially for those who balk at the cost of living, breathing essay graders. The British article notes that such systems are "already in use in America, where some children have learnt to write in a style which the computer appreciates, known as 'schmoozing the computer.'"

Defenders of the computer systems say they are every bit as accurate as humans. These days, that's a pretty low bar.

Be afraid. Be very afraid.


Hemingway ran on horribly in

Hemingway ran on horribly in his sentences. Horribly. And the quote you mentioned from Churchill was from a *speech,* which to my mind follows a different set of expectations from the audience.

Though we're agreed that computers are entirely too frequently and heavily relied upon. I myself have just received a proposed IEP for my non-verbal son which contained the name and diagnosis of another child. (Yes. It did.) Cut and paste, baybeee... and the nerve of it all was that the school wanted me to sign off on it. I only wish I were kidding!

I was surprised when we

I was surprised when we piloted one of these systems in the classroom with students -- a lot of them LOVED it. They got instant feedback on their writing instead of waiting a week or getting somewhat dubious editing from a peer. This is from ambitious urban high school students who will take all the advice and attention they can get, even if it is artificial, apparently.

Not that I'm necessarily a fan in principle, but sometimes technologies that seem completely bogus on the surface can have surprising application and resonance in the classroom. And vice versa!

I think the NY Times ran a

I think the NY Times ran a piece some time ago where they had someone analyze scores from the writing portion of the SAT. He could reliably (but not perfectly) call out a grade for a writing sample viewing it from across the room, because length was the main factor in the grading.

Will they get a computer like the HAL-9000 ("Hello, Dave") that really enjoys working with humans? Maybe then we could talk. What could go wrong?

Mrs. C-- It's true that a

Mrs. C--

It's true that a speech is distinct from a piece of expository writing. Still, I can see instances where repetition as a rhetorical figure could be very effective in an essay. My larger concern is that the most talented writers who go out on limbs could lose points under such a system. I'm more concerned by the computer's desire to replace "the might of the German army" with "the might have the German army." That's a pretty big limitation. Exceptional writers may be the ones who fall through the cracks.

Tom--You make a fair point. Clearly there was a teacher involved, however. A particularly good writer who feels cheated by the program could go to the teacher and get a human judgment. Does the same sort of thing happen in a more high-stakes environment? Thanks, by the way, for reminding me about the brighter side of this particular technology. There are great prospects for assessment systems--built in to serious games or other platforms--that collect rich data and make them immediately available to teachers and students.

David--Thanks for reminding me that it's not necessarily the computers here that are at fault--or the most limiting factor. (I wouldn't want to make Hal angry.)

Some dream that school will

Some dream that school will nurture creativity, while many others would happily settle for uniformity. Will there be a scantron for poetry one day? Pearson? Anyone?

anyone lived in a pretty how town
(with up so floating many bells down)
spring summer autumn winter
he sang his didn't he danced his did

Women and men (both little and small)
cared for anyone not at all
they sowed their isn't they reaped their same
sun moon stars rain

children guessed (but only a few)
and down they forgot as up they grew

cummings never did know how

cummings never did know how to capitalize or use punctuation so points off for him

can we dock him for the subversive message?

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options