Join the conversation

...about what is working in our public schools.

I KNEW IT! (Think Tanks Peddle Fishy Survey Results)

vonzastrowc's picture

A few months ago, the blogosphere was abuzz with news that American students are shockingly ignorant of U.S. civics and history. Research sponsored by conservative think tanks found that fewer than one in twenty public school students in Arizona and Oklahoma could answer six or more questions correctly on the U.S. Citizenship Test. The most alarming finding: Only one in four could name George Washington as our first president. It turns out that those findings were likely hogwash.

I suspected as much when the studies were released. The results of the Washington question in particular didn't pass the laugh test. Statistics guru Nate Silver had the same reaction in September. For example, he found the claim that not one out of 1,000 Oklahoma students could get more than 7 answers right well nigh impossible. "Isn't there some total nerd in Tulsa, some AP Honors student in Stillwater, who was able to answer at least eight of these ten very basic questions correctly?"

His suspicions grew when Oklahoma state representative Ed Cannaday re-administered the same test to seniors in 10 high schools across his district. According to Cannaday, almost 80 percent of his seniors answered all ten questions correctly, and 98 percent named Washington as our first president.

Is Cannaday faking his results? Silver doesn't think so: "I would put every dollar to my name on Cannaday, who has kept the surveys and is happy to show them to them to anyone who comes asking." I'm not as confident in Cannaday's numbers as Silver is, but Silver makes a strong case for tossing out the think tanks' original results. His final judgment is damning: The original research was "fabricated," he concludes.

It alarms me that so many people swallowed the think tanks' dubious research whole. Newspapers repeated it without reservation. Bloggers used it to fuel their outrage. The whole business confirmed what a journalist told me a couple of years ago: The public will believe anything you tell them about public schools, no matter how grim.

So what's next? Seventy-five percent of American high school students can't write their names? Count to ten? Tie their shoes?

I certainly don't believe American students know nearly enough about U.S. history and civics. But the think tank research revealed what a very dim view we have of our children, not just our schools.

And if we're willing to believe anything about our children and our schools, we'll accept any reform, no matter how extreme, no matter how destructive.


I would submit to you that

I would submit to you that "the public" can be a bunch of ninnies sometimes. Most people you talk to can tell you that almost all public schools are horrible... except for the one their child happens to attend. LOL

I object to testing public school children as if they were some captive bunch of animals who must "perform" their little circus tricks. Survey this, test that... and what is the real point? IMO if the "test" is not directly related to the curriculum, it shouldn't be happening in school. Why did these folks have access to high school students in the first place? Why do people somehow get to "survey" children about drug use, sex, the name of the first president or just about anything else?

As both a taxpayer and a parent, I object to this kind of thinking. Oh, and pep rallies. Pep rallies and early dismissal for homecoming also send the wrong message to children. Imagine them doing an entire rally and letting school out for the chess or science bowl team and you'll see where I'm headed with this. :]

In any event, you're not the only one discouraged by stupid statistics, or seeing statistics used to prove a stupid point. IMO this is where HSLDA is heading with its testing of homeschoolers. This whole idea that we must PROVE ourselves to the general public is disconcerting. It doesn't matter if we do well or not... it should be about whether we have the right to direct the education of our children, and that alone. I think that the self-righteous homeschoolers who are tooting about how hs-ers do better on certain tests than public schoolers aren't winning any friends anyway.

Well... just my opinion.

PS Just for fun... a survey

PS Just for fun... a survey on a high school campus:

http://www.youtube.com/watch?v=-uPcthZL2RE

Why do you, out of nowhere,

Why do you, out of nowhere, say "I'm not as confident in Cannaday's numbers as Silver is"?

What evidence do you have to doubt the survey he conducted? Such a dumb thing to say

Ouch, Matthew! Break it to me

Ouch, Matthew! Break it to me gently! Silver's statement was supremely confident, but I haven't seen Cannaday's materials, nor do I know the exact conditions under which he conducted his experiment. Were all tests administered at the same time? MIght some of the takers have gotten wind of the purpose beforehand?

I need hardly say that I give Cannaday's numbers more credence than those of the think tanks, which seem outlandish on their face. But if I'm going to be skeptical of survey results, it seems I should be consistent in my skepticism, don't you think?

Perhaps Cannaday's numbers are every bit as reliable as Silver thinks they are. I certainly hope so. That's just not a judgment I'm equipped to make right now. It's enough to note that the think tanks' numbers were almost certainly wrong.

Thank you for the clarity of

Thank you for the clarity of thought. It seems like so many people are blind to the obvious and potential problems with Cannaday's poll. You've hit on some of the potential ones - questions that should be answered, but here are a few obvious problems that don't need much or any further investigation.

He only polled one school district the other poll asked questions throughout the state. 325 students in 10 schools which is supposed to be close to all of the seniors. There must be some pretty small schools in that district. I think we had at least that many graduating seniors at the single high school I attended.

He only polled seniors while the other poll asked the questions of students in all four grades. This alone pretty much negates any comparison unless accompanied by a study that shows seniors perform the same as the other grade level - which seems unlikely.

The poll was taken after the original poll and the announcement of the results in a very public way that resulted in very large amounts of publicity not only about the results, but the questions that were asked. I'd expect this was a topic of discussion in classrooms as well.

So even if Cannaday did everything else associated with the poll correctly, the poll just isn't a good counterpoint.

BTW, my understanding from reading the Kos blog is that the "poll" was administered as a "test" in the schools - apparently by school teachers and administrators.

Hey, Claus. How about

Hey, Claus. How about identifying the names of the individuals and the names of the stink tanks that peddled the fishy propaganda?

The youtube clip indicating that kids can be suckered by a term like "suffrage" tells something about the status of US history instruction, but not a whole lot. "Tests" can be constructed to show about any result one likes by manipulating the way the items are stated.

A lot of parents would be opposed to their teenagers engaging in premarital interdigitation. And so on.

This isn't quite the same

This isn't quite the same thing, but here's something that happened to our school district, San Francisco Unified (where I'm a parent volunteer activist,not an employee).

Every year California requires that a 6-part physical fitness test be administered in certain grades (something like 4, 7 and 10), results reported to the state, disseminated to the public, blah-blah.

A couple of years ago, the Calif. Dept. of Education reported that zero students in those grades in all of the San Francisco Unified School District could pass the test. The issue was that zero students, allegedly, could pass the flexibility test that's 1/6 of the testing (sitting on ground, legs out straight, lean forward and touch fingers to a mark, or something similar). Many SFUSD students passed all other parts of the test, but that alleged "failure" meant that none supposedly passed the overall test.

This was duly reported far and wide -- a CDE press release, news reports hither and yon -- oh, woe, those fat and sluggish SFUSD students. The SFUSD PR dept. was caught flat-footed, so to speak, and could only stammer that they'd look into it.

One of the reported grades happened to be my daughter's grade -- my daughter who can pretzel herself into all kinds of weird shapes. Also, she has a classmate who is literally a performing contortionist, not to mention many who are dancers.

So I researched it myself. Well, duh, duh, duh. It was a computer coding error made by an SFUSD staffer (who was mortified when I talked to him, to put it mildly). Any moron should have known it was an error, and yes, I include the morons at the CDE who issued the press release and the morons in the press who reported it. I posted a comment in those words, and it turned out a local TV news editor who had created the broadcast and is a friend of a friend actually indignantly asked me if I was saying she was a moron -- yes, I was forced to tell her, sorry, but I am.

Same thing here -- Claus has it exactly right: "Isn't there some total nerd in Tulsa, some AP Honors student in Stillwater, who was able to answer at least eight of these ten very basic questions correctly?" These are the equivalents of the contortionist and the dancers in my daughter's class.

When people are so eager to believe the worst about public schools that they'll believe claims that are obviously untrue, they only demonstrate their own ignorance.

Mrs. C--You're sending me

Mrs. C--You're sending me back into the vortex of our testing and standardization discussion. (Noooooo! Nooooooooo!) I do see the need for testing and for standard measures of performance, even though I think we need a whole lot of other stuff.

Dick--Which think tanks, you ask? The Arizona study was sponsored by the Goldwater Institute. The Oklahoma study was sponsored by the Oklahoma Council of Public Affairs. As for the individuals who uncritically quoted from the dubious study, they're too numerous to list here. While I agree with you that the WAY you ask the question can determine the outcome, the surveys--as I understand them--merely adhered to questions as posed by the citizenship test. (Did the questioners actually ask the questions differently? Who knows? Does anyone have their hands on the raw results?)

Jay--You sum up some of my concerns about the Cannaday data, though I'm less concerned by the single-district sample or even by the the fact that Cannaday  focused only on seniors. I'm most concerned by the possibility that the students were on to the whole game, as it were. Still, the think tank results just don't seem remotely credible.

Caroline--Your comment nails why I don't think the original results were credible--though I can't take credit for Nate Silver's great line about nerds in Tulsa, etc. Even without the Cannaday numbers, which I do think cast further doubt on the original survey, Silver's analysis of the statistical improbability of the original results--what a crazy distribution--strikes me as pretty convincing. Your SF example drives it home nicely. Sometimes, we have to use a word like "moron!"

OH NO, Claus... I swear I

OH NO, Claus... I swear I didn't mean to do that (this time)! Honest to goodness, I was thinking more of drug and alcohol surveys and things like this:

http://www.psychsearch.net/news/COL.pdf

Old article, but the scary premise is still something that parents need to be cautious about. So, I was on a tangent by a little bit... just not the one you were imagining.

Bless you for putting up with me, though! I just enjoy your articles, and I like chatting, too. :]

Ah, yes. We were talking

Ah, yes. We were talking apples and oranges! Anyway, you're one of the few people who can make a vortex a pleasant place to be.

As for the individuals who

As for the individuals who uncritically quoted from the dubious study, they're too numerous to list here.

Could you give us the most notorious, Claus? Where is Jerry Bracey when we really need him.

Let's just say I linked to

Let's just say I linked to some of 'em!

Fair enough. A link is as

Fair enough. A link is as good as a hook, and the stink was yesterdays news anyway. Let's get on with the future. There are certainly lots of other fish to fry and stinks to avoid.

Seems like some people will

Seems like some people will believe anything they hear these days. Parents need to be careful about believing the supposed researches and surveys. They should avoid making hasty decisions or judgments without doing their own leg work.

As I recall, I took a dive on

As I recall, I took a dive on that issue, choosing to question the rightwing agenda of the Oklahoma Council rather than their dubious results. So I at least have some egg salad sandwich crumbs on my face.

True, it doesn't seem likely

True, it doesn't seem likely that not even a single kid knew more than 7 answers.

Still, the findings didn't seem that out of line with many other legitimate polls finding an appalling lack of civic knowledge even among adults.

E.g., Zogby: "When asked to name two of Snow White's Seven Dwarfs and two of the nine U.S. Supreme Court Justices, 77% of Americans polled were able to identify two dwarfs, while only 24% could name two Supreme Court Justices. . . . Knowledge of government did not fare much better with 73% of those polled able to name all three of The Three Stooges, while only 42% could name the three branches of government." http://www.zogby.com/Soundbites/readclips.cfm?ID=13498

Gallup: 24% of Americans didn't know that the US won independence from Great Britain (some said Russia or Mexico), while nearly 20% thought the sun revolved around the earth. http://www.gallup.com/poll/3742/new-poll-gauges-americans-general-knowle...

Pew Research Center: As of 2007, fully 31% of Americans could not name Dick Cheney as Vice-President, even though Cheney was one of the most prominent (or notorious) Vice-Presidents in history. http://people-press.org/report/319/public-knowledge-of-current-affairs-l...

Yes, Stuart, I agree that

Yes, Stuart, I agree that we've seen other none-too-cheerful assessments of civic knowledge in this country. That's one of the reasons why many groups have been concerned about claims of curricular narrowing in recent years.

Still, the data out of Arizona and Oregon were of an entirely different order of magnitude. It's one thing to say that only 1/4 of Americans can name two Supreme Court justices. It's another to say that only 1/4 of high school students can name the first President of the United States!

This is more than hair-splitting. We need to strike a balance between real urgency (we have to make things much better) and boundless alarmism (Things are so bad that we have to blow up the system and start from scratch using my favorite reform strategy.)

Things are so bad that we

Things are so bad that we have to blow up the system and start from scratch

Who could be described, without ridiculous exaggeration, as making that argument? John Taylor Gatto, maybe. Anyone else?

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options