Measure What Matters

Don't judge schools solely by their students' test scores in math and reading. Also judge them by those students' later success in college and work. That's the thrust of a new report by Education Sector's Chad Aldeman (PDF). It's a compelling piece of work.
First, Aldeman does a better job than most of exposing flaws in current state accountability systems. He finds little correlation between a school's success in making "Adequate Yearly Progress" on state test scores and its students' later success in college.
Two Florida schools help him tell his story. The state gave the first an "A" for two years running, and Newsweek anointed it as one of the best high schools in the country. But students from the second, a D-rated school across the state, did better in college:
D-rated Manatee was arguably doing a better job at achieving the ultimate goal of high school: preparing students to succeed in college and careers. But because Florida's accountability system didn't measure college and career success in 2006, nobody knew.
I hope people who read this report will cut educators some slack for being a bit skittish about NCLB. Teachers or parents may have a point when they say the current crop of state tests doesn't measure what really matters.
Aldeman's idea to include data on college and career success in evaluations of schools has a few good things going for it. For one, it could bring stability to a wacky system. School ratings often swing wildly from one year to the next. In New York City, for example, schools can shoot from a D to an A, or plummet from an A to a D, in one short year. That strains credulity. Aldeman shows that data on college and career success aren't as prone to such violent swings.
His idea might also get us closer to the goals of schooling. Under the current regime, schools have every incentive to make a fetish of test scores without minding the longer-term implications of what they're doing. Aldeman's idea gets at grander aspirations: success after school. (I wish we could find some reliable measures for civic engagement, but that could be a real can of worms.)
Aldeman acknowledges that his approach has shortcomings. Data systems are still far from perfect, and schools would have to wait a few years to know how their students fared in college or work.
I would add another concern: Schools serving poor kids could be unduly penalized. Poor preparation isn't the only thing that keeps poor students from enrolling or thriving in college. Money troubles too often derail the college plans of even very accomplished young students. And the current financial mess won't make things any easier. College costs keep rising, parents keep losing jobs, and need-based aid isn't keeping up.
Still, Aldeman's report is thoughtful and evocative. Yes, the Devil's in the details, but the report could spark some fresh conversations about how we measure school quality.
Do you agree? Or am I sadly deceived?
SIGN UP
Visionaries
Click here to browse dozens of Public School Insights interviews with extraordinary education advocates, including:
- 2013 Digital Principal Ryan Imbriale
- Best Selling Author Dan Ariely
- Family Engagement Expert Dr. Maria C. Paredes
The views expressed in this website's interviews do not necessarily represent those of the Learning First Alliance or its members.
New Stories
Featured Story

Excellence is the Standard
At Pierce County High School in rural southeast Georgia, the graduation rate has gone up 31% in seven years. Teachers describe their collaboration as the unifying factor that drives the school’s improvement. Learn more...
School/District Characteristics
Hot Topics
Blog Roll
Members' Blogs
- Transforming Learning
- The EDifier
- School Board News Today
- Legal Clips
- Learning Forward’s PD Watch
- NAESP's Principals' Office
- NASSP's Principal's Policy Blog
- The Principal Difference
- ASCA Scene
- PDK Blog
- Always Something
- NSPRA: Social School Public Relations
- AACTE's President's Perspective
- AASA's The Leading Edge
- AASA Connects (formerly AASA's School Street)
- NEA Today
- Angles on Education
- Lily's Blackboard
- PTA's One Voice
- ISTE Connects
What Else We're Reading
- Advancing the Teaching Profession
- Edwize
- The Answer Sheet
- Edutopia's Blogs
- Politics K-12
- U.S. Department of Education Blog
- John Wilson Unleashed
- The Core Knowledge Blog
- This Week in Education
- Inside School Research
- Teacher Leadership Today
- On the Shoulders of Giants
- Teacher in a Strange Land
- Teach Moore
- The Tempered Radical
- The Educated Reporter
- Taking Note
- Character Education Partnership Blog
- Why I Teach



I'm with you, including the
I'm with you, including the concern that poor districts would be unduly punished, if their students do not attend college for other reasons. You wrote a post before about the difference between college skills and actually attending college. We shouldn't expect 100% of our HS graduates to go to a 4 year school. We should expect them to have those *skills*.
Do you expand data to performance in trade school? Community College? What about other post-HS tracks? I don't really know either, but should a school's "rank" get docked because their graduates say, enlist in the army instead of attending State U?
Certainly grading a building simply by the virtues on a reading/math test isn't sufficient, but like you say, the devil is in the details.
While I was teaching, I was
While I was teaching, I was not aware of any standardized test that was designed to evaluate the progress of each child during the course of a year. Norm-referenced tests such as the Iowa Test of Basic Skills were designed to compare populations and correlated very highly with SES. Criterion-referenced tests were designed to test grade-level standards but did not evaluate students who were significantly above or below grade level. Nor did these tests adequately measure the progress of English Language learners. The only reliable test I knew of was one given one-on-one by a teacher or a psychologist. This test had to be given in September and then again in the spring. So, when people say they want to evaluate teachers by standardized test scores, which tests are they referring to? Thank you.
Mr. Brown, you raise good
Mr. Brown, you raise good questions. Will all post-secondary choices be created equal in this model? Would it assume 100% attendance in and graduation from college by 2014? Will it allow low-income schools to catch up with high-income schools, whose students often go to college as a matter of course? Tough questions, but I think the vision Aldeman offers is a great jumping off point for discussion. You're right that everyone should have the preparation to go to 4-year college, even if that's not the choice they make. Now, that choice is often made for low-income kids, because they too often lack the resources and the preparation to succeed.
It's interesting to look at the success of Say Yes to Education in this context. That program has almost closed the college graduation gap between low-income urban students and their suburban peers. But that program provides both academic and non-academic supports. It's critical to remember that it also provides college scholarships. In that kind of world, there would be fewer Devils in Aldeman's details. That said, his report is compelling in any context.
Linda--You hit on an important point. Many assessment experts are concerned that state tests aren't currently designed for teacher evaluation purposes. The National Academies cautioned the Education Department that merit pay programs tied to growth models aren't yet ready for prime time. This is not to say that they won't ever be ready, but it should make us deliberate in how we proceed with teacher evaluation.
Thanks Claus, for a very kind
Thanks Claus, for a very kind review!
Mr. Brown, the report strives to be agnostic in the college/ career choice. For our analysis in Florida, we included students who attended community or technical colleges or who chose to enter the workforce instead of attending a postsecondary school. The only situation where high schools were dinged is if the school was unable to graduate students or if graduates failed to find gainful employment or either a postsecondary institution ready to succeed.
My pleasure, Chad! And thanks
My pleasure, Chad! And thanks to you for your thoughtful response to Mr. Brown's question. Do you have any sense of whether affordability issues--ones that could affect enrollment & persistent--could skew the results for schools?
This is absolutely true. The
This is absolutely true. The poor students are not given the opportunity to reach their full success. It is easy to sweep it under the rug and say that the students that are receiving poor tests scores are "lazy" but this is not the case. Most students that do not have the financial means to pursue higher education are going to feel unmotivated to excel in high school, understandably. Once we open our eyes and stop ignoring the issue we will see many students that are willing to learn and are just looking for a means.
Post new comment