Join the conversation

...about what is working in our public schools.

Urban

Blog Entries

Imagine you open your newspaper in the morning to find a story about dietary supplements. The story includes a throw-away line or two noting that supplements aren't subject to FDA approval and that the research on supplements is mixed. It then proceeds to extol their virtues, list the ailments each is said to cure, and offer links to discount suppliers. I'm guessing you wouldn't think very highly of your paper.

In some respects, the recent LA Times story on teacher effectiveness isn't all that different from my hypothetical story. The authors mumble a few words about problems with the methods it used to rate 6,000 L.A. teachers. They then launch into full-throated advocacy for the approach. They even publish names and pictures of the city's "worst" teachers.

"No one suggests using value-added analysis [of test scores] as the sole measure of a teacher," the authors write. They then proceed to use value-added analysis as the sole measure of 6000 real teachers in real schools. They brand one as "least effective," name him, and  print his picture in the paper. Then they supply a database of 6,000 teachers rated solely by test scores. A few words about the limits of value-added measures won't blunt the overall effect of the article. Those teachers have been marked.

The authors note that "ineffective teachers often face no consequences and get no extra help." While I'm pleased that the Times has considered the need to help struggling teachers, I'm sorry to see that thought get swept away so quickly by stronger, darker currents. Regardless of what the authors ...

It is fast becoming a received truth that teachers, teachers, teachers make all the difference in a child's academic performance. But what if analysis of students' scores on state tests threw that belief into question? It may have in L.A.

That's not the impression you'll get from the recent L.A. Times story on teacher quality. The Times used student test data to estimate 6,000 L.A. teachers' relative effectiveness. The  story suggests that it's all about the teachers:

Year after year, one fifth-grade class learns far more than the other down the hall. The difference has almost nothing to do with the size of the class, the students, or their parents.

It's their teachers.

But blogger Corey Bunje Bower had a look at the report behind the Times analysis, and he drew another conclusion. The Times notes that the best teachers aren't all crammed into the "best" schools. Bower weighs the implications of that finding:

Teacher quality varies widely within schools--just as with test scores, there's far more variation within schools than across schools. ("Teachers are slightly more effective in high- than in low-API schools, but the gap is small, and the variance across schools is large"). Which means that the highest performing schools don't have all the best teachers and the lowest performing schools don't have all the worst teachers. Which means that something other than teacher quality is causing schools to be low and high performing. Which means we should probably focus our attention on more than just teacher quality.

Of course teachers are very important. Why would anyone teach if teachers didn't matter? But should we put ALL our eggs in the teacher basket?

Other bloggers have raised strong objections to the L.A. Times piece. Can we trust the tests? Can we trust "value-added" analysis of test scores? Is it right to publish names and even pictures ...

The big education story these days is the chilling effect of higher cut scores on New York State tests. The miracle in New York City seemed a bit less miraculous after after the state raised the bar. Most of the sniping among pundits and wonks has focused on the extent to which the new standard undermines the claims of New York City's school reformers. But I think the story raises even bigger questions. For example:

Where Have the Media Been for so Long?
Cut scores have by all accounts been low since 2006, but, as late as 2009, only a few newspapers had addressed that fact. Critics like Diane Ravitch had raised the issue for years. In August of 2009, teacher Diana Senechal showed that students could guess their way to a passing score. Only in September did the New York Times cover that story--and their story didn't mention Senechal.

By the time the Times ran the story, state board Chancellor Merryl Tisch was already on the case. She had the real courage to declare the cut scores bogus and call for a higher standard.

But in this case, the fourth estate lagged behind. Given how heated and political the school reform debate has become, and how ready parties on all sides are to make grand claims about success or failure, that's bad news.

Why Do We Have Such a High Tolerance for Data that Obscure as Much as they Reveal?
The answer to that question is easy: politics. When so much of the debate is driven by ideology, PR and even fear, you can't expect truth-tellers to get rewarded. Those whose jobs depend on the scores point out problems at their own peril. Those who stake their political ...

vonzastrowc's picture

Gaming the Tests

If there's a test, then there's a way to game it. It's crazy to think that we should therefore abandon standardized tests. But it also makes no sense to rely on test scores without looking for supporting or conflicting evidence elsewhere. Yesterday's New York Times piece on the City's gifted and talented Kindergartens drives this point home.

Two years ago, the score on a standard city-wide test became the sole basis for admission to those programs. Since then, the share of black and Hispanic children in those programs has plummeted. It appears that wealthy parents are buying pricey test-prep books and services for their children. Poor children are, of course, priced out of that market.

I don't know how healthy it is for wealthy four year olds to "turn to jelly on test day" because they've absorbed their parents' fears that a low score will blow their chances at Harvard. But I'm at least as worried about the fate of poor kids when the testing system gives rise to a market whose very premise is that money buys advantage.

As usual, the intentions behind the testing program were noble. Schools chancellor Joel Klein wanted an objective measure that put all children on an equal footing.

But I'm not sure the unintended outcome should really surprise us. We need look no further than the college admissions industry to see what can happen. Wealthy parents buy test prep services, and some even hire college consultants to help them craft the perfect ...

About two weeks ago, we posted a conversation with two leaders from Boston's City Connects (CCNX) program, which is working with 11 schools to link each child to a "tailored set of intervention, prevention and enrichment services in the community." The approach has helped raise grades and test scores for the mostly low income children in these schools.

We recently spoke with people in two CCNX schools. Traci Walker Griffith is principal at the Eliot K-8 School, and Kathleen Carlisle is the CCNX site coordinator at the Mission Hill School. Each has an insider's view of this remarkable program at work.

Public School Insights: How has City Connects worked in your school? What changes have been made since it began?

Traci Walker Griffith: A number of changes have occurred at the Eliot School. I came in as principal in March of 2007. In May of 2007 the school was identified as one that would take on City Connects.

We were fortunate because the mission and vision of the Eliot School aligned with City Connects in that we are serving the whole child--academically, socially, emotionally. So we have worked amazingly well together in identifying students’ academic and social/emotional needs. And as we began the program I found that the structures and systems that it offers—whole class review, individual student review, and providing a school site coordinator to maintain and sustain partnerships—really aligned with what we wanted to start at the Eliot School at the time.

Kathleen Carlisle: I would echo many of the things that Traci just said. The whole child philosophy especially stands out in my mind—that is a City Connects and also a Mission Hill philosophy. And I think that the presence of City Connects in Mission Hill has especially impacted the identification of student needs and ways to meet those needs, be they social/emotional, academic, health or family. I think there has been greater connection between supports and needs, and also consistent follow-up.

Public School Insights: Do you have a sense of the results of the City Connects work in your respective schools?

Traci Walker Griffith: When I came on at the Eliot, a school identified as underperforming and in correction, all of the pieces we needed to put in place to increase student achievement were aligned with what City Connects was working on: identifying services and enrichment opportunities for students both inside and outside the school; working with community agencies that in the past had difficulty working ...

An innovative program out of Boston College is making a big difference for children in 11 Boston elementary schools. City Connects (CCNX) works with the schools to link each child to a "tailored set of intervention, prevention and enrichment services located in the community."

Its efforts have gone a good distance towards closing achievement gaps between the low-income children in the program and children who meet state averages. CCNX's results offer powerful support for what should be common sense: When we address the challenges poor students face both within and beyond schools, they flourish.

A rigorous study (PDF) of the program's outcomes tells a pretty stunning story:

  • The beneficial impact of BCNX [the former name of CCNX] on student growth in academic achievement (across grades 1 to 5) was, on average, approximately three times the harmful impact of poverty.
  • By the end of grade 5, achievement differences between BCNX and comparison students indicated that the BCNX intervention moves students at the 50th percentile up to or near the 75th percentile, and the students at the 25th percentile up to or near the 50th.
  • For multiple outcomes, the treatment effects were largest for students at greatest risk for academic failure. For example, English language learners experienced the largest treatment benefits on literacy outcomes, by third grade demonstrating similar report card scores to those proficient in English in comparison schools. In fact,as a result of BCNX, there was no longer an achievement gap between these students.
  • After grade 5, the lasting positive effects ofthe BCNX intervention can be seen in middle-school MCAS scores. The size ofthe positive effect of BCNX ranged from approximately 50% to 130% as large as the negative effects of poverty on these scores.1

We recently caught up with two of the program's leaders: Dr. Mary Walsh, its Executive Director, and Patrice DiNatale, its Director of Practice.

Public School Insights: What is City Connects?

Walsh: City Connects is a systemic, evidence-based approach to school-based student support. It involves assessing, in conversation with teachers and other school staff, each child in the school at the beginning of the school year and then developing a tailored student support plan based on that student's strengths and needs in four areas: academic, social emotional/behavioral, health and family.

That support plan involves accessing services, supports, resources and enrichment for the child, both school-based resources as well as, and importantly, community resources. A trained professional with a Master’s degree—either ...

Do you want to write a news story about school reform? Here's how you do it.

Choose two neighboring schools: a successful charter school and a struggling traditional public school. Then choose one student from each school. Profile both students' humble or even tragic beginnings, but then compare the charter school student's great efflorescence with the continued struggles of the other student.

Use the two students to contrast the promise of charters in general with the problems many urban public schools face. Toss in a sentence somewhere about the uneven quality of charter schools, but don't belabor that point.

That has become a tried-and-true formula for quite a few national journalists lately. The Wall Street Journal ran the most recent variation on the theme last Sunday. It's not a bad article on its own merits. The two students' stories are gripping, and the piece drives home the vital message that a school can change the odds for low-income students.

But can you imagine a national news outlet carrying the reverse story? Can you imagine a tale of two schools in which the traditional public school outshines the charter school down the street? Such stories surely exist, but there is apparently no need to write them in the current political climate. The media are, by and large, turning a blind eye to non-charter public schools that are succeeding against ...

The dream of college for all is one of the first casualties when jobs dry up and the future looks bleak. More and more people are questioning the wisdom of paying big tuition for what could be a small return. Technical school may be a better bet, they say, especially for poor youth who can't afford to get into debt.

They may have a point. But I think it's a very bad idea to retreat from our commitment to get many, many more poor students through college. At the same time, it's unwise to assume that education alone will solve our economic woes.

The "college for all" argument is important, because it offers a vision for overcoming stubborn class inequities. Let's face it, the vast majority of wealthy parents expect their kids to go to college. Even some of those pundits who pooh pooh college in the pages of the Times or The Wall Street Journal would likely pitch a fit if their own children decided to go the voc-ed route. Poor children face a very different reality.

It may be true that college isn't for everyone. But until student inclination--and not income--becomes the major sorting mechanism for college, I'm not ready to abandon the focus on college. After all, those who never went to college are ...

Paying anyone--students or teachers--for test scores might be a bad idea. That's one of the big lessons I draw from Roland Fryer's now famous study of programs that pay students for good behavior, hard work or test results.

In fact, I think the implications of Fryer's study reach farther than that. The study offers a glimpse of how dangerous it could be to attach any big consequences--good or bad--to test scores alone. Here are some of the things I took away from Fryer's report:

We ignore inputs at our peril. It has become received wisdom that outcomes--and that usually means test scores--are all that really matter in school reform. But Fryer's study suggests that people in schools who call for more attention to inputs and the processes of arriving at outcomes aren't just whiners after all. The study found that cash rewards for certain behaviors--like reading more books--were more effective than cash for test scores. In fact, cash for scores seemed to have no effect on student achievement. Why? Incentives to do the right things, the things that promote learning, might well work better than incentives to do well on a test.

Getting kids too focused on their test scores may do them little good--and may even harm them--in the long run. Fryer's team noted that students getting cash for scores naturally grasped at test-taking strategies rather than, say, better study skills or deeper engagement in class materials:

Students [who were asked what they could do to earn more money on the next test] stated [sic.] thinking about test-taking strategies rather than salient inputs into the education production function or ...

When Melissa Glee-Woodard became principal of Maryland’s Lewisdale Elementary School four years ago, it was struggling. The school was in the dreaded “school improvement” process because of the performance of multiple subgroups of students, and it needed change.

Change is what it got. But not the dramatic “fire-all-teachers” change that has been making the papers. Rather, Glee-Woodard inspired teachers, parents and students with a new vision. The staff began focusing on student data in a meaningful way. Targeted professional development addressed areas of weakness in the instructional program. And new summer programs ensured that students kept their academic success going even when school was not technically in session.

As a result, Lewisdale has made AYP every year Glee-Woodard has been principal. The National Association of Elementary School Principals recently honored her for her transformational leadership.

She joined us for a conversation about the school and its journey.

Public School Insights: How would you describe Lewisdale?

Glee-Woodard: Lewisdale Elementary School is located in an urban setting in Prince George's County, Maryland. We are in the backyard of the University of Maryland, College Park. It is a working-class neighborhood. 80% of our students are Hispanic. 17% are African-American.

All of our students walk to school each and every day, and we are a neighborhood school. Our parents are very actively involved. Anytime that you are outside in the morning, you will see a lot of parents either walking their children to school or dropping their children off in cars.

Lewisdale is also a Title I school. 84% of our students qualify for free or reduced meals. And 54% of our students speak English as their second language. So that gives you a general idea of ...

Syndicate content