Join the conversation

How you measure a school's progress matters. A lot. Just ask Beth Madison, principal of a school that is thriving by common-sense measures and failing by official measures.

George Middle School has made robust gains over the past decade. Over 80 percent of George students receive free or reduced price lunch, and a full 23 percent are special education students. Yet students' test scores are at or above state averages in most subjects.

Still, the school has not made Adequate Yearly Progress seven years running. Why? Because year after year, Madison tells us, it has been a hair's breadth away from meeting its targets for one particular subgroup of students in one particular area, like attendance. Madison is bracing herself for the impact of the H1N1 flu, which could hurt her attendance numbers for yet another year. You can't win.

What does Madison want? In short, some flexibility. She feels her school should be judged for its students' academic growth over time rather than against absolute performance targets. The school has made steady strides despite big demographic shifts that have increased its share of low-income students. But it still falls short of state goals.

Madison is no whiner. She praises No Child Left Behind for pushing schools to do much more for vulnerable children. She believes the extra money she has received for missing performance targets has helped the school improve. But she still feels No Child Left Behind is a "messed up" law.

She can thank her lucky stars that the Portland school district will not throw George Middle School on a Procrustean bed of reform. District leaders will not hobble her by imposing one-size-fits-all reform strategies. (Madison has particularly harsh words for strategies that require struggling schools to fire most teachers. She calls them a “train wreck.”)

The district listens when she describes her school’s success, Madison told us. And the district offers support tailored to her school’s specific needs.

George Middle School is not in thrall to the official version of success. That's good news for teachers and students alike.

Listen to Madison's interview on the Public School Insights podcast (~26 minutes).

Or read an edited transcript:  

Public School Insights: George Middle School has made tremendous strides since the early 2000s. But you've missed Adequate Yearly Progress seven times. Could you tell me a little bit about how you see the school’s progress in the light of the AYP issue?

Madison: AYP in Oregon is not a growth-based model. It is a model with many subcategories within English language arts and math in which [the state] judges students' ability based on a RIT score [which is essentially] a simple score of grade level. [AYP also includes student attendance measures, again divided into subcategories].

So regardless of the fact that the kids who come in at very low levels of previous performance may make years and years of growth gains in one year — or at least their testing shows they do — that may not be enough to meet the magic number.

If Oregon used a growth-based model, then I think that we would not have had any trouble making AYP the last three years. But we have a very large population of special education students -- about 23 percent. Many of these kids come in [to sixth grade] with their learning achievement level between Kindergarten and second grade. We have one of the shortest academic days, in terms of minutes, in the country. We have many, many, many things working against us in how we serve these kids.

What we've consistently done is missed AYP in one subcategory within one of the categories by a couple of percent. So last year, for example, we would have made AYP based on "Safe Harbor," which is what the Oregon Department of Education calls a certain increment of improvement: in our case, an increase in reading for students with disabilities. However, we were unable to use the Safe Harbor provision because that same category of students' attendance was two percent too low. So based on that one thing, we didn't make AYP.

No one looks at the fact that our kids actually had miracle performance last year. We never believed, when beginning the year, that our kids could show that kind of growth — could show those kinds of test scores.

Public School Insights: If I can just clarify, I've seen from some public data that in 2008 you made tremendous gains in your students’ absolute proficiency overall.

Madison: Yes, and we missed AYP in students with disabilities in reading by about 3 percent, I think it was. So it was, once again, just one tiny subcategory.

And in attendance, as well. We are judged on attendance. With the influence of H1N1 this year, we think no matter how heroic our efforts are, we may just have the whole system work against us. Because while right now the flu’s only just ramping up, the state doesn't have any plan for how they'll adjust attendance based on the fact that children cannot even come to school with a cold now. They’re sent home.

Public School Insights: To switch gears a bit, you implied when we were speaking earlier that you've got a very different demographic coming into the school year after year. Is that true?

Madison: That is true. They are two reasons behind it. The first is that we had a boundary change to add another elementary school feeder, and we now have a significant population of immigrants and refugees who may have never attended public school.

The second reason is No Child Left Behind. The federal law requires us to send notice home to all of our families that we are a failing school. It requires us to send notification home to parents that our teachers are not highly qualified, even if that's because of something as simple as the fact that the licensing agency for the state is two months behind and didn't get the paperwork done.

[Because of] the publicity around failing schools under No Child Left Behind, my guess is that nationwide higher performing students are no longer coming to our schools. So what we get is a very large population of students in need of very high levels of interventions.

Last year we estimated that of our incoming sixth-graders, about 35 percent needed Tier III interventions in reading, and 40 percent did in math. [Tier III students require the most aggressive interventions.] This year it's looking closer to 45 and 50 percent. That is to say, then, that the students are coming in farther behind, and yet we're supposed to test them at grade level just like everyone else.

Public School Insights: Yet even with this influx of students who are in need of greater intervention, have you been raising student performance almost across the board, with the exceptions you've noted, for quite some time now?

Madison: Yes, we have, and we are optimistic that what we're doing this year, under restructuring, is even more powerful. Having had seven years of trial and error, we really believe that we are finally centering in on the formula, that mysterious recipe of what to do with 83 percent free and reduced lunch, 75 percent minority, 23 percent special ed students, a huge population of English language learners, and kids coming in so far behind. So I would say that we're feeling positive that what we are doing will have a great impact again this year.

Will it make a big enough impact to make every subcategory of adequate yearly progress? We are not sure. We are certainly going to try. But even if it does this year, state AYP targets go up by 10 percent next year -- 70 percent [of tests meeting standards] would be the target. Can our students with disabilities make that 70 percent, and then the 80 percent target [that’s also fast approaching]? That's where we really see the whole thing starting to become a target that may be almost impossible.

Public School Insights: You’re describing the restructuring effort that was triggered by failing to meet AYP five years running. But it's a restructuring effort that has gone on while you have in fact made consistent gains for as many years running. Is that right?

Madison: That is correct, and I would say that those two things go together. The fact that George Middle School continues to be here is a reflection of the support that Portland Public Schools is giving us in acknowledgment of the fact that we are on the right path to doing what it takes to reform this kind of school.

Had we not posted the kind of successful gains that we have…When you walk into this school, it almost glows. This is a very happy school in terms of how it feels and in terms of how we're perceived internally. So there's the feeling that we're successful. Had we not been successful in that way, my feeling is we simply would have been closed.

But what the school district has done is allowed us to become team-based. We have a written, detailed restructuring plan that has [my teacher leaders] at the helm, and then we have an internal group of teachers on special assignment that are acting as instructional coaches. We've built from the inside for our restructuring model.

Public School Insights: So you did not have to wipe the slate clean, fire most of your teachers and start from scratch?

Madison: That's correct. I think that in part that's because this is not an easy place to work. But people love it here. We love our middle schoolers. We love each other as a community of professionals in terms of the teaching and support staff.

Over the last five years we had a considerable enrollment decline. But then the district changed the boundaries, which brought our enrollment back up. As a result [of the decline] we lost a significant number of teachers. Those teachers were ones who really were not on board with changing their instruction or willing to indulge in the massive number of initiatives that we brought forth to change everything we do here. So we had a large number of teachers leave. The teaching population that we have had that replaced them has some absolutely fantastic teachers.

What the district knows—and they've [reconstituted a school] once before—is that it doesn't work. Unless you change who your students are, if you have already a successful population of people teaching them and managing them, you don't gain anything. You'll be right back where you started, but what you'll also have is a train wreck. Because when you force teachers out and you bring a whole new group in, especially when you have a very union-strong district, you create a real mess.

Public School Insights: So what you’ve done is taken the teachers you have and created a restructuring plan that you feel is keeping the school on a strong track.

Madison: My teacher leaders amaze me on a daily basis with the extreme to which they feel empowered. Because they have been working it long enough, because they have been teaching long enough, and also because of the extent to which they've received professional development.

They feel not only empowered to lead their teams, but to come to me and say, "Okay, here's what we think we really need to be doing. Here's the research behind it, here are the trainers behind it, here's the work, and here's what we've been doing with it already."

They are already just far exceeding my greatest expectations of what they might bring to the table.

Public School Insights: To change topics a bit…. One of the things we're interested in learning is how schools like George Middle School, schools that have made impressive gains but not necessarily made AYP, find measures of progress they can trust. Not just test scores, but other indications that you're on the right path or that you're straying from that path and have to make corrections. Do you have any thoughts you can share about how you've done that?

Madison: Yes. Things are going fairly smoothly now. I will tell you that a few years ago, things weren't smooth.

The master schedule is one of the very important foundations of your school, especially in a middle school. I came in six years ago to a master schedule that had students of mixed grade level and mixed ability in the core classes together all day long. So you could have a sixth-grader who was functioning at a third-grade level in the same core classes all day as an eighth-grader who's functioning at a 12th-grade level. Teachers were absolutely unable to provide differentiation of that kind of width. However, for culture -- for community -- it was great.

When I came in I said, "By my observation and by your own admission, the average teaching level in here is about a high sixth grade. No wonder that of our eighth graders, we have about 30 percent meeting the benchmarks." So the next year, we retooled.

The impact that your reforms are having on your building climate is a really big teller of whether or not what you're doing actually makes sense to your learning community. What happened [in this instance] was that as a result of the way that we broke the day apart, we lost too much of the feeling of community and our behavior worsened. And if behavior's not good, then you can't have the kind of instructional time that you need in your classrooms, nor can you foster that feeling between kids and the community that makes the kids want to come to school, want to achieve.

So one of the ways we measure whether or not an initiative is being successful is the impact it's having on the climate of the kids, and the impact that it's having on the climate of the teachers. You have to keep an excellent sense of humor to work in such a challenging environment, and when the teachers are really dragging, you have to look at that as a measure. What is it that they are not getting? What are they getting too much of? Are we introducing this thing too fast? Have we not provided adequate support and backup for them? What is it that we need to be doing differently there?

Public School Insights: In measuring this, do you look at discipline information about students and then survey teachers to get a sense of what they feel the teaching climate has become?

Madison: We've done a little bit of formal surveying of teachers. I participated in a pilot for the National Middle School Association last year, and that provided quite a lot of very interesting data. But really, it's an open door school. People march right through that door and tell me right what they think.

And we're very electronic. They will e-mail me in a heartbeat with what they think or, if they just can't bring themselves to say it, they'll send someone else to say it. So we haven't used a lot of formal tools, but the tool we did use last year showed that the staff felt that we really were on the right track.

Public School Insights: You mentioned that you've had a lot of support from Portland Public Schools in the work that you're doing. It strikes me that Portland Public Schools therefore have a different set of indicators or measures of progress in their minds than AYP sets forward.

Madison: There's no way that you can say that any school district thinks that any school not making AYP is anything but a bad thing. I've had to produce reams of data and bring the right people to the table to show them what it is we're up against. Because had I not, we really wouldn't be looked upon as the success that we are.

We bring people into our school, walk them around, show them what the classrooms look like, show them what we're doing. So we educate a lot of key support people in the district as to exactly what we're doing. We have four research studies going on here. We have over 30 community partnerships. The list of what we do here is amazing.

But even more is that some of the initiatives that we've brought in are not things the district's ever heard of. For example, ENVoY Nonverbal Classroom Management is not something the district ever heard of, and yet in our Year 3 of training and coaching our teachers in this, no one could dispute the fact that for having the population we do, you come in and it's amazing here.

Public School Insights: One of the things that we've done at the Learning First Alliance is create principles for measuring the progress of turnaround schools. Because we're worried that the “official vision” of success, so to speak, could compel people to stop a strong turnaround process in its tracks before it actually can reach its goals.

Madison: I think that is a very valid worry.

[But] I will say that as a result of having this "failing school" AYP designation, the influx of federal funding into our school has been phenomenal. There's just no question that heavy Title I funding can provide things for you that are really crucial in helping turn your school around.

For example, when you have 50 percent of your kids coming in far below grade level and your grade level textbook adoptions are literacy-based….Our math textbook is literacy-based. Our kids can't read. Our sixth-grade language arts textbook is at about a 6.9 grade level. Yet half of our kids are coming in reading at about a third or fourth grade level.

We've taken this federal money and we have purchased supplemental curriculum. We've purchased software. Software that does direct instruction with kids as part of their academic intervention has been very successful, and when you have to do mass intervention, that strategy can be one of the things that saves you when you have larger class sizes. We have been able to bring in coaches and consultants who have shared amazing wisdom with us. We've been able to coach teachers directly. The resources have been incredible.

We've pushed this school so far. If you asked me six years ago when I first walked in, nobody had any inkling that we needed to change, much less that the change would be as long and difficult as it would be to even get to where we are today.

Public School Insights: It sounds like you are seeing the brighter side of No Child Left Behind. Is that a fair statement?

Madison: I think No Child Left Behind has been absolutely invaluable. Invaluable. I really do. What is happening now in our school…Our kids are so much more successful. I think the legislation itself is really messed up. I think there are some parts of it that really need to be addressed. But I think that change as a result of it has been outstanding. It has been the catalyst.

You have to have a pretty big hammer to get people to change. Just because it's best practice, just because the kids will learn better.… [Those aren’t necessarily] strong enough motivators to change practice. And a lot of what happens in classrooms now is based on how teachers were raised. They were raised in schools where you had direct instruction, you did some worksheets, you took a test, and then “have a nice day.”

To change practice is incredibly difficult. To have federal legislation and funding behind it…It's still incredibly difficult but at least you have a way to say to your staff, "You have to do this."

[However,] the idea that by 2014, 100 percent of students, including those with disabilities, can test at grade level and meet the standards is, if you ask me, in total opposition of IDEA [The Individuals with Disabilities Act]. I really see IDEA and NCLB as in huge conflict with each other. Until it really gets settled out that growth is what should be measured, rather than arbitrary RIT scores, that we should expect every child to be at grade level at any point I think is not only unfair, but I think it's demoralizing to children whose potential may not allow them, under our current system of education, to reach the level they need to reach.

If we had endless resources, then we could take some of these children who struggle so much and possibly get them to grade level by the time they were much, much older. We don't have that as a nation. And in Oregon, it's pathetic. Our funding here is scary awful. And until those kinds of resources exist, I think that there are parts of the law that are way out of whack.

Public School Insights: Let me conclude with a couple of questions about your work as a community school. As a community school you're looking outside of your own walls to find support for kids. Does that mean that you're also changing your definition of success somewhat, too, given that you've got this community component to the work that you do?

Madison: No, because we [consider] our community school as our extended day. To us, it's all part of the same thing. I'll give you an example. This is a late start day. So our teachers have two hours of professional development in the morning, and our community school was was open for our students to come in. They had time for math and reading help. They had Zumba dance class, and then they had some open gym.

The goal there is all the same. The goal really is, get these kids to view school as something that's positive, as a place for them in the time in which they're not actually enrolled in their classes, and then extend on the reading and math.

Right now we're very nervous about the potential for meeting AYP in reading this year. So we've asked our community school to, on top of the academic support classes they already give after school, add another class in the morning where we would have kids come in voluntarily and get on some of our software to compete with each other on how much work they do to get ready for their first round of state testing. It's just a disguised way to learn more reading and math. We see the line between the extended day and the academic day as being very blurred.

In fact, when I was having one of my moments this past weekend in absorbing what my language arts team leader said about the extent to which she's concerned about reading, one of the first things I thought of was, “extended day.” There's only so much you can do in a six and a half hour school day, and especially in a very short school year.

But we have Title I funding for supplemental education services, tutoring. Right now we have about 90 kids out of our 400 enrolled. And we've got enough formative data now -- because we have test scores from the past and because all of our children in need of Tier II or III interventions are enrolled in academic support every day where they do their own testing – that we can really say, "These are the kids we need to target." We then get together with community school and say, "These are the kids we want in tutoring. We need your help.” I think it's a very powerful tool.

Public School Insights: Is there anything I should have asked you but didn't?

Madison: The one thing that really does come to mind...My current direct supervisor, who's now one of the deputy superintendents, became my boss starting last year. And of all of the people in this district, he has been, in my opinion, one of the greatest influences for us in helping guide where we're going.

If you were really to look at district support…This is a big district. It's a big district in a very poorly funded state. So you can’t always have the kind of leadership from outside the building that would really be the best that it can be for your growth. I lucked out last year. Having someone assigned to me who's not only a visionary, but who so totally believes in every child's potential, has been one of the things that I think has put our school on an even greater learning curve these last two years than we were in the past.

He actually came to my building 19 times last year. He carved an incredible amount of time to listen to me saying, "Here's what our kids are like." And his response back was always, "Tell me what you need. What will make the difference?"

Then boom, this year, after six years of asking for it, we have an intervention textbook that's being used in a formal sense for both language arts and math. We have never had that kind of movement. It's always been, "Well, we'll do the research to see what will work with these kids.” Now it's, "We've done the research, we think these are two dynamite programs, here they are. Use them." Wow.

[Editor's note: Thanks to John Norton for reminding me that the phrase is "Adequate Yearly Progress," not "Annual Yearly Progress."  I made the appropriate change!]