The State of Education Research
Discussions on education policy should be rooted in evidence, rather than hunch or politics. The Learning First Alliance is committed to promoting research that can help us develop the best ways to improve America’s public schools. The Alliance works to find solid research that can be used to support America’s educators, students, and schools, while also promoting the value of research itself.
In addition to believing that research should inform practice, LFA believes that practice should inform what is researched. The Alliance works with the research community to enhance both parties' understanding of how best to form the research agenda.
Alliance Resources on the State of Education Research
Using Evidence for a Change: Challenges for Research, Innovation and Improvement in Education
A Report on a Forum Hosted by the Knowledge Alliance and the Learning First Alliance
The Knowledge Alliance and Learning First Alliance recently joined forces to explore how research affects policy and practice in education. With generous support from the William T. Grant Foundation, the two organizations came together on June 8th to host a Capitol Hill forum on this timely topic. Dr. Denise Borders, who chairs the Knowledge Alliance Board, introduced the forum with a clear statement of its overarching goal: “To raise awareness about how, when, and why—or why not—research evidence is used in education policy and practice.”
The forum brought together a group of experts, each of whom sees the issue from a different vantage point: a foundation head, an association leader, a teacher, a reporter and two researchers. All agreed that we have much to do to bridge the gap between research and practice. But all felt that the nation had some real opportunities to close that gap.
Understanding the Challenge: The Gulf Between Research and Practice
Steve Fleischman helped define the forum’s central challenge by laying out findings from a recent study of how policy makers and practitioners use evidence from research: Toward a Research Agenda for Understanding and Improving the Use of Research Evidence. The report, a collaborative effort by Education Northwest, the Grant Foundation, and the Center for Knowledge Use at the Knowledge Alliance, detailed findings from five focus groups and ten one-on-one interviews with leaders in policy and practice. In Fleischman’s words, those leaders revealed “a gulf between research design and real-world practice.”
The report presented a litany of barriers to using research evidence to shape practice or policy. “Participants were challenged to apply research, because of their own lack of knowledge and skills in acquiring and interpreting research,” Fleischman said:
They found numerous obstacles, such as time constraints, the volume of research evidence available, the format in which it was presented, and the difficulty of applying the research to their own situations. Participants…were concerned that it was conducted and reported for ulterior motives—often for political motives—and that research can be shaped to say anything.
And most dispiriting: “No participant in this study mentioned any breakthrough research or cited any findings they felt had a dramatic effect on policy or practice,” Fleischman reported.
But the study offered some ideas for improving matters. Participants called for research that builds stronger “links to local context.” They asked for research that is concise and easy to read. And they stressed the importance of trusted intermediary organizations that can share new ideas and evidence.
Fleischman noted that such organizations might include “research institutions, professional organizations, partners, coalitions, peers, networks, [and] the media.”
Fleischman’s remarks set the stage for a panel of experts with diverse perspectives on research. The moderator was Dr. Sharon Robinson, president of the American Association of Colleges for Teacher Education and a past director of the Office of Educational Research and Improvement. The panel itself included a teacher, a researcher and an education reporter:
- Susan Freiman, Staff Development Teacher at Viers Mill Elementary School in Silver Spring, Maryland. Freiman and her colleagues helped turn a school that once bore the inglorious nickname “slumville” into a place where almost every student is proficient on state tests.
- Dr. Robert Balfanz, a research scientist at Johns Hopkins University and associate director of the Talent Development Middle and High School Project, which works with more than fifty secondary schools to develop, implement, and evaluate whole-school reforms. Balfanz is one of the nation’s leading authorities on the drop-out crisis.
- Debra Viadero, an associate managing editor at Education Week. Viadero is an award-winning journalist, a longtime research writer and the author of an influential blog called Inside School Research.
When Research Shines: Bringing the Big Problems to Light
Panelists agreed that research has often revealed profound problems—such as inequities in education—that might otherwise go undetected. Balfanz offered an example. He recalled a collaboration between Johns Hopkins and Philadelphia City Schools to create “early warning” interventions for struggling high school students. Before researchers came on the scene, he said, people in the district were hard-pressed to say how many students were entering high school behind grade level. Nor did they know when the districts could best intervene to improve the odds for these students.
The researchers gave the district a much clearer picture of the problem they faced—it was worse than they thought. They also helped the district shape strategies for attacking it early. At a more basic level, Balfanz added, they prompted the district to collect data that otherwise might have remained scattered across separate departments that rarely communicated.
Balfanz described this partnership as a model for collaboration between researchers and people in the field:
It addressed a problem of practice. It gave them something comprehensible. It made sense to school folks. It gave them data they could act upon. And it seemed manageable. And that’s why I think it rapidly spread and had an impact on policy and practice.
Unfortunately, panelists agreed, such stories are all too rare.
Does Research Get Lost in Translation?
Reporter Debra Viadero summed up a major theme that ran through the panel discussion: “The problem is trying to find the interventions that work on a large scale, and I don’t think we’ve seen that happen much at all.” Instead, our evidence of what works seemed to amount to what moderator Sharon Robinson called “random acts of excellence”: a set of isolated anecdotes and hopeful research findings about strategies that seldom take root or flower in new soil.
Research too often lacks validation in practice, Balfanz added. That is, it often fails to prove that a promising idea “can work in a living school, under living school conditions.”
When Knowledge and Power Collide
Some on the panel and in the audience worried that politics could skew the lessons of research. Asked what should happen when advocacy groups push dubious research through press releases, panelists agreed that researchers and reporters should cry foul.
Viadero noted that Education Week often points out shortcomings in the research advocacy groups release to advance their agendas. The paper has even run stories on the lack of research behind many of the administration’s proposals in Race to the Top, she said. But she was careful to add that most reform strategies championed by any group have little research behind them. Policy makers often have to follow their best hunches.
Balfanz pointed out that school systems, too, can fall prey to politics. “Oftentimes, at the district level, power and knowledge aren’t aligned,” he said. A leader’s pet idea can trump evidence. Freiman replied that shared leadership in schools is the best defense against this tendency. No reform strategy will work if leaders simply thrust it on teachers.
Is the Whole Less than the Sum of its Parts?
Robinson asked panelists a question that got to the heart of the whole education research enterprise: Are all the often conflicting research reports that come out year after year actually adding up to anything? Are they building to an understanding of how we can fix schools?
Viadero offered guarded optimism. She acknowledged that dueling research reports were frustrating, but she felt that “we are building an accumulation of knowledge that is heading somewhere.” Yet she also offered a sobering qualification: “I don’t think many in the field share this perception.”
In the end, it was hard to avoid the conclusions of the focus group study: The people who have to carry out the work of school reform do not believe that education research is lighting any clear way forward.
Research will Fizzle If We Don’t Empower People in the Field
Yet the discussion did offer glimmers of hope. Panelists saw the promise of strategies that give people in schools a more direct role in the research enterprise.
Freiman was a forceful advocate for this approach. She drew on her experience as the staff development teacher at a school whose stunning success prompted a visit from President Obama. She explained that the staff of her Maryland elementary school adapts the research it receives from the district. She and her colleagues study it, test the strategies it supports, review their outcomes, adjust their approach when they need to, and then begin the cycle again. “It’s the whole plan, do, study, act, use [cycle],” she said.
But none of this can happen if people in schools lack the conditions for doing it well, she told the audience. They need time to study the research, collaborate with their colleagues and review the results of their work. They need staff development to build capacity for this work. They need the shared authority to act on the needs they see in their own students—and to change their practice when the need arises. And they need a school-wide commitment to a focused research agenda.
As long as people on the front lines lack these conditions, she said, they won’t think much of research:
If we don’t have the time to use the research, to collaborate with our peers, to…look at the data to make decisions—the time to change what we’re doing—then [research] is not useful.
Like Politics, Is All Research Local?
Freiman’s remarks lent support to an observation Steve Fleischman offered in his review of focus group findings at the beginning of the forum: “If all politics is local, all research is local as well.” It is in the local context—and in collaboration with people in local districts and schools—that research can show the most promise. Yet Freiman noted that research can often seem irrelevant to local needs. “If there is no need,” she said, “it does not matter.”
Balfanz called for more investment in local research partnerships:
We have to invest in more place-based research. Places like the Chicago Consortium [for School Research], the Baltimore Education Research Consortium—New York has one, Newark has one—these are growing consortiums of researchers and school districts partnering on problems of practice. And this gets to that local issue.
The consortia offered one way to bridge the biggest gulf between research on the one hand and practice or policy on the other. Researchers work closely with practitioners in specific places to study specific problems and test interventions in specific contexts. But Balfanz was quick to add that research should not have to stay local. Networks among local consortia would allow them to share insights so that they “don’t have to reinvent the wheel,” he said.
So What Is a Federal Policy Maker to Do?
Participants in the forum wrapped up their comments with some words of advice for federal policy makers.*
- Help people build on their successes. Bob Granger of the Grant Foundation stressed the importance of “continuous improvement over time.” Rather than expecting each new study to yield big answers to big problems, he said, education leaders and policy makers should support continuous learning: “working with people you trust, reflecting on what you are learning, trying to assimilate new stuff you’re hearing about to your existing notions about the way the world works.”
- Strengthen the research infrastructure. Balfanz called for an infrastructure that would help researchers focus on problems of practice. He reiterated his belief that place-based research consortia could be an essential part of this infrastructure.
- Don’t shy away from big gambles. Balfanz said that education research had been too risk averse. Caution is warranted about 90 percent of the time, he argued, but for the other 10 percent, “we should throw some long balls and try to do some big studies to get some big answers quickly.” His main example: extended learning time. It’s all the rage, he said, but no one knows whether we should extend the day, the week or the year. And no one knows how much of that time we should devote to academics or enrichment. It’s high time to put some real money into a big study of the issue, he said.
- Focus on Parent Engagement. Freiman called on policymakers to invest more money in parent engagement. Existing research has revealed its power, she said, but efforts to engage parents in schools remain starved for resources.
- Don’t Forget the Intermediaries. Fleischman urged much greater support of intermediary organizations that summarize or translate research findings for people in the field. The focus group study found that educators and policy makers rely on these intermediaries to help them make sense of research.
- Recite the Solemn Research Pledge. Fleischman did not limit himself to the responsibilities of federal policy makers. He added a tongue-in cheek reminder of everyone’s responsibility to improve the state of research. He proposed a universal pledge: “I’d like policy makers to take a pledge that says, ‘I’m going to use research as a guide, not a justification,’ and I’d like practitioners to say, ‘I will not reject research just because it wasn’t done in my school or classroom.’ And I would like researchers to say, ‘I will be more interested in research use than research misuse,’ because a lot of researchers worry endlessly about how their research will be misinterpreted, and therefore they over-qualify stuff.”
- And finally--slow down! Balfanz described the price of impatience. Few studies have more than two or three years to show any effectiveness. Few agencies are willing to fund a big study and wait as long as 10 years for it to bear fruit. The short time frame of most research limits our ability to build a robust body of knowledge about what works, he told the audience.
Freiman drove home a similar point in her closing appeal to the Department of Education: “Please slow down. Please stop rushing educators to make changes and then make a new change when it doesn’t work…. When it’s too much, nothing gets done, and we don’t even know what would have been successful and what wasn’t successful…. We can’t keep racing. You are going to lose great educators, because they are going to get stressed out.”
Parting Thoughts: A Moment of Real Opportunity
Bob Granger closed the forum with some reflections about the discussion and where it might lead us in the coming months.
- Change the “research to practice” slogan. A much better phrase is “practice to research,” Granger said. Research must begin with the places where educators work and the problems they confront.
- Value research that reveals the problems we face. We tend to see research as a review of specific interventions, Granger told the audience. We must not forget that it can also lay bare inequities that were once hidden. It can more clearly define the problems we need to tackle.
- Seize the moment. Granger reminded his listeners that unprecedented amounts of federal money will go towards innovations that aim to improve the lot of young people. The Investing in Innovation Fund (I3) will support promising new work in hundreds of sites around the country. Granger saw this as “an extraordinary opportunity for the government to add on top of the I3 evaluations some additional data collection to try to get down to the level of school buildings or counties to figure out when an…intervention makes a difference and when it doesn’t.”
Our Work Is Cut Out for Us
As she thanked the speakers and the audience, Denise Borders left them with a sense of the work that awaits us. We have to focus “not only on the [research] investments that are needed,” she said, “but on the institutional conditions and supports that enable the research and the interventions to work.” And that means all of us--not just the researchers--are on the hook.
*Viadero abstained from answering this question, citing her need to remain impartial.