 |
|
|
|
|
|
|
|
|
|
|
|
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
|
||||||||||||||||||||||||||
|
|
Find more Kappan articles in the Keeping the Faith Until the Outcomes Are Obvious Reform initiatives do not produce instant results, Ms. Rallis and Ms. Zajano point out. The question becomes how to maintain the support of the various stakeholders during the trial-and-error stage of implementation. THE SCHOOL improvement team at Charles Webber School was meeting. Now in its third year, the team was pleased that the faculty had accepted its proposed mission, goals, and improvement plan. A variety of professional development activities had already taken place, and a substantial portion of the faculty was using the instructional strategies recommended in the plan. The team was now grappling with the matter of how to measure the impact of the school improvement effort on student learning. "The real question," stated Paul, the project evaluator, is "How do we know that students are learning more with the new instructional strategies that teachers are using?" "Well, I know my students are learning more. Their thinking is more complex," responded Sandra. "But what exactly does that mean?" asked Paul. "How do you know their thinking is more complex?" "For one thing, they can apply a concept they learn in one situation to another," Sandra offered. "That's a start. But we need to collect evidence of how they are applying those concepts," Paul responded. "I find it kind of hard to talk about collecting evidence to measure impact -- mostly I just feel a whole lot better when I'm teaching because students in my class are so busy interacting and solving problems together. You know what I mean," said Carmen. "Fine. But are parents and taxpayers going to be satisfied with a measure like 'I feel better because students in my class are working together'?" asked Paul. "The superintendent told me yesterday that he needs results or the board may recommend that we drop some of the strategies." "I don't think the board understands that this change stuff takes a long time and is pretty messy. We don't exactly know what students will do differently if we believe that each learner constructs his or her own meaning," said Tony. "Besides, as teachers, we are still learning the new strategies. It might take me years before I am totally comfortable using some of them. So how can we expect to see changes in the students right away?" "That's what I was trying to say," interjected Carmen. "And I am not even sure how to assess what I am expecting. I don't mean that I do things without a purpose, but some of what I want my class to be doing will not show up on any kind of test that I know of." "You're right. But I can see how it would help if we had some sort of image of what we expect to see students doing. Something more than goals and objectives -- a sort of picture that we can all share, so we know we are all talking about the same thing. Restructuring and learner-centered schools are such broad and general terms that they can mean different things to different people. What do we mean?" said Anna. "Think about it. Columbus might never have received any support at all if he had actually been heading for parts unknown. But his stated destination was the Indies, and people at that time did carry a similar picture in their heads of what they thought China looked like, even though they had never been there. We need some picture like that." "I like that analogy. To keep us and others going until we can see the actual results, we need to agree on a picture of what a classroom in a restructured school will look like, a picture of what students and teachers in these places will be doing," Paul agreed. "What you're talking about is what makes restructuring so difficult for me. We don't yet have a picture of what we expect classes and students and teachers to look like. When a parent asks me, I can describe cooperative learning, but I have trouble going much deeper. Still, I guess you're right that we need clear shared images of what restructured classrooms will look like.
Many of the reform efforts schools are engaged in today are designed to achieve long-term effects. Thus the improvements are not immediately obvious and may not be obvious for years to come. Moreover, these change efforts are asking teachers and learners to interact in ways very different from those of traditional classrooms, and the expected results demand different indicators of success. At the same time, however, parents, funders, policy makers, and taxpayers are demanding to see results that they can understand. And they want to see them soon. The demise of a reform initiative is often due not to its ineffectiveness, but rather to its inability to deliver immediate results, however inappropriate or impossible that expectation might be. For example, the Pew Charitable Trusts recently terminated a multimillion-dollar children's initiative because of an assessment that it was not likely to meet its goals within the limits of allotted time and resources.1 Proceeding with the planned initiative did not seem politically feasible. Despite the uncertainties of the process, those involved in the work of school reform must be able to keep the faith until the outcomes are obvious. In this article we show how educators can do just that by making explicit what eventual outcomes are expected and by demonstrating how specific activities will lead to these longterm outcomes. Program evaluation plays an important role here by helping to define the outcomes and by chronicling progress through the use of intermediate benchmarks. In her 1987 summary of lessons learned from studying 20 years of local implementation of federal policy changes, Milbrey McLaughlin concluded that policy makers cannot mandate what matters. To be successful, policy changes must rely on local capacity and local will. Teachers and administrators must develop the capability of acting in the ways a new policy calls forth, and they must have the will to make the changes.2 More recently, she has added a third necessary ingredient: "getting it," that is, knowing what the new vision looks like. For us, "getting it" means having a concrete image of what you are working toward, having a deep understanding of the envisioned change. For those on Columbus' three ships, that image was a mental picture of the Indies, a vision they could cling to as they endured long months of looking at nothing but water. Some educators are developing a concrete image of what their restructured schools will look like and of how they can use evaluation to sustain the faith of those within and outside the school until the desired outcomes are obvious. Their efforts reveal reforms that are working. Getting It Clarity of vision is considered the hallmark of a remarkable leader and an absolute necessity for school reform.3 Helping all members of an organization share that vision is the first step needed to change daily practice. To have the will to increase their capacity, educators must be able to see where reform will take them. Yet how does this happen? How does a principal come to "see" what critical thinking looks like? Who can describe what a learner-centered school looks like? How would you recognize a "coach" in a classroom? How do you conjure up a concrete image of the learning situations you are trying to create in a restructured school? How do you share the images with those outside the building who must support the reforms-the school board, parents, funders, the state education agency, and the legislature? Conjuring images requires action. Curiously, people need to make a move in order to see where they are going. Michael Fullan has described the process of acquiring a concrete image as proceeding through the stages of "ready, fire, aim." He alters the expected sequence so that innovators can "fire" before they have perfected their aim. "As people talk, try things out, inquire, retry-all of this jointly- people become more skilled, ideas become clearer, shared commitment gets stronger."4 The action of firing gives practitioners the substance for their reflection and inquiry, which in turn leads to new plans for action and brings about a change in beliefs. Ready. The staff of the Regional Laboratory for Educational lmprovement of the Northeast & Islands worked with a group of schools that have explicitly undertaken the charge to become learner-centered. Laboratory staff members saw these schools move through the stages in Fullan's order. During the "ready" stage, educators gather knowledge about an innovation from others, often by going to workshops or reading about the work of national experts. At first, they are passive recipients of the knowledge constructed by others; they are catching up on what is already known. They become less passive as they conduct strategic planning and identify their mission and goals. Still, their work is largely in words, not yet connected with nor tested by reality. Fire. At the ''fire'' stage, local educators try out a specific version of the generalized innovation in their local setting. For example, they choose a specific innovation to try, such as multi-age grouping or integrated curriculum. As they converse about how they would apply the general idea to their local site, with its unique history, constraints, and personalities, they are beginning to construct an image of their new school. They answer the question "How will it look?" by testing a mental construction. As Fullan points out, "fire" involves both action and inquiry, both trying and talking. In this phase, educators are actively constructing their own knowledge about the innovation as they interact with one another. For example, one of the schools working with the regional lab decided to adopt the middle school model described in the Carnegie Council report Turning Points.5 After reading and discussing the material, staff members "fired" quickly, regrouping students into teams and instituting a behavior code. Their attempt to regroup students didn't "take," because it had no grounding in a locally shared purpose. However, their attempt did provoke intense discussion of what kind of student grouping would be best for them and why one choice would work better than another. These teachers and their principal are now designing a middle school that is grounded in their own beliefs about what their students need. Their "misfire" has helped them aim. Aim. At the "aim" stage, educators reflect on what they have done and evaluate its merits. Some external eye, such as a program evaluator's, can be especially important at this stage to help clarify the image. Is this what we wanted? Is this how we thought it would look? What's different about the way students are acting? How are teachers different? Do they interact with youngsters differently? Is this better or worse than what we imagined? What unexpected things happened? Why did we do this? Do we like it? Why? This reflection takes the form of conversations among the practitioners who carried out the innovation. These conversations can lead to the big "ahas," those great insights that bring a flash of clarity. As people talk, they resonate with one another, get excited, build on one another's ideas, discover synergy, and collectively construct an image that no single person had beforehand. They engage in what has been called "real talk," an optimal setting in which "half-baked and emergent ideas can grow." 6 Such insights help improve the "aim" next time. At this stage, educators are interactive learners, building their capacity by generating a collective knowledge and wisdom to guide subsequent actions. As they determine how to aim, they are deepening their understanding of the envisioned change; they are "getting it." Clear images are emerging. Notice that they cannot be clear about their image until they have taken some action and reflected on the experience. Keeping the Faith As educators move through the sequence of "ready, fire, aim," they create a change process that creates "products" along the way. Both the change process and its products can be evaluated to identify weaknesses and to determine whether progress toward the envisioned future is taking place. Charting progress can help to maintain everyone's faith in the reform effort. Intermediate versus long-term outcomes. In charting their progress, educators must first distinguish between long-term outcomes and intermediate outcomes or benchmarks. That is, they must clearly distinguish the eventual goal of their journey from the milestones along the way. As Phillip Schlechty has described it, they must distinguish between a goal of restructuring and a restructuring goal.7 The goal of restructuring is improved student learning, whereas a restructuring goal could be improvement in the type of assignments teachers give to students. Improvement in the nature of teacher assignments is an intermediate outcome that is expected to lead to the ultimate outcome of improved student learning. The intermediate outcome is a benchmark to use to measure progress on the way to the ultimate outcome. Checking on and publicizing progress toward benchmarks is a way for the participating educators to maintain their faith in an innovation and avoid getting discouraged. Furthermore, providing such periodic information to those watching the change process from the outside helps sustain their interest and support and helps them to understand why the eventual outcomes cannot be achieved immediately. The intermediate outcomes chosen for benchmarks need to fit the particular initiative. For example, if an expectation of restructuring is that teachers will act for the benefit of the whole school (rather than stay divided by categorical programs or disciplines), then an appropriate benchmark might be the degree to which teachers collectively plan for and evaluate instruction across all subject areas. 8 Including teachers of special education, bilingual education, the arts, and other disciplines as part of the regular instructional teams provides evidence that the school is making progress toward the intermediate outcome of working together. Thus it is reasonable to expect that this is one step on the road to the eventual outcome of improved student learning. The intermediate outcome is visible and attainable and demonstrates progress toward the less visible, more visionary outcome. As another example, the school improvement team of an urban learner-centered school developed a behavior code for students. In the role of critical friends and evaluators, staff members of the regional lab asked three sets of questions. The first set focused on the purposes of the behavior code and its relationship to student learning. If the team could articulate why the code was important to student learning and was satisfied that the connection was a strong one, the researchers moved on to the second set of questions. These questions focus on the implementation of the code itself: Are students following it? What sense are students making of it? What changes are occurring in the school as a result of implementing the code? Finally, how do you know this has made a difference? What kind of evidence can you gather? Conversations as benchmarks. Developing a process through which educators can change themselves is critical to sustaining reform efforts.9 This change process is heavily dependent on conversations. As Tony Wagner describes it, "The real methodology for system change begins and ends with ongoing, authentic conversations about the important questions." size="-1" face="Arial">10 The important questions focus on what participants want the school to become and what steps are needed to get there. After the middle school "misfired" in its first attempt to regroup students, the teachers began intense, focused discussions about what they wanted from their new grouping practices. That the discussions occurred where there had been none before is evidence of progress. As educators evaluate progress toward their envisioned future school, they can use several layers of conversations as benchmarks. For example, since conversations are an essential step in the change process, the regional lab brought together teachers and principals in different learner-centered schools for ongoing, focused discussions about an issue of mutual interest. These discussions were intended to spark further conversations back in each school, which would reflect new understandings of how students learn. These discussions, in turn, should lead to different conversations between teachers and students, different kinds of assignments given to students, and different conversations among students as they work on the new types of assignments. The program evaluator can watch for changes in both the process and the products. Changes in process can be observed in the four levels of conversation: teachers from different schools, teachers within schools, teachers and students, and students and students. Changes in products can be noted in terms of the assignments given to students (teacher product) and the student work in completing each assignment (student product). Evidence to evaluate progress toward the ultimate outcome of student learning can be gathered by checking for these intermediate process and product outcomes. Articulating what is expected from all these conversations provides what Robert Evans refers to as "a bridge to the future." 11 It provides a road map of how changes in teachers' understanding and actions (as shown in their conversations with and assignments to students) are expected to change student learning. In sum, while we are only beginning to see evidence of improved student learning as a result of some school reform efforts, we believe that we are looking at reforms that are working. We recognize that change processes are complex and take time. Thus, rather than look prematurely for evidence of long-term effects, we focus on developing concrete images of what we are aiming for and on identifying intermediate outcomes. Images of where we would like to end up are necessary before we begin the quest. Few people are willing to embark on a journey without a sense of the destination. Although the images change as the journey progresses, they serve as a kind of standard. Conversations among the travelers both create the initial image and shape the changing image as the work proceeds. Conversations and the new products that result from them serve as evidence that changes are indeed occurring. Student work also serves to demonstrate to external audiences what the reforms are producing. The broader communities that surround our schools want and deserve to know what teaching and learning in schools look like today. The intermediate products that we identify as benchmarks can help to answer their questions. Together with the images of what schools can become, they serve to maintain the faith of all involved so that educators can continue the work of reform until the ultimate outcomes are clear.
1. Deborah L. Cohen, "Demise of Pew Project Offers Lessons to Funders," Education Week, I June 1994, p. 1. SHARON F. RALLIS is a lecturer on education at the Graduate School of Education, Harvard University, Cambridge, Mass. NANCY C. ZAJANO is director of the Legislative Office of Education Oversight, Ohio General Assembly, Columbus. They wish to thank Gretchen Anderson for her comments on an earlier draft of this article. The work discussed here was funded in part by the Office of Educational Research and Improvement, U.S. Department of Education. However, the opinions are those of the authors. |
|
|
