Kappan Home PDK Home

Making Sense of Educational Renewal  By Kenneth A. Sirotnik Renewal is not about a point in time, Mr. Sirotnik points out. It is about continuous, critical inquiry into current practices and principled innovation that might improve education.

LIVING A LIFE is not typically a simple, straightforward affair. Take your own, for example. Suppose you wanted to make sense of it -- in fact, evaluate it. As a reader of this journal, you have probably had several decades or more of interesting experience to consider. Would you hire an outside evaluator? Would you identify your initial goals, develop outcome measures, and assess yourself? Would you find another life, matched to your own, so that you could make judgments by comparison? Would you submit to the bottom-line questions: Has your life so far made a difference, and what's the evidence? Would you follow the admonition to keep it simple, perhaps get it down to a single indicator that most others could understand?

Well, perhaps you would answer some very qualified yeses to these questions. For example, you might appreciate the perspectives of a psychologist, of critical and sympathetic friends, and of others who you think know you very well. You might reflect on what you had hoped for in life and how far you've come. You might even have some measurable goals, such as having a family, owning a home, being out of debt, having a job you like, and vacationing at least four weeks out of a calendar year. You might also have some people in mind whose lives are like what you imagined for yours, and you find yourself making comparisons. (I doubt, though, that you could roll it all into a single indicator.)

Yet I would like to suggest, at least for the purposes of this article, that a life worth living is not well appraised by the kind of conventional evaluation paradigm suggested by the questions above. Making sense of your life is worth more profound and complex attention. And the importance of the task escalates the more you live your life in a renewing way.

Renewal is about process, not about anticipated outcomes at certain points along the way. To be sure, there are events and critical incidents in our lives, but their sum is not the best (or even a good) indicator of our worth. In talking about both self-renewal and a renewing society, John Gardner reminds us that "we are talking about processes of great complexity -- but not so complex as to defy understanding."1 Of course we all grow and change in some way or another, yet renewal is much more than maturation; it is how we mature. Referring to a self-renewing society, Gardner argues that a "society whose maturing consists simply of acquiring more firmly established ways of doing things is headed for the graveyard -- even if it learns to do these things with greater and greater skill. In the ever-renewing society what matures is a system or framework within which continuous innovation, renewal, and rebirth can occur"2 (italics in the original).

This is a much more ecological way of thinking about life and social organization, and it places much more emphasis on chance, serendipity, unanticipated circumstances and events, and ongoing interpretation rather than on predetermined goals, planned interventions, designated outcomes, and causal explanations. Again, the focus is on process, as Gardner points out by quoting Arnold Toynbee: "Civilization is a movement . . . and not a condition, a voyage and not a harbor."3

The voyage, however, is not without direction. There can be no renewal, whether of individuals or society, without purpose, without some sense of fundamental values about what is better or worse in individual and collective living. Now, before I have postmodernists attacking me from all directions (not that anything I say will stop them), I am not suggesting some kind of god's-eye vision of what is true, just, and beautiful for all people for all times. But I am also no advocate of relativism. Some values are better than others, even if they are not automatically operational and have to be interpreted and reinterpreted historically, contextually, culturally, and demographically. Such ideas as freedom, social justice, and democracy seem better to me than tyranny, white supremacy, and fascism, even though we must continually interpret and live up to cherished ideals like social justice in this democracy of ours.

Consistent, then, with the discussion so far and with Gardner's analysis of self-renewal, people leading renewing lives would be more apt to 1) recognize and break out of ruts or patterns, 2) reflect on and carry out an ongoing process of self-inquiry, 3) see education as a lifelong process, 4) embrace failure as one of the best of all possible learning experiences, and 5) be capable of mutually respectful, just, and caring relationships with others.4 Not coincidentally, these features are consistent with what can be argued to be the moral dimensions that ought to guide those who accept the responsibility to educate others in a democratic society.5

Considering the challenges presented by this list, making sense out of a life, especially a self-renewing life, would be a complex undertaking, indeed. Notwithstanding the possibilities of constructing more conventional benchmarks and assessment strategies along the way, I submit that only an ongoing, formative, interpretive process based on critical and reflective inquiry would capture the real story -- the story about the whys, whats, and hows of a lived experience. If I were to try to get methodological about this in a more formal sense, I would be advocating longitudinal designs, including journaling techniques, document analysis, reflective discussions with knowledgeable others, critical questioning of assumptions and guiding interests, and interpretive techniques to further understanding of the whole. But you and I don't typically make sense of our lives in such formal ways.

Now, when it comes to organizations, we sometimes forget that there are people in them leading complex lives, personally and professionally. We conjure up theories of organizations and organizational change as if the organization itself were a thinking and acting entity that somehow innovates (or doesn't) as a function of the average of its inhabitants and its interactions with its social, political, and economic environment. Of course, there is some truth in this formulation. Yet, as with the attempt to objectify an individual life, it overlooks the deep ecology of organizational living, especially that of an organization -- and its members -- seriously engaged in renewal (as opposed to reform).6

Let's focus on educational organizations, particularly formal ones like schools, colleges, and universities. Consistent with my discussion so far and with the themes being addressed in this issue of the Kappan, my take on the differences between education reform and renewal is simply this: "Reform" is about whatever is politically fashionable, pendulum-like in popularity, and usually underfunded, lacking in professional development, and short-lived. Renewal is about the process of individual and organizational change, about nurturing the spiritual, affective, and intellectual connections in the lives of educators working together to understand and improve their practice. Renewal is not about a point in time; it is about all points in time -- it is about continuous, critical inquiry into current practices and principled innovation that might improve education.7

The serious recognition of differences between renewal and reform does not appear to be a primary feature of the contemporary education scene. I say this based on my appraisal of the still-popular practices of trying to reform education through mandates and accountability schemes that are largely empty in terms of the necessary resources and commitments to build organizational and personal capacities to innovate and change. Thus, when we study the effects of such attempts, we rediscover what we've already learned over the past three to four decades of similar studies: innovations and changes, especially the promising ones, rarely get past the classroom door or beyond the level of tinkering around the edges, which teachers do largely in order to figure out how to accommodate new accountability systems and outcome tests of one type or another.

This is not the teachers' fault. This is a systemic problem, politically, socially, and economically. For example, no major corporation trying to stay competitive would fail to invest at least 7% to 10% of its budget in professional development (i.e., inservice education). When it comes to public education, however, the bottom-line effect of no commitment to the enterprise is reflected in the less than 1% of budgets typically available to spend on professional development. It is not really surprising, then, that researchers studying the effects of California's attempt to change mathematics education almost a decade ago found very little actual pedagogical change in classrooms, even on the part of teachers familiar with the NCTM standards.8 Now, researchers looking for similar effects in states implementing new systems of content standards and performance assessments are finding the same result.9

The fault is not necessarily with the reform ideas either; there is considerable merit in many of them. Once again, however, we would never know, because they never get implemented with the kind of fidelity required in a long-term process of innovation and school renewal with a critical, reflective inquiry process to match. Instead, the reform agenda often becomes immediately politicized and polarized (phonics versus whole language, bilingual education versus English as a second language, the basics versus higher-order thinking, and so forth). Moreover, the agenda (and, necessarily, any traditional evaluation design) is complicated no end by myriad other pressures, activities, and programs that are part of educational organizations.

For example, I know of one middle school in my own state of Washington that is trying to experiment with interdisciplinary curriculum, team-teaching, cooperative learning, and an alternative block schedule other than the five-period-per-day routine. Not all teachers are involved, and some are actually resisting changes. All of this is happening in the state context of major, standards-driven reform and performance assessment based on content standards that are quite different (especially those for mathematics) from what currently drives the curriculum. Some teachers are now trying to adjust their teaching to these standards so that students can pass the tests.

Even the best and brightest evaluator would have trouble coming up with a good design to assess the effect of reform in this school. What is the "treatment"? What are the independent and dependent variables? Yes, there are very sophisticated multivariate approaches, and lots of variance can be partialed out and inferences made over some period of time. But little would be learned about how and why anything happened, and within several years anyway, in a reform environment, this school will probably be reacting to new leadership, new issues, different pressures, and alternative fads.

"Reform" typically breeds "accountability" as the primary evaluative medium. Although important distinctions can be made between accountability and evaluation, they tend to blur in high-stakes reform environments seeking to reward or punish institutions and their members. While they were engaged in some creative and cutting-edge thinking about evaluation in the late Seventies -- thinking that is still ahead of its time -- Lee Cronbach and his colleagues posed some interesting theses. For example, they asserted that a "demand for accountability is a sign of pathology in the political system. . . . Accountability emphasizes looking back in order to assign praise or blame; evaluation is better used to understand events and processes for the sake of guiding future activities."10

As I have argued elsewhere, thinking about renewal rather than reform as the operative change model suggests a major deconstruction of traditional notions of accountability.11 Responsibility is a more useful concept than accountability; it suggests, for example, the moral obligations of educators as stewards of their schools to create and nurture learning environments for their students as well as for themselves. And it suggests that educational and political leaders should provide the necessary resources and time to help make this happen. In a renewing educational organization, then, the "program" to be "evaluated" becomes the renewal process itself, and "evaluation" becomes the ongoing process of rigorous and active self-examination, reflection, and critical inquiry.

Among the more enduring and compelling voices on these matters is that of Seymour Sarason, who continues to point out "the predictable failure of educational reform" when political and educational leaders do not heed the decades of accumulated wisdom on what it means to create and sustain new settings.12 "Creating new settings" is Sarason's phrase for the kind of long-term renewal I've been describing, whether stemming from a brand-new initiative (e.g., building a school/university partnership) or from an effort to change current practices in fundamental ways (e.g., detracking a secondary school). What Sarason told us in the early Seventies still rings true today, namely, that the creation of settings gives rise to a host of "related problems which, despite their frequency and importance, have remained relatively unrecognized and undiscussed to a surprising, if not amazing, degree."13

What Sarason means by this is not that we don't know anything about complex renewal and change. Rather, what we do know is flat out ignored by reformers, either deliberately or through ignorance. Moreover, to know more would require long-term commitments to critical inquiry and action research methods of the type alluded to above. More recently, he tells us:

The creation of a new setting brings in its wake several predictable problems. These problems include the following:

• The creators almost always underestimate the complexity of creating a new setting. It is the opposite of an easy, smooth, untroubled process.
• They also underestimate the time factor, an underestimation that too frequently pressures them to make compromises and to gloss or seal over conceptual and interpersonal problems that will later come to haunt them.
• Imbued as they are with enthusiasm about being able to create a new setting, the creators cannot deal with the fact that resources are always limited, with the result that when reality confronts them with that brute fact, they are not prepared to deal with it.
• If anything is predictable about creating a new setting, it is that unpredictable problems will be encountered -- for example, something happens to the leader, the inadequacies of this or that staff member become manifest, outside opposition or criticism appears.14

Serious recognition of the predictable problems in educational renewal requires a serious critique of traditional evaluation methods and a willingness to experiment with more ongoing, reflective, critical, interpretive methods. It requires more of an action research model that involves participants in understanding their own renewal process. This can happen more or less rigorously and more or less deliberately, but in any case, methods will be much less oriented to specific, pre-identified objectives and outcomes and much more sensitive to how things change, planned or otherwise, over the long haul. Quantitative, qualitative, and critical methods are all relevant here; I'm talking more about how various methodological approaches are used than about whether one approach is more useful than another.

The complexity of educational renewal is evident whether focused on change in a single school or on a more ambitious effort to stimulate new relationships and ways of doing business within and between institutions, such as school/university partnerships. My colleagues Pam Grossman and Sam Wineburg, for example, are involved in an exciting renewal project designed to create an interdisciplinary learning community in a high school -- not even in the whole school necessarily, just among the teachers in two potentially related departments, English and history. In their words, "Our goal is nothing short of changing the intellectual environment in which teachers work. In contrast to the quick-fix culture of staff development, our project tries to create an ongoing venue for teachers' learning -- not an isolated feature during a summer institute or a one-shot workshop whose only trace is a shiny binder."15

Playing off the words of one teacher, their description of how the project unfolded as "a painfully necessary process" is telling. What with difficulties releasing teachers, working around uncertainties of school scheduling, and coping with the other cultural regularities of schools that are antithetical to inquiry and reflective practice, Wineburg and Grossman came to recognize that alternative evaluation methods were necessary: "Our attempt to study this process has taught us an important lesson: real progress does not come neatly packaged for display in bar charts and growth lines. We have learned that progress is more akin to making meaning from a Faulkner novel, a process filled with switchbacks and blind curves, ultimately satisfying but long in coming."16

The search for generalizations in the sense of lawlike propositions that can be packaged and transferred from setting to setting is neither possible nor desirable in these kinds of renewal efforts. To be sure, we can learn from these efforts and share this learning with others. But this is generalization of a much different sort. It is building heuristic understanding, developing and refining ideas that others can play with and reconstruct in their own settings.17 By accumulating what might be called cases of understanding, we can have an ever-expanding source of examples from which others can learn.

So, if you are an evaluator and are looking for highly definitive, generalizable, cause/effect relationships between measurable, high-validity, independent and dependent variables, then I suggest you go to work for a pharmaceutical company -- and even then, you will have enough methodological challenges to keep you busy for several careers. However, if you have a high tolerance for ambiguity and like to mess around with too many variables (and interactions) to measure (or measure well), with interventions or "treatments" that are usually ill defined and often hopelessly confounded with others, with social/political/economic/organizational contexts that are always colliding and changing, with ideological wars around what ought to be the very purposes and functions of the enterprise you think you are studying, and with pages and pages of text that represent what you see, what people write, and what people say in these settings that are engaged in long-term processes of renewal and change -- well then, have I got a career for you!

I do not want these comments to suggest that well-designed, conventional research studies are impossible or of no value in understanding educational renewal. I believe that there are, in fact, good examples of evaluative studies that, although not definitive, are certainly informative regarding the possible effects of major renewal efforts. For example, renewal programs that attempt to build coalitions of educational organizations to guide long-term change pose innovative complexities that play havoc with conventional evaluation paradigms. Yet much that is useful can be learned by synthesizing the findings of studies on similar efforts (i.e, programs driven by similar principles and beliefs), which often include some of the same educational institutions as the network in question.

A good example of this is the Coalition of Essential Schools and the several national studies of restructuring that either included some of these schools or were guided by similar propositions for innovation and change.18 Although the news was generally good for the Coalition's change agenda, it is still very clear that when it comes down to any given school, there is much that is not known and much left to do. The hard work of school renewal in the trenches, and the many ways that such work is manifested in the life of a school, is never adequately captured by a handful of outcome indicators. And this would be magnified exponentially when it comes to making sense out of the role of the Coalition itself and attributions as how and if "it" made a difference. Anyone intimately involved in the Coalition or Coalition schools knows that much has been accomplished and is still being accomplished. But to understand the whole story would likely require a good deal of interpretive unpacking of years of work (and documentation) between and within schools, and the artistry of telling this story in ways that go well beyond measurable (and necessarily reductionistic) indicators.19

Unfortunately, the notion of storytelling carries with it the connotation of fabrication. That is not what I have in mind. What I have in mind is capturing the wonderful tangle of purpose, activity, unexpected twists and turns, and beliefs and motives that must be interpreted if one is truly bent upon understanding human endeavor. So I return to where I began. I think we can learn a lot from what it means to try to understand just one interesting life. Mary Catherine Bateson tries to do precisely that -- for each of five lives, actually -- in her book Composing a Life. And in doing so, she comes to realize that each life presents a text "where energies are not narrowly focused or permanently pointed toward a single ambition. These are not lives without commitment, but rather lives in which commitments are continually refocused and redefined. We must invest time and passion in specific goals and yet at the same time acknowledge that these are mutable."20

Making sense of a life is no mean task. Making sense of the lived experiences of people and organizations in a project of educational renewal and change is more challenging still. To do so will require the courage to resist pressures to make what is complex simple and to rely only on inquiry paradigms with limited potential to explain and understand this complexity.

2. Ibid., p. 5.

3. Ibid., p. 7. The quote is from Arnold Toynbee, "The Graeco-Roman Civilization," in Civilization on Trial/The World and the West (New York: World Publishing Co., 1958), p. 50.

4. See Gardner, chap. 2.

5. Kenneth A. Sirotnik, "Society, Schooling, Teaching, and Preparing to Teach," in John I. Goodlad, Roger Soder, and Kenneth A. Sirotnik, eds., The Moral Dimensions of Teaching (San Francisco: Jossey-Bass, 1990), pp. 296-327.

6. More detail on the ecological metaphor for renewal and change can be found in Kenneth A. Sirotnik, "Ecological Images of Change: Limits and Possibilities," in Andy Hargreaves et al., eds., International Handbook of Educational Change (Boston: Kluwer Academic Publishers, 1998), pp. 181-97.

7. "Critical inquiry" can mean many different things to different people. For me it is an informed process of equitable communication among all stakeholders that encourages explicit and constructive debates about underlying assumptions and human interests. See, for example, Kenneth A. Sirotnik, "Critical Inquiry: A Paradigm for Praxis," in Edmund C. Short, ed., Forms of Curriculum Inquiry (Albany: SUNY Press, 1991), pp. 243-58; idem, "Evaluation in the Ecology of Schooling: The Process of School Renewal," in John I. Goodlad, ed., The Ecology of School Renewal: 86th NSSE Yearbook, Part I (Chicago: National Society for the Study of Education, University of Chicago Press, 1987), pp. 41-62; and Kenneth A. Sirotnik with Jeannie Oakes, "Evaluation as Critical Inquiry: School Improvement as a Case in Point," in Kenneth A. Sirotnik, ed., Evaluation and Social Justice (San Francisco: Jossey-Bass, 1990), pp. 37-59.

8. See the entire issue of Educational Evaluation and Policy Analysis, Fall 1990.

9. See William A. Firestone, David Mayrowetz, and Janet Fairman, "Performance-Based Assessment and Instructional Change: The Effects of Testing in Maine and Maryland," Educational Evaluation and Policy Analysis, vol. 20, 1998, pp. 95-113.

10. Lee J. Cronbach and Associates, Toward Reform of Program Evaluation (San Francisco: Jossey-Bass, 1981), p. 4.

11. A fuller discussion of the move from accountability to responsibility can be found in Kenneth A. Sirotnik and John I. Goodlad, "The Quest for Reason Amidst the Rhetoric of Reform: Improving Instead of Testing Our Schools," in William J. Johnston, ed., Education on Trial: Strategies for the Future (San Francisco: Institute for Contemporary Studies Press, 1985), pp. 277-98.

12. See, for example, Seymour B. Sarason, The Predictable Failure of Educational Reform: Can We Change Course Before It's Too Late? (San Francisco: Jossey-Bass, 1990); and idem, Political Leadership and Educational Failure (San Francisco: Jossey-Bass, 1998).

13. Seymour B. Sarason, The Creation of Settings and the Future Societies (San Francisco: Jossey-Bass, 1989), p. 1.

14. Sarason, Political Leadership, p. 121.

15. Sam Wineburg and Pam Grossman, "Creating a Community of Learners Among High School Teachers," Phi Delta Kappan, January 1998, p. 350.

16. Ibid., p. 352.

17. See, for example, Robert K. Yin, Case Study Research: Design and Methods (Newbury Park, Calif.: Sage, 1989), pp. 43-45.

18. A good example of this kind of storytelling is Theodore R. Sizer, Horace's Hope: What Works for the American High School (Boston: Houghton Mifflin, 1996); see also Patricia A. Wasley, Stirring the Chalkdust: Tales of Teachers Changing Classroom Practice (New York: Teachers College Press, 1994). My colleagues and I will be struggling with these issues ourselves as we attempt to make sense of the last 15 years of renewal activities forming the mission of the Center for Educational Renewal's National Network (of school/university partnerships) for Educational Renewal.

19. Margaret M. MacMullen's approach is illustrative; see Sizer, Horace's Hope, Appendix B, pp. 160-76.

20. Mary Catherine Bateson, Composing a Life (New York: Atlantic Monthly Press, 1989), p. 9.

PDK Home | Site Map
Kappan Professional Journal
Last updated 15 April 1999
URL: http://www.pdkintl.org/kappan/kksir9904.htm
Copyright 1999 Phi Delta Kappa International