Promoting Responsible Accountability in Schools and Education Mr. Sirotnik envisions the day when neither his high school friend Helen nor any of our nation's young people would be punished on the basis of a test-driven system that is ill suited to their abilities and ambitions and to the broader purposes of public schooling in our multicultural and democratic society. By Kenneth A. Sirotnik   Illustration by Jem Sullivan

LATELY, I've been thinking about my friend Helen. We went through junior and senior high school and college together. She was a good student in all but one subject, mathematics. In fact, she was brilliant in English, but she was absolutely stymied when it came to mathematics. Helen would have liked to be better at math, but it really didn't bother her that she wasn't. Somehow she made it through the basic math classes in high school without flunking out and was admitted to and graduated from UCLA with a B.A. in English and a secondary teaching credential. Helen is now one of the most successful and honored English teachers in Los Angeles. She loves teaching, and, after more than 30 years, she still has no plans to retire. She has raised two children; aside from a few traffic tickets, she has never violated any laws; she votes in elections (and takes them seriously); she doesn't use illegal drugs; she contributes time and money to social causes; and she looks after her aging parents.

There's much more I could say about my friend Helen. But it's instructive to note that if she were to be a high school student preparing to graduate in the state of Washington in 2008, she wouldn't be able to. No matter how many times she took it, Helen would never have passed the mathematics test, which is part of the "certificate of mastery" that will be required for graduation. Would she have dropped out of high school, knowing she would never graduate? Would she have gone on to complete a GED at a local community college? Could she ever have become an English teacher? And could she have made an important difference in the educational lives of some 9,000 young people over a 30-year career?

Don't mistake my point in sharing Helen's story. It is not about gender or about one subject being more or less important than another. Helen might have been a whiz in mathematics instead of English, or I could have reminisced about my friend Hal instead. I share Helen's story because I care deeply about the well-being of each and every young person in today's and tomorrow's public schools. And because of this concern, I have serious reservations about the punitive nature of traditional accountability practices, particularly as these practices have been playing out in the current high-stakes policy environment.1 For example, high-stakes accountability testing has exacerbated the already problematic dropout rates of marginalized students.2 Evidence is emerging about teacher demoralization and attrition as a result of frustration with the overemphasis on mandated testing for high-stakes accountability purposes.3 And movements, some court-based, are already occurring in several states to counter the fallout from high-stakes testing and accountability practices.4

The acronym suggested by the title of this article is no accident. I wish to share here what might serve as a foundation for PRAISE, not punishment -- a foundation for rethinking substantially what it means to think and act responsibly when it comes to calling our schools, school systems, educators, and students to account. My colleagues and I are currently trying to "think out of the box" regarding what ought to constitute appropriate accountability practices.5 We are trying to counter the prevailing rhetoric of high-stakes accountability and to promote, instead, a new rhetoric of educational responsibility. We are asking for more caring and for justifiable educational practices that serve the interests of all children and their families. At the same time we are calling for finding creative and useful ways to demonstrate publicly who students are, what they know, what they care about and are able to do, and what they can become. We are arguing for holding our political systems accountable as well, for expecting them to "walk their talk" by providing the necessary resources to truly leave no child behind, especially those children of poverty and of color who are typically forgotten along the way.

In sharing our initial thinking on these matters, we admit to not having worked out all the details for a more responsible accountability system. This is an easy admission to make, since it would be foolish and embarrassing to claim otherwise. Instead, we want to build on earlier work6 and present alternative concepts that we believe are more responsive to the purposes and functions of public schooling in a truly democratic society. We want, therefore, to propose ideas that are consistent with a set of beliefs and knowledge-based claims about good teaching and learning and the educational needs of students and their families, particularly in economically poor communities and in communities of color.

Beliefs

No human endeavor, especially one so intrusive as education, is without underlying beliefs and values, let alone interests and political positions. We are guided by a set of beliefs rooted in a vision of the purposes of public schooling in a political and social democracy.7 One such purpose, for example, is what Benjamin Barber has called "civic literacy":

This is the fundamental literacy by which we live in a civil society. It encompasses the competence to participate in democratic communities, the ability to think critically and act with deliberation in a pluralistic world, and the empathy to identify sufficiently with others to live with them despite conflicts of interest and differences in character.8

Thus we believe public education to be a moral endeavor, and we believe that the dimensions of this endeavor must be articulated in order to determine how the whole might be accounted for in a reasonable and responsible way.9 For our purposes here, the following seven belief statements and brief descriptions should suffice.

1. Public education plays a vital role in our pluralistic and democratic society. The very survival of a political democracy depends on a participating and educated citizenry. Our public education system must guarantee an equitable and excellent education for all the nation's young people, and our federal, state, and local policies must support and nurture the schools in this effort by helping to create and sustain the conditions within which the guarantee can be realized.

2. The functions of public education must be construed broadly to encompass the character and competencies of fully educated human beings, capable of filling multiple roles in our social and political democracy. Although schools must attend to career opportunities for all students, solely utilitarian or economic narratives should not control the purposes of schooling. There is no better preparation for the future than to cultivate the habits of lifelong learning in the present.

3. Government and the public have a right to know how well children are faring in our public education systems. To be accountable, according to the American Heritage Dictionary, is to be "liable to being called into account; answerable." Those who are responsible for educating our children, therefore, must be called into account by parents, communities, the state, and perhaps even the nation -- assuming these constituent groups are willing to fund appropriate and responsible educational practices.

4. Just as educators need to be held accountable, so do policy makers and the public as a whole -- for both the validity of the educational accountability systems they establish and the impact these systems have on equity and excellence in teaching and learning. A constitutional democracy demands that governing policies and those who make and enforce them be called into account by the people and by constituent groups. A society that is still marked by substantial racism and classism cannot expect just and equitable public schools no matter how much rhetoric is heard about better leadership, better teaching, and "closing the achievement gap." Schools and society are inexorably bound together, as they should be. Improving both requires the will and work to make both better.

5. A responsible approach to "being called into account" assumes that public school educators, parents, government officials, and others want to do the right things for our children, even though they may not always know how and are often overwhelmed by the problems they face. An accountability system must listen carefully to the people most involved in educational activities (teachers, parents, children, welfare workers, mental health workers, and workers in other community agencies) and must seek to develop, cooperatively, viable plans of action.

6. The distribution of resources in response to school- and community-based needs is not a fiscally or morally neutral event. Taking a moral stance will require the courage to operate on the principle that "equal is not necessarily equitable," that more resources will have to be distributed to the least advantaged schools and communities.

7. Accountability and responsibility must go hand in hand. Responsibility certainly includes accountability but also, according to the dictionary, includes the more layered qualities of being "able to make moral or rational decisions," being "trustworthy or dependable or reliable," and "showing good judgment." To be both responsible and accountable demands that we care deeply about the well-being of our children and that we bring the best ideas, the best knowledge, and the best practices and professional judgments to bear on the education of future citizens of our society and our world.

These seven beliefs form a foundation on which to build more responsible practices for holding accountable not only ourselves as a social and political democracy but also the political infrastructure supporting public schooling and the system of schooling itself. To develop responsible practices, however, also requires using what we already know about good teaching, learning, and assessment and about the conditions and circumstances within which good educational practices can flourish.

Knowledge-Based Claims

In Project PRAISE, we are proposing the following related clusters of knowledge-based claims to begin the process of thinking creatively about more responsible accountability practices.

First, it is essential to differentiate clearly between systems of assessment and systems of accountability. Assessment systems are about creating and using ways to collect information on teaching and learning and about making appraisals or judgments based on that information. Accountability systems are about what is done with these appraisals, and typically this has taken the form of rewards for or sanctions against students, schools, and school systems, based on a few test score indicators. Bringing responsible practice to the notion of "being answerable" will require "showing good judgment" and taking full advantage of the decades of knowledge already accumulated on these important matters.10

Second, good assessment is a natural part of good teaching and learning. Good assessment also takes time and is not cheap. Other than for reasons of economy and efficiency, there is little educational justification for using easily scored tests -- and only those tests -- to make high-stakes decisions about the educational well-being of children and their schools. No modern organization would ever use a lone indicator to judge the worth of its operation -- the GNP for the federal government, the average temperature of patients in a hospital, the number of barrels produced by an oil company, the average weight of cattle for a rancher, or the average test score for children in a school. No sensible hospital director would mandate more frequent temperature-taking to cure patients, and no governmental body would endorse more frequent computation of the GNP to improve the economy. Yet we find ourselves -- once again -- mandating more and more testing of students and expecting this practice to result in better teaching and learning.

There is much more to life in such complex organizations as schools than can be indicated by mandated, point-in-time measurements, and there is much more to a human being than can be assessed by a few tests. The ultimate misuse of current test-driven accountability systems is as a hurdle over which a child must jump in order to move forward in education and in life. Imposing graduation tests -- even the minimum-competency variety -- does not appear to accomplish anything except to promote higher dropout rates for those students who are already underserved by society and the schools (typically poor students and students of color).11 A responsible accountability system would be based on professional judgment using multiple indicators and assessments -- both quantitative and qualitative and over extended periods of time -- that are sensitive to the needs of each individual and to the purposes and complexities of schooling, including contextual conditions, schooling processes, and the outcomes of teaching and learning.

Third, although there is still much more to be learned, much is already known about better assessment practices that align with the curriculum that educators and policy makers think should be included in schools. A responsible accountability system will include many forms of assessment that tap directly into the actual performances that students are expected to demonstrate, especially performances that take place in classrooms. Such performances include reading, writing, speaking, problem solving, experimenting, inquiring, creating, persisting, deliberating, collaborating -- abilities that many of today's leaders in both the public and the private sectors say they wish were exhibited more strongly by the people they employ.

Moreover, what goes on after school is equally important, including what happens to students after they graduate from high school. Do they become decent people, good parents, good community members? Do they participate constructively in civic deliberation and democratic practice? Are they economically productive citizens? Have they continued to learn in the many ways that are possible to continue learning and growing as human beings? And much more.

Fourth, it is critical to overcome the well-known outcome of test-driven, high-stakes accountability: namely, the narrowing of what gets emphasized and how it gets emphasized in school curricula.12 Only the subjects tested -- and only the limited ways in which these subjects are tested -- receive the bulk of attention, much to the detriment of other valued goals and pedagogical practices. Study after study and poll after poll suggest that parents want much more for their children than what can be assessed by a few tests.13

Most important, the vitality of a democratic nation -- the ability of its citizens to participate thoughtfully and responsibly -- is obviously threatened by a narrowly educated public. Parents demand -- and the nation must demand as well -- that future citizens develop intellectually well beyond "the basics," that they are on the path to becoming vital and creative individuals, that they demonstrate the habits of mind of thoughtful and informed participants in a democracy, and that they are ready to engage their futures in productive ways for themselves and for the well-being of the nation and the global community. A responsible accountability system will include ways to assess all these outcomes in forms consistent with what we know about children and their developmental processes.

Fifth, educators and researchers have long known the importance of individual differences. Children develop differently and have different styles of learning. Thus a "one size fits all" policy of teaching, learning, assessment, and accountability is doomed to failure. Moreover, students' emotional and social maturation is every bit as important as their intellectual growth. Holding students back in grades because of their scores on narrowly defined tests given to all students at the same arbitrary point in their chronological growth makes no logical sense, and empirical research continues to demonstrate the futility of such practices.14

Furthermore, research also demonstrates that retention policies promote increased dropout rates and thus serve only to diminish the economic and social well-being of our society.15 A responsible accountability system would accommodate the needs for the public's right to know how well education systems are doing, while at the same time accommodating and promoting ways of powerfully addressing the developmental needs of the individuals these systems are obligated to serve.

Sixth, students need extensive, high-quality opportunities to learn. It is hard to believe that something so obvious would need to be researched, but it has been, and the findings have been consistent. More important, however, is the fact that, in spite of the well-worn rhetoric about providing both "equity and excellence" in our nation's schools, we do anything but. Many schools -- particularly those with high concentrations of children of poverty and children of color -- are vastly undersupplied (in both human and fiscal resources) to do the job reasonably well. Such widespread conditions as tracking and large school and class sizes further contribute to the decline of opportunities to learn. When clearly inequitable conditions exist from school to school (and from classroom to classroom within schools), there is no moral way -- and should be no legal way -- to argue that sufficient opportunities to learn obtain for all students.16

These are serious problems that reflect the social, political, and economic inequities of our struggling, pluralistic society. Students, teachers, and public schools are caught in the middle of this struggle, with the predictable outcome of inequitable opportunities to learn in schools that are considerably less than excellent. A responsible accountability system will find ways to negotiate and assess the very difficult but critical balance between 1) the constraints of deplorable conditions and circumstances within which many public schools and educators try to do their work and 2) the moral responsibility (notwithstanding these constraints) to do their work anyway -- with the care and stewardship required for a high-quality education for all students.

Seventh, also both obvious and research-based, is the fact that better teaching produces better results.17 However, the magnitude of resources required for professional development -- consistent with new developments in the disciplines and higher expectations for teaching, learning, and assessment -- is huge compared to the minuscule amounts provided in current education budgets. Successful corporations invest as much as 10 times the paltry 1% or so that school systems spend on professional development. Responsibility in an accountability system will be demonstrated not only by focusing on contextual circumstances, student outcomes, and meaningful opportunities for student learning but also by providing substantial and continual opportunities for teachers and administrators to develop their leadership and teaching capabilities.

Eighth, punishment and the threat of punishment are not productive ways to change behavior, either for individuals or for groups. There is little or no evidence from the research in behavioral psychology to support the use of punishment in getting people to change their behavior in positive and enduring ways, especially without considerable negative side effects.18 In fact, research suggests that coercive systems of schooling will cause a backlash of distrust and avoidance on the part of both teachers and students. In a critique of the conventional wisdom surrounding the use of punishment as a technique for controlling human behavior, Murray Sidman warns:

Entire environments can become reinforcing or punishing in their own right. Students who are reinforced by high grades, respect from their teachers, and admiration from their peers are likely to attend school regularly. Students who are punished by failing grades, disapproval and humiliation from the teachers, and lack of recognition or even contempt from their peers are likely to stay away from school as much as possible. The reliance on punishment places the stamp "coercive" on the whole system, and many young people find a major segment of their environment aversive. For students who are punished in class, school becomes a punisher. Instead of getting them to learn, punishment causes them to shun the environment where learning is supposed to take place, and perhaps even to avoid the whole process of formal learning.19

Policy makers, however, continue to believe (or at least to promote the notion) that the only way to improve teaching and learning is to have the threat of sanctions in accountability systems -- real "teeth" that have biting consequences, such as linking high school graduation to passing standards-based exams. Otherwise, they argue, teachers won't take reform seriously, nor will their students.

Notwithstanding the rather low estimation of teachers and students that such reasoning suggests, there is simply no evidence to support the logic of test-driven reform, even with punishment added. In many ways, we've been at this for decades in one form or another of traditional accountability systems, and apparently it hasn't worked -- at least not for sustained and positive improvement.20 Yet there seems to be something fundamental in the American psyche that gravitates toward punishment or the threat of punishment -- a kind of "spare the rod and spoil the child" ethic.

We don't buy this reasoning. Moreover, there is some recent evidence to suggest that good teaching and learning can be made to happen without having a high-stakes testing and accountability system in place. Connecticut's decade-and-a-half attempt to promote better-trained and better-paid teachers, ongoing professional development for educators, and continued attention to high-quality curriculum, including student assessment, appears to be showing results without having to punish students in any way for failing to achieve some standard of performance on any given test.21 A recent study of the motivational effects of high-stakes testing on low-achieving students concludes that such effects, if they occur, do so (not surprisingly) only under conditions that we already know characterize good teaching and learning. Even so, there are still substantial numbers of students left behind who require additional supports. The authors offer the interpretation that "any set of policies that put the onus on students will fall short in the end."22 It seems to us that a responsible accountability system must be built on trust and good will -- a bargain, as it were, that educators and students will do their best in exchange for the proper conditions and circumstances in which to do it.

Ninth, if we have learned anything from all the research on educational change, it is that to change institutions and institutional practices is neither simple nor immediate.23 Although there may be political urgency to produce quick results, meaningful change comes only from well-developed, deeply integrated social, political, and economic changes generally, as well as concomitant specific educational changes in resource allocation, curriculum, instruction, and organizational structures in schools.

All of this takes time, a lot of time. Responsible accountability systems will require a long-term focus. Rather than reach conclusions based on short-term results, a more truly educative model of accountability and change must prevail -- a paradigm that is formative and sees multiple forms of information at any point along the way as new evidence that informs present practices and guides even better ones for the future.

Challenging Deep-Rooted Axioms: A Thought Adventure

The question my colleagues and I face in our current work is how to bring all these beliefs and knowledge-based claims to bear on promoting more responsible accountability practices in education and schools. This is not your average task, and we are trying to learn as much from current and past writings and practices as we can.

One thing is quite apparent now, however. We do not believe that much can really change by tinkering around with and trying to ameliorate conventional accountability models. We have to challenge ourselves to think "out of the box," to identify and seriously question deep assumptions that have led, in our view, to the current cul-de-sac in traditional accountability practices.

Posing what is problematic may be among the more problematic undertakings in the world of social inquiry and action. For how we choose to pose, frame, or otherwise ground problems may well limit our ability to solve them -- or even to think creatively about what the problem is. Seymour Sarason has often reminded educators that we labor under axioms or world views that are so deeply rooted that they remain both unformulated and unchallenged. For example, Sarason formulates and challenges the axiom that "education best takes place in classrooms in school buildings" and poses the following question to help start an alternative conversation: "What if it were illegal to teach subject matter in a classroom in a school?" In engaging the thought adventure suggested by the question, Sarason reminds us that he is not out to formulate a blueprint of alternatives. In fact, his aims are quite the contrary:

A blueprint, whether for a building or an institutional change, has a concreteness that too frequently facilitates rejection: either the plan strikes one as strange or, as is sometimes the case, it gets a quick acceptance that deprives one of both the benefits of the sketching process and the chance to gain understanding and acceptance from those who will be affected by implementing the blueprint.24

In a similar vein, we need to challenge our own thinking when it comes to seeking truly meaningful alternatives to traditional accountability practices. So bear with me on what follows. My colleagues and I are still trying to work it out ourselves, but here is the gist of it.

Another of those unformulated and unchallenged axioms so deeply rooted in educational practice can be formulated as follows: accountability for public education must rely on test scores collected from students in sufficiently standardized fashion so that they can be aggregated upward to school, district, state, and national levels. A corollary axiom is that all of this is also necessary so that normative comparisons can be made between individuals, schools, districts, and so forth. Even when standards-based systems are ostensibly criterion-referenced, the results are often reported school-by-school in a format that begs for comparison.

But let us borrow Sarason's thought-provoking approach and ask, What if it were "against the law" to aggregate and compare student test scores? In other words, we could no longer give students tests for the express purpose of aggregating the results upward and computing means or percentages at the school, district, or state level. Creating fancy composites of multiple indicators for the purposes of aggregate accountability indexes would also be "illegal."25 Now, dear readers, if you are thinking I have lost my mind, perhaps it is so. But your reaction may also indicate just how deeply rooted our traditional assumptions are about how accountability must be practiced. So let yourself go, at least for the moment, and open up to the possibilities presented by this thought adventure.

To be sure, assessing students in all sorts of creative and useful ways would still be entirely legal and highly desirable. Some of the current standards-based tests are actually pretty good; they signify better curriculum and higher standards for teaching and learning. We could easily see ourselves as teachers making good use of these evaluation devices as an ongoing part of good instructional practices in the classroom.26 Finding out how well any given student can read, write, think, explain, interpret, communicate, create, solve problems, collaborate, tolerate, compete, invent, evaluate, perform, persist, and so forth would be essential, especially at the many times when doing such an appraisal for a particular student makes sense for that student. This is simply good practice in teaching and learning. Accumulating the results of such assessment for any given student tells us much of what we need to know about that student's learning and achievement. By the end of 13 years or so, we should know whether or not that student ought to graduate and get on with his or her life.27 Isn't this what we want for each and every young person in our public schools?

Yet think of all the problems we have created for ourselves with our obsession for test-driven accountability and the need to aggregate upward from individual-level information. For example, in order to aggregate and compare information, the information needs to be comparable enough to do so. You can't average "apples and oranges" -- test scores from one individual to the next have to "mean the same thing." Any thought of localizing accountability in a responsible way, therefore, presents immediate difficulties. We can't tailor assessment and accountability procedures and decisions to individual students; we can't leave these decisions to schools and districts -- if we did, how could we possibly make sense of potentially disparate information or even information on the same issues but defined or collected in different ways? The upshot is that we never even think about possible alternatives that seem to compromise our ability to aggregate information.

Parenthetically, adherence to the aggregation axiom explains in part the hegemony of quantitative data. No one has yet figured out acceptable ways to aggregate qualitative information (the kind in student portfolios) other than by begging the question and turning it into quantitative data by using scoring rubrics of some sort. And then, of course, comes the flood of psychometric problems associated with obtaining reliable and valid scoring for qualitative work (such as writing) and creating statistically equivalent forms of performance tasks from one year to the next. Moreover, the costs of scoring qualitative data are enormous and prohibitive for highly valid and reliable uses in large-scale assessment systems.

So we have standardized tests -- both norm- and criterion-referenced -- that have to be given to all students at the same time in the same ways so that the results can be aggregated and compared. We have a multibillion-dollar testing and psychometric industry that thrives on solving the problems created by the need to aggregate and compare, such as devising sophisticated scaling methods to create common metrics and equivalent tests, and no end of new problems in so doing.

This is not an argument against the value of quantitative information. Both quantitative and qualitative information can be extraordinarily useful for understanding teaching and learning and how any given child is progressing. Rather, ours is an argument against the uncritical acceptance of the aggregation axiom noted above and against the failure to honor the professional judgment of educators as a central and critically important feature of any responsible system designed to demonstrate what students know and are able to do. Ultimately, educators should know more about any given child than any test can tell us. And if they worked in organizational and political settings that valued them as professionals and provided the training, resources, and environments necessary to do their work well, they could make good judgments about each and every young person in our schools.28

Okay, if we can't test all students at the same points in time and aggregate the results upward to inform an on-demand, test-driven accountability system, what are we left with? One emerging view seems compatible with our beliefs and knowledge-based claims detailed earlier, and it leads directly to something teachers have known ever since there have been teachers -- the importance of individualization. Unshackling ourselves from the demands of traditional accountability systems provides wonderful opportunities to think anew about authentic ways in which no child is ever left behind at any time during his or her years in public education.

Moreover, this is an invitation not just to revisit such useful pedagogical principles as individualization but also to focus directly on issues of equity. Each and every student has a "learning style" -- each black child; each white child; each Hispanic child; each Native American child; each child with a background from any of the Asian and Pacific Rim groups; each boy and girl; each child whether from a poor, middle-class, or wealthy family; and so on. Yes, there may be important characteristics distinguishing the learning of various ethnic, racial, gender, or economic groups, but there is no doubt more within-group variation that is also critically important. Ultimately, what is crucial is nourishing each and every child's potential to continue to learn and to continue to become.

As Israel Scheffler so eloquently argues, human potential is not a finite concept. It is not as if a child's potential can be measured in a cup that spills over once it is filled. Helping children "reach their full potential," although a phrase often heard and well intended, is a statement ill suited to the true task of education. Children have unlimited potential, or as Scheffler would put it, "capacity, propensity, and capability to become." The task of political infrastructures and school systems is to create environments within which this "becoming" is possible. We need to coach and nurture students on their way to becoming; they are cups with infinite volume. The last thing we need to do, as educators and as policy makers, is to turn off the tap prematurely on a child or young adult on the basis of information that has little or no predictive validity with respect to the future life chances of that human being.

We need to remind ourselves that we live in America. This is the nation to which so many students from so many other countries (who scored higher on international comparisons) flock to participate in our system of postsecondary education and, ironically, join our students (who scored lower) in their processes of becoming. Our country ought to be proud of its education systems, notwithstanding much that could be improved. We say that we value the capacities, capabilities, and propensities of all people to become. Our education systems are (or should be) designed to promote this end, not to obstruct it. And we do all of this in the name of democracy. Connecting the democratic vision with reconstructing the concept of human potential, Scheffler puts it this way:

In nondemocratic societies, education is two-faced: it is a weapon or an instrument for shaping the minds of the ruled in accord with the favored myth of the rulers; for the rulers, however, it is an induction into the prerogatives and arts of rule, including the arts of manipulating the opinions of the masses.

The democratic ideal precludes the conception of education as an instrument of rule; it is antithetical to the idea of rulers shaping or molding the mind of the pupil. The function of education in a democracy is rather to liberate the mind, strengthen its critical powers, inform it with knowledge and the capacity for independent inquiry, engage its human sympathies, and illuminate its moral and practical choices.29

Our thought adventure, therefore, leads us metaphorically to "IEPs" (individualized education programs) for everyone.30 Every student is special. One of the many side effects of uncritically accepting the aggregation axiom and traditional test-based accountability systems is the current worry over testing modifications and accommodations for special education students. Given the standardized process of test administration and concerns for reliability and validity, making any alterations or allowances (reading a math item to students, for example) in the testing process for certain groups of students raises obvious red flags.

However, our thought adventure suggests that every student deserves "accommodations." A better word for this would be education. If we were less concerned with test-based indicators and aggregation systems for high-stakes accountability and more concerned with demonstrating a record of opportunities for and outcomes of learning for each and every student, we would be well on the road to a more responsible accountability system.31 As noted by Robert Stake more than a decade ago:

Knowing the rank order of students as to proficiency is not at all the same as knowing what students know. . . . Education is not so much an achieving of some fixed standard. In a true sense, it requires unique and personal definition for each learner. . . . Education is a personal process and a personally unique accomplishment.32

 

Concluding Thoughts

My colleagues and I have proposed a set of beliefs, knowledge-based claims, and a thought adventure to help think less conventionally and more responsibly about accountability practices. First, these beliefs and claims reinforce the notion of a broad and high-quality curriculum for all students that generally focuses on the skills and habits of thought necessary for full and participating citizenship in our social and political democracy. Second, they suggest accounting for the whole in ways that do not stress on-demand, high-stakes tests and the fallout from such practices but that focus instead on a broader array of indicators and resource-rich educational environments characterized by all the good conditions and practices known to facilitate better teaching and learning.

Our thought adventure has led us to reaffirm the importance of individualization and classroom-based information that ought to be naturally accumulated for each student during the formative processes of teaching, learning, and assessment. Each child, adolescent, and young adult needs to be cared for in terms of intellectual, social, personal, and career-oriented educational needs -- not to meet some arbitrary level of performance on an on-demand test, but to develop the ability and likelihood "to become." Each child has combinations of strengths and weaknesses, and all children deserve to thrive in an educational environment without the test-based threat of failing to graduate hanging over their heads.

Yes, the public has a right to know how well our public schools are educating future citizens, but, at the same time, those who fashion accountability systems for schooling must themselves be held accountable for doing it responsibly. It is essential that educators not let themselves off the hook when it comes to ensuring equity and excellence in our schools and closing the "achievement gap." Yet it is equally essential that the public not let our "educational politicians" off the hook with regard to closing the "rhetorical gap" -- the gap between what politicians and policy makers say they want for public education and the actual mustering of the will, commitment, and resources necessary to do something authentic about it.

A responsible "ecology of accountability," therefore, must operate on two fronts simultaneously: the day-to-day efforts to improve the education of children in schools and the concerted efforts of educators and their constituencies to demand that the political infrastructure dramatically alter its priorities and invest the necessary resources where they are needed most. We are just beginning to think about how this ecology might be characterized.

For example, states need to set global goals for education, fund professional development substantially, and implement auditing processes that are flexible enough to allow for local variability but have enough teeth to identify problem districts and schools. Several major features should characterize such auditing processes: 1) they should be developed collaboratively with educators and their communities and cannot be based primarily on testing schemes and aggregated data;33 2) they should be based on coordinated site visits by teams of education professionals and representatives of the public, all of whom are adequately versed in using case study methods to really understand each school; and 3) they should focus on the school's community, families, and students, on its teachers, teaching, and opportunities for learning, on its conditions and circumstances (e.g., physical plant, resources, student mobility, etc.), on its educational goals and means of assessing them, and on its information to suggest how well it is doing with respect to each individual student in its care. We are not against strong intervention processes when all else -- in terms of positive assistance and support -- seems to fail.

Districts and schools need to focus their resources on teaching, learning, and assessment with particular attention paid to issues of equity and opportunities to learn for all students. Districts and schools also need to work with their local communities to set educational goals and accountability strategies that are comprehensive and meaningful.34 Real people are involved here; they have real hopes and human desires. These hopes and desires are varied, but we suspect that there will be remarkable consistency and congruence across schools, districts, and even states.35 And there are good examples over the last decade or so of assessments and accountability practices that are based on the professional judgment of educators, that take account of a rich and broad curriculum, and that seek to assess the individual merits of each and every student.36

We intend to make use of these examples and find others as we continue to explore more responsible practices for "calling into account" both our social and political systems and our public schools. Yes, the devil may be in the details, but it is a devil worth confronting. We envision the day when neither my high school friend Helen nor any of our nation's young people would be punished on the basis of a test-driven system that is ill suited to their abilities and ambitions and to the broader purposes of public schooling in our multicultural and democratic society.

2. In addition to Haney's analyses cited above, see Jay P. Heubert and Robert M. Hauser, eds., High Stakes: Testing for Tracking, Promotion, and Graduation (Washington, D.C.: National Academy Press, 1999), chap. 7.

3. Again, the accountability reforms in Texas and Kentucky are informative. See Linda M. NcNeil, Contradictions of School Reform: Educational Costs of Standardized Testing (New York: Routledge, 2000); and Whitford and Jones, op. cit. If you want to move on to other states, see, for example, M. Gail Jones et al., "The Impact of High-Stakes Testing on Teachers and Students in North Carolina," Phi Delta Kappan, November 1999, pp. 199-203; or, moving to the north, Abby Goodnough, "Strain of Fourth-Grade Tests Drives Off Veteran Teachers," New York Times, 14 June 2001, p. A-1.

4. Notwithstanding the court rulings thus far in favor of allowing high-stakes exit examinations (e.g., New York, Louisiana, and Texas), momentum against such use of tests is mounting and may eventually lead to a successful challenge in some state(s). Much of this momentum in the form of protests and boycotts is documented in the FairTest Examiner, the newsletter of the National Center for Fair and Open Testing (www.fairtest.org). An interesting and enlightened parent movement can be found at http://pasaorg.tripod.com. See also Peter Schrag, "High Stakes Are for Tomatoes," Atlantic, August 2000, pp. 19-21. The Texas case (GI Forum et al. v. Texas Education Agency et al., No. SA-97-CA-1278-EP) has been widely criticized and is based on a basic-skills test, not a high-standards-based test like the current system in Washington State. When substantial numbers of middle- and upper-class students start failing high-standards exit exams, the protests are likely to pick up considerable steam. Already parents in some communities -- Scarsdale, New York; Marin County, California; Panama City, Florida; and others -- are launching protests. See, for example, Kathleen Kennedy Manzo, "Protests Over State Testing Widespread," Education Week, 16 May 2001, pp. 1, 26. Still, the students really hurt by all of this continue to be those disenfranchised children in poorer communities and in grossly underfunded and poorly staffed schools. There is no way in which reasonable arguments can be made that students in such systems have equal opportunities to learn, and it may be on this or similar bases (e.g., discrimination in terms of disproportionate failure rates) that future court cases may be won. See, for example, the "public advisory" issued by the American Civil Liberties Foundation of Massachusetts, 11 April 2000 (www.aclu-mass.org/youth/studentrights/mcasadvisory.html). Arizona has recently put off its high school exit exam until the year 2006 precisely out of fear of impending lawsuits and in response to the filing of a civil rights complaint with the U.S. Department of Education, according to Darcia Harris Bowman, "Delayed Again: Ariz. Moves High School Exit Exam to 2006," Education Week, 5 September 2001, p. 27. Finally, what may turn out to be a landmark case -- Williams et al. v. State of California -- is the current lawsuit in California filed by the American Civil Liberties Union and a coalition of other rights advocates arguing that equal opportunities to learn are simply not possible under such documented conditions as lack of materials and basic resources, inadequate instruction, massive overcrowding, and degraded, unhealthful facilities and conditions. See "Landmark Lawsuit on Behalf of Public School Students Demands Basic Education Rights Promised in State Constitution," 17 May 2000, available at www.aclu.org/news/2000/n051700a.html.

5. I refer here to Project PRAISE, currently funded by the Rockefeller Foundation, and to my immediate colleagues Shawn Olson Brown, Paul Heckman, Roger Soder, and Patricia Wasley. Although I use "we" in what follows, I take full responsibility for the views expressed here. Moreover, these views are not necessarily those of the Rockefeller Foundation.

6. See Kenneth A. Sirotnik and Kathy Kimball, "Standards for Standards-Based Accountability," Phi Delta Kappan, November 1999, pp. 209-14; and Kenneth A. Sirotnik, "Making Sense of Educational Renewal," Phi Delta Kappan, April 1999, pp. 606-10.

7. See, for example, Roger Soder, "Education for Democracy: The Foundation for Democratic Character," in Roger Soder, John I. Goodlad, and Timothy J. McMannon, eds., Developing Democratic Character in the Young (San Francisco: Jossey-Bass, 2001), pp. 182-205. See also the collection of work in Roger Soder, ed., Democracy, Education, and the Schools (San Francisco: Jossey-Bass, 1996).

8. Benjamin R. Barber, "America Skips School," Harper's, November 1993, p. 44.

9. For more on what is meant by "moral endeavor," see John I. Goodlad, Roger Soder, and Kenneth A. Sirotnik, eds., The Moral Dimensions of Teaching (San Francisco: Jossey-Bass, 1990).

10. Examples of this knowledge include much of the work produced by the National Center for Research on Evaluation, Standards, and Student Testing at UCLA (www.cse.ucla.edu) and by the Center for the Study of Testing, Evaluation, and Educational Policy at Boston College (www.csteep.bc.edu). Moreover, there is still more to be learned from educators in the field who are inventing new and robust ways to work more responsibly under increasing demands for higher content standards and more accountability. One example is the current work of the Shared Accountability Committee in the Edmonds School District, just north of Seattle. Other examples have been shared in the pages of this journal. See the recent article by Selma Wassermann, "Quantum Theory, the Uncertainty Principle, and the Alchemy of Standardized Testing," Phi Delta Kappan, September 2001, pp. 28-40. In addition, there is much to be learned from the work of the Coalition of Essential Schools about the various approaches to exhibiting what students know and can do and about the making of good professional judgments about student progress. See, for example, Kathleen Cushman, "Documenting Whole-School Change in Essential Schools," Horace, January 1996, pp. 1-7. Finally, the National Center for Fair and Open Testing has sponsored a number of interesting critiques and useful publications, including "Principles and Indicators for Student Assessment Systems" and "Implementing Performance Assessments: A Guide to Classroom, School and System Reform," available on the FairTest website (www.fairtest.org).

11. Brian A. Jacob, "Getting Tough? The Impact of High School Graduation Exams," Educational Evaluation and Policy Analysis, Summer 2001, pp. 99-121.

12. Lorrie Shepard, "The Effects of High-Stakes Testing on Instruction," paper presented at the annual meeting of the American Educational Research Association, Chicago, 1991; McNeil, chap. 7; and George F. Madaus, "The Influence of Testing on the Curriculum," in Laurel N. Tanner, ed., Critical Issues in Curriculum: Eighty-Seventh Yearbook of the National Society for the Study of Education (Chicago: University of Chicago Press, 1988).

13. John I. Goodlad, A Place Called School: Prospects for the Future (New York: McGraw-Hill, 1984); and Ernest L. Boyer, High School: A Report on Secondary Education in America (New York: Harper & Row, 1983). Moreover, recent polls suggest that a sensible public has become thoroughly confused by all the political rhetoric surrounding standards and high-stakes testing and accountability. The most recent Phi Delta Kappa/Gallup poll indicates that more than half (57%) of the public favors the use of a single standardized test as a criterion for high school graduation. Yet two-thirds of these same respondents say that standardized tests should be used to guide instruction, and only about one-third think they should assess how much students have learned. Finally, nearly two-thirds think student learning should be assessed by the day-to-day work in classrooms, including homework. See Lowell C. Rose and Alec M. Gallup, "The 33rd Annual Phi Delta Kappa/Gallup Poll of the Public's Attitudes Toward the Public Schools," Phi Delta Kappan, September 2001, p. 54.

14. On individualization, see Lyn Corno and Richard E. Snow, "Adapting Teaching to Individual Differences Among Learners," in Merlin C. Wittrock, ed., Handbook of Research on Teaching (New York: Macmillan, 1986), pp. 605-29. On developmental issues, see Richard Sprinthall and Norman A. Sprinthall, Educational Psychology: A Developmental View (New York: McGraw-Hill, 1998). On learning styles, see Geneva Gay, Culturally Responsive Teaching: Theory, Research, and Practice (New York: Teachers College Press, 2000), chap. 6; and Howard Gardner, The Unschooled Mind: How Children Think and How Schools Should Teach (New York: Basic Books, 1991). On retention issues, see Lorrie A. Shepard and Mary Lee Smith, eds., Flunking Grades: Research and Policies on Retention (London: Falmer Press, 1989); and Heubert and Hauser, chap. 6.

15. See Russell W. Rumberger, "Why Students Drop Out of School and What Can Be Done," paper prepared for a conference on Dropouts in America: How Severe Is the Problem? What Do We Know About Intervention and Prevention?, Harvard University, Cambridge, Mass., 13 January 2001. For example, Rumberger reports a study by James Catterall, who estimated costs of $3.2 billion plus $400 million in lost earnings and increased social services respectively due to the impact of one year's worth of high school dropouts in one U.S. city -- and this was in 1987 dollars! See James S. Catterall, "On the Social Costs of Dropping Out of School," High School Journal, November 1987, pp. 19-30.

16. For an overview of "opportunity to learn" issues, see Educational Evaluation and Policy Analysis, Fall 1995. On issues of unequal resources, see Jonathon Kozol, Savage Inequalities: Children in America's Schools (New York: Crown, 1991); and Gretchen Guiton and Jeannie Oakes, "Opportunity to Learn and Conceptions of Educational Equality," Educational Evaluation and Policy Analysis, Fall 1995, pp. 323-36. On tracking, school size, and class size, see Jeannie Oakes, Keeping Track: How Schools Structure Inequality (New Haven, Conn.: Yale University Press, 1985); Deborah Meier, The Power of Their Ideas: Lessons for America from a Small School in Harlem (Boston: Beacon Press, 1995); Jeremy D. Finn and Charles M. Achilles, "Answers and Questions About Class Size: A Statewide Experiment," American Educational Research Journal, vol. 27, 1990, pp. 557-77; and the special issue of Educational Evaluation and Policy Analysis, Summer 1999.

17. See, for example, Penelope L. Peterson and Herbert J. Walberg, eds., Research on Teaching: Concepts, Findings, and Implications (Berkeley, Calif.: McCutchan, 1979); and Jere E. Brophy and Thomas L. Good, "Teacher Behavior and Student Achievement," in Wittrock, pp. 328-75. For a recent review at the policy level, see Linda Darling-Hammond, Teacher Quality and Student Achievement: A Review of State Policy Evidence (Seattle: Center for the Study of Teaching and Policy, University of Washington, 1999). See also What Matters Most: Teaching for America's Future (New York: National Commission on Teaching and America's Future, 1996).

18. See John D. Baldwin and Janice I. Baldwin, Behavior Principles in Everyday Life (Upper Saddle River, N.J.: Prentice-Hall, 1998), pp. 320-23. An important and related argument could be made regarding the ineffectiveness of extrinsic motivation and its negative impact on "natural human self-regulation." See Richard M. Ryan and Edward L. Deci, "When Rewards Compete with Nature: The Undermining of Intrinsic Motivation and Self-Regulation," in Carol Sansone and Judith M. Harackiewicz, eds., Intrinsic and Extrinsic Motivation: The Search for Optimal Motivation and Performance (San Diego: Academic Press, 2000), p. 48.

19. Murray Sidman, Coercion and Its Fallout (Boston: Authors Cooperative, 1989), p. 78.

20. For a general analysis of the failures of top-down reform, see Thomas B. Timar and David L. Kirp, "Educational Reform and Institutional Competence," Harvard Educational Review, August 1987, pp. 308-30.

21. Suzanne M. Wilson, Linda Darling-Hammond, and Barnett Berry, A Case of Successful Teaching Policy: Connecticut's Long-Term Efforts to Improve Teaching and Learning (Seattle: Center for the Study of Teaching and Policy, University of Washington, 2001).

22. Melissa Roderick and Mimi Engel, "The Grasshopper and the Ant: Motivational Responses of Low-Achieving Students to High-Stakes Tests," Educational Evaluation and Policy Analysis, Fall 2001, p. 221.

23. See, for example, John I. Goodlad, The Dynamics of Educational Change (New York: McGraw-Hill, 1975); Seymour B. Sarason, Revisiting "The Culture of School and the Problem of Change" (New York: Teachers College Press, 1996); and Michael G. Fullan, The New Meaning of Educational Change (New York: Teachers College Press, 1991).

24. Seymour B. Sarason, Schooling in America: Scapegoat and Salvation (New York: Free Press, 1983), pp. 5-8.

25. Clearly, we're using "against the law" and "illegal" as rhetorical devices to frame the thought adventure we have in mind.

26. For an example from writing assessment, see Linda Mabry, "Writing to the Rubric: Lingering Effects of Traditional Standardized Testing on Direct Writing Assessment," Phi Delta Kappan, May 1999, pp. 673-79.

27. By the way, keeping track of such information, both qualitative and quantitative, is relatively simple these days with computer technology.

28. Wassermann, op. cit.; Linda Darling-Hammond, "Reframing the School Reform Agenda: Developing the Capacity for School Transformation," Phi Delta Kappan, June 1993, pp. 753-61; and Meier, op. cit. See also Catherine S. Taylor, "Incorporating Classroom-Based Assessments into Large-Scale Assessment Programs," in Gerald Tindal and Thomas M. Haladyna, eds., Large-Scale Assessment Programs for All Students: Validity, Technical Adequacy, & Implementation (Mahwah, N.J.: Erlbaum, 2002).

29. Israel Scheffler, Of Human Potential: An Essay in the Philosophy of Education (Boston: Routledge & Kegan Paul, 1985), p. 124.

30. We are using the idea of IEPs here as a metaphor. The last thing we want to do is create another huge bureaucracy, myriad forms, mazes of legal entanglements, and, most important, a "remedial" model of correcting "deficits" in student learning. Without endorsing any packaged program, a better acronym and set of pedagogical principles to use here might be IGE (Individually Guided Education), a 35-year-old idea that is still alive and relevant. (For information on IGE, see www.idea.org/igehome.htm.)

31. We would also save at least half a billion dollars yearly, which states now spend on standardized testing of one form or another, and we could put that money back into day-to-day education, where it belongs. Estimating the cost of testing is a difficult task, and published results are probably serious underestimates because of all the hidden costs associated with preparing for tests, maintaining item banks, making continued refinements, etc. With this in mind, see the $400 million yearly estimate recently by Tiffany Dantz, "States Pay $400 Million for Tests In 2001," 27 February 2001, available at www.stateline.org. Yet even back in early-1990s dollars, estimates went past $500 million when school district spending was taken into account, according to David J. Hoff, "States Spend Nearly Half-a-Billion on Testing," Education Week, 14 March 2001. A substantial portion of whatever is saved should be reinvested in new and useful classroom assessment methods and in the professional development to support them. Useful resources already exist that attempt to refocus assessment and accountability and develop the expertise of teachers. See, for example, Linda Mabry, Portfolios Plus: A Critical Guide to Alternative Assessment (Thousand Oaks, Calif.: Corwin Press, 1999); and Richard J. Stiggins, Student-Centered Classroom Assessment (Upper Saddle River, N.J.: Merrill, 1994). Moreover, there is plenty of new and useful work here for psychometricians interested in where the real action is -- the intersection of teaching, learning, and assessment.

32. As cited by Mabry, Portfolios Plus, p. 26. Mabry's critique of "psychometric," "contextual," and "personal" paradigms is highly relevant to our argument here. See her discussion in chapter 2. The quote is from Robert E. Stake, The Invalidity of Standardized Testing for Measuring Mathematics Achievement (Madison: National Center for Research on Mathematical Sciences Education, University of Wisconsin, 1991).

33. If not aggregating and reporting data on some type of test battery, standards-based or otherwise, is simply too hard to swallow, politically or educationally, then at least consider seriously this alternative to high-stakes testing of individuals: use matrix sampling, which provides reliable estimates of mean performance at the school, district, and state levels and uses only a fifth to a tenth of the time required for testing at the individual level.

34. One interesting approach to accountability is that put forward by Larry Cuban, "Why Is It So Hard to Get 'Good' Schools?," in Larry Cuban and Dorothy Shipps, eds., Reconstructing the Common Good in Education: Coping with Intractable American Dilemmas (Stanford: Stanford University Press, 2000), pp. 148-69. In addition, a number of new sources of ideas for alternative accountability practices are surfacing; see, for example, the website of the Annenberg Institute for School Reform and its "accountability toolbox" at www.annenberginstitute.org/accountability/toolbox/.

35. Years ago, in A Study of Schooling, we found remarkable consistency in the formal curriculum documents across states and in the operational curriculum in schools and classrooms. The main differences were in the contextual circumstances and affective environments of the schools. Parents, teachers, and students, however, valued all four major goal areas of schooling: the academic, personal, social, and vocational. They wanted it all. See Goodlad, A Place Called School, chap. 2; and Boyer, chap. 3.

36. I have already cited examples throughout this article. Another one worth noting is the proposed accountability system for Massachusetts put forth by the Coalition for Authentic Reform of Education (www.fairtest.org/arn/masspage.html). This proposal includes locally defined authentic assessments, limited standardized testing at parents' discretion, a school audit review process, and annual reporting by districts and schools to parents, the community, and the state.

PDK Home | Site Map
Kappan Professional Journal
Last updated 14 May 2002
URL: http://www.pdkintl.org/kappan/k0205sir.htm
Copyright 2002 Phi Delta Kappa International