A Balanced School Accountability Model:
An Alternative to High-Stakes Testing

The health of our public schools, Mr. Jones argues, depends on defining a new model of accountability - one that is balanced and comprehensive. And it needs be one that involves much more than test scores.

By Ken Jones

FOR SOME time now, it has been apparent to many in the education community that state and federal policies intended to develop greater school accountability for the learning of all students have been terribly counterproductive. The use of high-stakes testing of students has been fraught with flawed assumptions, oversimplified understandings of school realities, undemocratic concentration of power, undermining of the teaching profession, and predictably disastrous consequences for our most vulnerable students. Far from the noble ideal of leaving no child behind, current policies, if continued, are bound to increase existing inequities, trivialize schooling, and mislead the public about the quality and promise of public education.

What is needed is a better means for evaluating schools, an alternative to the present system of using high-stakes testing for school accountability. A new model, based on a different set of assumptions and understandings about school realities and approaches to power, is required. It must be focused on the needs of learners and on the goals of having high expectations for all rather than on the prerequisites of a bureaucratic measurement system.

Premises

In the realm of student learning, the question of outcomes has often been considered primary: what do we want students to know and be able to do as a result of schooling? Once the desired outcomes have been specified, school reform efforts have proceeded to address the thorny questions of how to attain them. Starting from desired outcomes is an important shift in how to think about what does or does not make sense in classroom instruction.

In the realm of school accountability, however, little attention has been paid to corresponding outcome-related questions. It has simply been assumed that schools should be accountable for improved student learning, as measured by external test scores. It has been largely assumed by policy makers that external tests do, in fact, adequately measure student learning. These and other assumptions about school accountability must be questioned if we are to develop a more successful accountability model. It would be well to start from basic questions about the purposes and audiences of schools. For what, to whom, and by what means should schools be held accountable? The following answers to these questions provide a set of premises on which a new school accountability system can be based.

For what should schools be accountable? Schools should be held accountable for at least the following:

• The physical and emotional well-being of students. The caring aspect of school is essential to high-quality education. Parents expect that their children will be safe in schools and that adults in schools will tend to their affective as well as cognitive needs. In addition, we know that learning depends on a caring school climate that nurtures positive relationships.
• Student learning. Student learning is complex and multifaceted. It includes acquiring not only knowledge of disciplinary subject matter but also the thinking skills and dispositions needed in a modern democratic society.
• Teacher learning. Having a knowledgeable and skilled teacher is the most significant factor in student learning and should be fostered in multiple ways, compatible with the principles of adult learning. Schools must have sufficient time and funding to enable teachers to improve their own performance, according to professional teaching standards.
• Equity and access. Given the history of inequity with respect to minority and underserved student populations, schools must be accountable for placing a special emphasis on improving equity and access, providing fair opportunities for all to learn to high standards. Our press for excellence must include a press for fairness.
• Improvement. Schools should be expected to function as learning organizations, continuously engaged in self-assessment and adjustment in an effort to meet the needs of their students. The capacity to do so must be ensured and nurtured.

To whom should schools be accountable? Schools should be held accountable to their primary clients: students, parents, and the local community. Current accountability systems make the state and federal governments the locus of power and decision making. But the primary clients of schools should be empowered to make decisions about the ends of education, not just the means, provided there are checks to ensure equity and access and adherence to professional standards for teaching.

By what means should schools be held accountable? To determine how well schools are fulfilling their responsibilities, multiple measures should be used. Measures of school accountability should include both qualitative and quantitative approaches, taking into account local contexts, responsiveness to student and community needs, and professional practices and standards. Because schools are complex and unique institutions that address multiple societal needs, there should also be allowances for local measures, customized to meet local needs and concerns. A standardized approach toward school accountability cannot work in a nation as diverse as the U.S.

Given these premises, what are the proper roles of a government-developed and publicly funded school accountability system?

• It should serve to improve student learning and school practices and to ensure equity and access, not to reward or punish schools.
• It should provide guidance and information for local decision making, not classify schools as successes or failures.
• It should reflect a democratic approach, including a balance of responsibility and power among different levels of government.

A Balanced Model

An accountability framework called the "balanced scorecard" is currently employed in the business world and provides a useful perspective for schools.1 This framework consists of four areas that must be evaluated to give a comprehensive view of the health of an organization. The premise is that both outcomes and operations must be measured if the feedback system is to be used to improve the organization, not just monitor it. In the business context, the four components of the framework are: 1) financial, 2) internal business, 3) customer, and 4) innovation and learning.

Applying this four-part approach to education, we can use the following aspects of school performance as the components of a balanced school accountability model: 1) student learning; 2) opportunity to learn; 3) responsiveness to students, parents, and community; and 4) organizational capacity for improvement. Each of these aspects must be attended to and fostered by an evaluation system that has a sufficiently high resolution to take into account the full complexity and scope of modern-day schools.

1. Student learning. Principles of high-quality assessment have been well articulated by various organizations and should be followed.2 What is needed is a system that

• is primarily intended to improve student learning;
• aligns with local curricula;
• emphasizes applied learning and thinking skills, not just declarative knowledge and basic skills;
• embodies the principle of multiple measures, including a variety of formats such as writing, open-response questions, and performance-based tasks; and
• is accessible to students with diverse learning styles, intelligence profiles, exceptionalities, and cultural backgrounds.

Currently, there is a mismatch between what cognitive science and brain research have shown about human learning and how schools and educational bureaucracies continue to measure learning.3 We now know that human intellectual abilities are malleable and that people learn through a social and cultural process of constructing knowledge and understandings in given contexts. And yet we continue to conduct schooling and assessment guided by the outdated beliefs that intelligence is fixed, that knowledge exists apart from culture and context, and that learning is best induced through the behaviorist model of stimulus/response.

Scientific measurement cannot truly "objectify" learning and rate it hierarchically. Accurate decisions about the quality and depth of an individual's learning must be based on human judgment. While test scores and other assessment data are useful and necessary sources of information, a fair assessment of a person's learning can be made only by other people, preferably by those who know the person best in his or her own context. A reasonable process for determining the measure of student learning could involve local panels of teachers, parents, and community members, who review data about student performance and make decisions about promotion, placement, graduation, and so on.

What is missing in most current accountability systems is not just a human adjudication system, but also a local assessment component that addresses local curricula, contexts, and cultures. A large-scale external test is not sufficient to determine a student's achievement. District, school, and classroom assessments must also be developed as part of a comprehensive means of collecting data on student learning. The states of Maine and Nebraska are currently developing just such systems.4

Most important, locally developed assessments depend on the knowledge and "assessment literacy" of teachers.5 Most teachers have not been adequately trained in assessment and need substantial and ongoing professional development to create valid and reliable tasks and build effective classroom assessment repertoires. This means that an investment must be made in teacher learning about assessment. The value of such an investment is not only in the promise of improved classroom instruction and measurement. Research also shows that improved classroom assessment results in improved student achievement on external tests.6

Last, the need to determine the effectiveness of the larger state school system can either support or undermine such local efforts. If state or federal agencies require data to be aggregated from local to state levels, local decision making is necessarily weakened, and an undue emphasis is placed on standardized methods. If, however, the state and federal agencies do not rely on local assessment systems to gauge the health

of the larger system, much may be gained. In New Zealand, for example, a system of educational monitoring is in place that uses matrix sampling on tasks that include one-to-one videotaped interviews, team tasks, and independent tasks.7 No stakes are entailed for schools or students. The data are profiled and shared with schools for the purpose of teacher professional development and as a means of developing model tasks for local assessments. Such a system supports rather than undermines local assessment efforts.

2. Opportunity to learn. How can students be expected to meet high standards if they are not given a fair opportunity to learn? This question has yet to be answered with respect to school accountability. Schools should be accountable for providing equitable opportunities for all students to learn, and we must develop ways to determine how well they do so.

At the heart of the matter is that the responsibility for opportunity to learn must be shared by the district and state. The inequitable funding of public schools, particularly the disparity between the schools of the haves and those of the have-nots, places the schools of disadvantaged students in unjust and often horrifying circumstances. Over the past decade, there have been lawsuits in various states attempting to redress this imbalance, which is largely a result of dependence on property taxes for school funding. Yet not a great deal of progress has been made.

How should we define and put into practice our understanding of opportunity to learn? How will we measure it? How can an accountability system foster it?

At a minimum, one might expect that schools and school systems will provide qualified teachers, adequate instructional materials, and sound facilities. This is the contention in a recent lawsuit, Williams v. State of California, in which the plaintiffs argued for an accountability system that is reciprocal - that is, while schools are held accountable for performance, the state is held accountable for ensuring adequate resources.8

But there is more to this issue than just funding. Jeannie Oakes describes a framework that includes opportunity-to-learn indicators for access to knowledge, professional teaching conditions, and "press for achievement."9 Linda Darling-Hammond stresses the "fair and humane treatment" of students in a set of standards for professional practice.10

As such standards for opportunity to learn are articulated, the question arises as to how to monitor and report on them. Clearly, the degree of adherence to these standards cannot be determined through the proxy of testing. It is necessary to conduct observations in schools and classrooms and to evaluate the quality both of individual teachers and of the school as a whole.

Teacher evaluation has received a great deal of criticism for being ineffective. The hit-and-run observations that principals typically conduct do little to determine whether teachers are meeting established professional teaching standards. Unions have been described as more interested in protecting their membership than in ensuring high-quality teaching. A promising development that has potential for breaking through this impasse is the recent initiation of peer-review processes by a number of teacher unions. Adam Urbanski, president of the Rochester Teachers Association and director of the Teacher Union Reform Network (TURN), has been a leader in advocating for and implementing such teacher evaluation processes. In a recent unpublished manuscript, he describes how the process should work:

• Some classroom observation by peers and supervisors, structured by a narrative instrument (not a checklist) based on professional standards such as those of the National Board for Professional Teaching Standards (NBPTS) and framed by the teacher's goals for the lesson/unit;
• Information from previous evaluations and feedback, such as structured references from colleagues and other supervisors;
• Portfolios that might include examples of teaching syllabi, assignments made, feedback given to students and samples of student work, feedback received from parents and students as well as colleagues, data on student progress, teaching exhibitions such as videotaped teaching samples, professional development initiatives taken, and structured self-evaluation. All summative evaluation decisions about promotions or continued employment should be made by a specially established committee of teachers and administrators.

Urbanski goes on to describe safeguards for due process and for preventing malpractice. He also describes how such a process could be used in conjunction with professional development for improving teaching and school practice.11

In order to evaluate the performance of a school as a whole, a school review process will be necessary. Variations of inspectorates and school-quality reviews have been developed in New York, Rhode Island, Maine, and other states, as well as in Britain, New Zealand, Australia, and other countries.12 In order for such reviews to serve the purpose of school improvement, the data should be collected in a "critical friend" manner, through a combination of school self-assessment and collegial visitations. Findings from such a process should not be employed in a bureaucratic and judgmental way but rather should be given as descriptions to local councils charged with evaluating school accountability. As with all aspects of any school renewal initiative, the quality and effectiveness of a review system will depend on the time, resources, and institutional support given to it.

Who will ensure that adequate opportunities to learn are present in schools? As described below, a system of reciprocal accountability must be set up so that both local accountability councils and the state itself serve to "mind the store" for all students. The issue of equitable funding will undoubtedly be resolved through the courts.

3. Responsiveness to students, parents, and community. Current accountability systems move power and decision making away from the primary clients of the education system and more and more toward state and federal agencies. As high-stakes testing dictates the curriculum, less and less choice is available for students. Parent or community concerns about what is happening in the classroom and to the students have become less important to schools than meeting state mandates.

As the primary stakeholders in the schools, parents and communities must be made part of the effort to hold schools accountable. There are many examples of local community organizations, especially in urban areas, that have taken on the task of insisting that schools are responsive to the needs of children.13

To demonstrate responsiveness to students, parents, and the community, schools must go beyond sponsoring parent/teacher organizations or encouraging parent involvement as a means to gain support for existing school practices. They must also do more than gather survey information about stakeholders' satisfaction. True accountability to the primary clients for schools entails shifting power relationships.

Local school-based councils must be created that have real power to effect school change. These councils would review accountability information from state and local assessments as well as from school-quality review processes and make recommendations to school boards about school policies and priorities. They would hold school boards accountable for the development and implementation of school improvement plans. Phillip Schlechty discusses how such councils might work:

Community leaders who are concerned about the futures of their communities and their schools should join together to create a nonprofit corporation intended to support efforts of school leaders to focus on the future and to ensure that lasting values as well as immediate interests are included in the education decision-making process. It would also be the function of this group to establish a small subgroup of the community's most trusted leaders who would annually evaluate the performance of the school board as stewards of the common good and would make these evaluations known to the community. . . .

In a sense, the relationship between the school district and the monitoring function of the new corporation should be something akin to the relationship between the quality assurance division of a corporation and the operating units in the corporation. . . .

When the data indicate that goals are not being met, the president of the corporation, working with the superintendent and the board of education, would seek to discover why this was the case, and would seek as well to create new approaches that might enhance the prospect of achieving the stated goals and the intended ends. It is not intended that the new corporation simply identify problems and weaknesses, it is intended that the leaders of this organization also participate in the creation of solutions and participate in creating support for solutions once they have been identified or created.14

Communities must determine how to sustain such councils and ensure that they do not pursue narrow agendas. The composition of councils in urban settings will probably be different from those in rural or suburban settings. Standards and acceptable variations for councils will be important topics for public discussion.

4. Organizational capacity. If schools are going to be held accountable to high levels of performance, the question arises: Do schools have the internal capacity to rise to those levels? To what degree are the resources of schools "organized into a collective enterprise, with shared commitment and collaboration among staff to achieve a clear purpose for student learning"?15

The issue of meaningful and ongoing teacher professional development is especially pertinent to whether or not schools are capable of enabling all students to meet higher standards of performance. A great deal of research has shed light on what kind of professional development is most effective in promoting school improvement.16

Schools must also attend to the issue of teacher empowerment. Teachers are increasingly controlled and disempowered in various ways. This leads to a declining sense of efficacy and professionalism and a heightened sense of job dissatisfaction and has become a factor in the attrition that is contributing to the growing teaching shortage.17 Principals must share leadership with teachers and others as a means of sustaining capacity.

To be an effective collective enterprise, a school must develop an internal accountability system. That is, it must take responsibility for developing goals and priorities based on the ongoing collection and analysis of data, it must monitor its performance, and it must report its findings and actions to its public. Many schools have not moved past the stage of accepting individual teacher responsibility rather than collective responsibility as the norm.18 States and districts must cooperate with schools to nurture and insist upon the development of such collective internal norms.

The New Role of the State

For a balanced model of school accountability to succeed, there must be a system in which states and districts are jointly responsible with schools and communities for student learning. Reciprocal accountability is needed: one level of the system is responsible to the others, and all are responsible to the public.

The role of state and federal agencies with respect to school accountability is much in need of redefinition. Agencies at these levels should not serve primarily in an enforcement role. Rather, their roles should be to establish standards for local accountability systems, to provide resources and guidance, and to set in place processes for quality review of such systems. Certainly there should be no high-stakes testing from the state and federal levels, no mandatory curricula, and no manipulation through funding. Where there are clear cases of faulty local accountability systems - those lacking any of the four elements discussed above (appropriate assessment systems; adequate opportunities to learn; responsiveness to students, parents, and community; or organizational capacity) - supportive efforts from the state and federal levels should be undertaken.

Are there any circumstances in which a state should intervene forcibly in a school or district? If an accountability system is to work toward school improvement for all schools, does that system not need such "teeth"? This question must be addressed in a way that acknowledges the multi-level nature of this school accountability model. One might envision at least three cases in which the state would take on a more assertive role: 1) to investigate claims or appeals from students, parents, or the community that the local accountability system is not meeting the standards set for such systems; 2) to require local schools and districts to respond to findings in the data that show significant student learning deficiencies, inequity in the opportunities to learn for all students, or lack of responsiveness to students, parents, or communities; and 3) to provide additional resources and guidance to improve the organizational capacity of the local school or district. Is it conceivable that a state might take over a local school or district in this model? Yes, but only after the most comprehensive evaluation of the local accountability system has shown that there is no alternative - and then only on a temporary basis.

It is of great importance to the health of our public schools that we begin as soon as possible to define a new model for school accountability, one that is balanced and comprehensive. Schools can and should be held accountable to their primary clients for much more than test scores, in a way that supports improvement rather than punishes deficiencies. The current model of using high-stakes testing is a recipe for public school failure, putting our democratic nation at risk.

2. National Forum on Assessment, Principles and Indicators for Student Assessment Systems (Boston: FairTest, 1993), available at www.fairtest.

org/k-12.htm.

3. Lorrie A. Shepard, "The Role of Assessment in a Learning Culture," Educational Researcher, October 2000, pp. 4-14.

4. Debra Smith and Lynne Miller, Comprehensive Local Assessment Systems (CLASs) Primer: A Guide to Assessment System Design and Use (Gorham: Southern Maine Partnership, University of Southern Maine, 2003), available at www.usm.maine.edu/smp/tools/primer.htm; and "Nebraska School-Based, Teacher-Led Assessment Reporting System (STARS)," www.nde.state.ne.us/stars/index.html.

5. Richard J. Stiggins, Student-Centered Classroom Assessment (Columbus, Ohio: Merrill, 1997).

6. Paul Black and Dylan Wiliam, "Inside the Black Box: Raising Standards Through Classroom Assessment," Phi Delta Kappan, October 1998, pp. 139-48; and Paul Black et al., Working Inside the Black Box: Assessment for Learning in the Classroom (London, U.K.: Department of Educational and Professional Studies, King's College, 2002).

7. Terry Crooks, "Design and Implementation of a National Assessment Programme: New Zealand's National Education Monitoring Project (NEMP)," paper presented at the annual meeting of the Canadian Society for the Study of Education, Toronto, May 2002.

8. Jeannie Oakes, "Education Inadequacy, Inequality, and Failed State Policy: A Synthesis of Expert Reports Prepared for Williams v. State of California," 2003, available at www.decentschools.org/experts.php.

9. Jeannie Oakes, "What Educational Indicators? The Case for Assessing the School Context," Educational Evaluation and Policy Analysis, Summer 1989, pp. 181-99.

10. Linda Darling-Hammond, Standards of Practice for Learning Centered Schools (New York: National Center for Restructuring Education, Schools, and Teaching, Teachers College, 1992).

11. Adam Urbanski, "Teacher Professionalism and Teacher Accountability: Toward a More Genuine Teaching Profession," unpublished manuscript, 1998.

12. Jacqueline Ancess, Outside/Inside, Inside/Outside: Developing and Implementing the School Quality Review (New York: National Center for Restructuring Education, Schools, and Teaching, Teachers College, 1996); New Zealand Education Review Office, Frameworks for Reviews in Schools, available at www.ero.govt.nz/EdRevInfo/Schedrevs/SchoolFramework.htm; Debra R. Smith and David J. Ruff, "Building a Culture of Inquiry: The School Quality Review Initiative," in David Allen, ed., Assessing Student Learning: From Grading to Understanding (New York: Teachers College Press,1998), pp. 164-82.

13. Kavitha Mediratte, Norm Fruchter, and Anne C. Lewis, Organizing for School Reform: How Communities Are Finding Their Voice and Reclaiming Their Public Schools (New York: Institute for Education and Social Policy, Steinhardt School of Education, New York University, October 2002).

14. Phillip Schlechty, Systemic Change and the Revitalization of Public Education (San Francisco: Jossey-Bass, forthcoming).

15. Fred M. Newmann, M. Bruce King, and Mark Rigdon, "Accountability and School Performance: Implications from Restructuring Schools," Harvard Educational Review, Spring 1997, p. 47.

16. Judith Warren Little, "Teachers' Professional Development in a Climate of Educational Reform," Educational Evaluation and Policy Analysis, vol. 15, 1993, pp. 129-51; and Milbrey W. McLaughlin and Joan Talbert, Professional Communities and the Work of High School Teaching (Chicago: University of Chicago Press, 2001).

17. Richard M. Ingersoll, Who Controls Teachers' Work? Power and Accountability in America's Schools (Cambridge, Mass.: Harvard University Press, 2003).

18. Charles Abelman et al., When Accountability Knocks, Will Anyone Answer? (Philadelphia: Consortium for Policy Research in Education, University of Pennsylvania, CPRE Research Report Series RR-42, 1999).

PDK Home | Site Map

Last modified 25 March 2004
URL: http://www.pdkintl.org/kappan/k0404jon.htm
PDK International respects your privacy
© 2004 Phi Delta Kappa International