 |
|
|
|
|
|
|
|
|
|
|
|
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|
|
||||||||||||||||||||||||||
|
|
Find more Kappan articles in the The Impact of School Reform for the Middle Years: Longitudinal Study of a Network Engaged in Turning Points-Based Comprehensive School Transformation The authors discuss research findings that strongly support the view that high-quality schooling, well implemented, can make profound contributions to the achievement, mental health, and socio/behavioral functioning of students who are often left behind and for whom there is often a sense that school cannot make a difference in their lives. FOR THE PAST six years, the Project on High Performance Learning Communities -- in partnership with what was originally called Project Initiative Middle Level (PIML) but is now known as the Illinois Middle Grades Network (IMGN) -- has been studying what is now a network of more than 97 schools as they undergo the process of restructuring from more traditional organization toward the type of school envisioned in the Carnegie Council's report Turning Points: Preparing American Youth for the 21st Century.1 This report, perhaps more so than any other, provides a holistic, integrated model of education for 10- to 15-year-olds. The work to implement its recommendations began with a pilot effort involving a smaller set of schools during the 1990-91 school year. The schools in the study are all members of the IMGN and are drawn from the broader membership of the Association of Illinois Middle Level Schools (AIMS). Schools must apply to join the IMGN and are admitted as members only after meeting a number of rigorous criteria. Not the least of these criteria is a written commitment from the district and school (supported by more than 80% of its teachers) to make the change from more traditional junior high school structures and practices to those that are consistent with the recommendations of Turning Points. Some of the schools are admitted to the IMGN as demonstration schools. These schools are paired with less advanced "partnership" schools to provide technical assistance, ongoing exchange of visits, and intensive training across teacher/administrator teams in structured workshops and conferences. The demonstration schools must make the same commitments as the partnership schools to continue their implementation efforts, to provide for visits to their schools by their partner schools, and to participate in consultation and staff development with those partner schools. In 1995 the IMGN became an intensive subset of schools within the broader network of schools supported through Carnegie Corporation's Middle Grade School State Policy Initiative (MGSSPI). The Project on High Performance Learning Communities is built on the premise that, despite the strong appeal of the principles of Turning Points and the state-level plans that have resulted from MGSSPI efforts, the degree to which implementation of these recommendations -- or of any other current restructuring and reform prescriptions -- will produce the desired results is not clear. There are a great number of theoretical and empirical studies pertaining to the development and education of young adolescents. There are also several notable qualitative case studies that typically focus on one or several schools undergoing restructuring or implementing Turning Points-related recommendations.2 There is, however, a dearth of empirical research -- especially intensive longitudinal evaluation studies -- on school restructuring that focuses directly on its impact or informs its design and implementation. Joseph Murphy argues that very little of the current understanding of restructuring efforts has come from studies of what is actually happening in districts, schools, and classrooms that are engaged in the reform of schooling. His conclusion that "there is a notable paucity of empirical studies and research on school restructuring"3 has been echoed by other researchers.4 Similarly, Valerie Lee and Julia Smith state that, despite the wealth of conceptual discussions of school reform, "only one study, in a single school district . . . has evaluated the effects of school restructuring on student outcomes."5 It has also been suggested that the structural elements of school reform, especially as they relate to recommendations for middle-grades education, have been largely accomplished, adequately understood, and accepted. Among authors expressing this opinion, John Mergendollar argues that change in instructional and curricular practices in schools has moved forward far more slowly than change in structural areas, such as the development of schools-within-schools and teacher/student teams, and that data pertaining to instructional changes that can inform policy and practice changes are seriously lacking.6 While both our own work and the research literature strongly support the view that there is a paucity of evaluative information on changes in pedagogy, neither the research literature nor our own findings to date support the contention that structural changes have been institutionalized to the degree that we no longer need to give them further consideration. For example, preliminary findings in our parallel study of the middle-grades schools that are participating in the MGSSPI show that even relatively "mature" and highly motivated middle schools (i.e., schools that have been teaming and attempting the implementation of practices consistent with Turning Points for several years or more) have not realized the full extent of structural changes that would fulfill the recommendations of Turning Points. Although a common response to Turning Points has been "We are already doing that," Anthony Jackson notes that "few of the recommended actions, although frequently proposed, are actually practiced in schools."7 A national study by Joyce Epstein and Douglas Mac Iver provides additional support for this view. These researchers found that the majority of the schools participating in their study had neither teacher-based advisory (or similar) programming nor team structures in which there was common planning time afforded to teachers on a daily (or nearly so) basis. The authors state, "At this time most schools do not use interdisciplinary teams, including about 60% of middle schools and 75% of schools with other organizations [that have middle grades in them]."8 There is a clear need, then, for additional research that directly addresses the process of middle-grades restructuring and its impact. Although a better-developed research base does not, by itself, ensure more successful reform efforts, without such a foundation the progress and fruits of reform efforts will continue to be disappointing. The urgency of the need for such research that is felt by those concerned with the reform of education for young adolescents is evident in John Mergendollar's comment, "A past characteristic of middle-grades reform that has slowed change is that it has been powered more by rhetoric than by research. . . . Reform cannot be left to rhetoric alone. Research is needed to validate, guide, and extend it."9 In this article we describe the evolution and current status of one effort to evaluate a comprehensive, integrated set of recommendations for transforming the education of young adolescents. Specifically, we sought to assess and evaluate the process of implementation of the recommendations of Turning Points for middle-grades reform, as well as their impact on students' academic achievement, socio/emotional development, and behavioral adjustment. We particularly wished to explore the association between the levels of implementation of the reforms that participating schools attained and relevant student outcomes. As we will explain in greater detail below, the research is also designed to provide reliable data to reforming schools. Such data are required to make effective decisions pertaining to school management and to develop strategic plans for school improvement. Moreover, schools must have available to them systematic and objective data on student achievement outcomes that relate to the reforms they are making, as well as data on the degree to which changes that are implemented are consistent with the Turning Points dimensions. Such data will enable schools to create greater congruence between intended and actual changes. Research Questions The overall evaluation research initiative on which this article is based focuses on several core questions, including:
Conceptual and Measurement Models In conducting research on a school reform effort, it is necessary to use a measurement model and assessments that will provide data about the actual levels and patterns of implementation. Douglas Mac Iver and Joyce Epstein note, "Few middle grades schools have implemented many of the practices recommended for the education of early adolescents, and even fewer have implemented them well."10 Larry Cuban and others have argued that structural changes that have been at the core of most middle-grades reform recommendations may do little to change the fundamental experiences of students. According to Cuban, such structural changes as creating teacher teams, implementing advisory periods, and making scheduling modifications that allow for longer and more intensive class periods are "too distal to improve students' learning and attitudes."11 Further, such critics argue that there is often too little linkage between the implementation of these structural changes and actual changes in the practices and experiences of teachers and learners. For these reasons, it is clear that a research design that simply assigns schools to categories of "implemented" or "nonimplemented" or "experimental" versus "control" will not be informative. The reality of school reform does not lend itself to such dichotomous categories. Changes across schools are not simply either/or affairs. Rather, reform is an evolutionary and developmental process. Thus, when considering the differences between groups, it is critical to know that the researcher's distinction between schools is meaningful and reflects real differences in the levels and forms of what is occurring in these settings. It should also be understood that these issues are endemic to scientific research. Even "classical" experimental design requires that there be "manipulation checks" and assessments of the impact of various levels and combinations of exposure to the independent variables if the investigators are to plausibly assert that the results obtained are attributable to the experimental manipulations. In research to evaluate school reform, the "manipulation check" can and should be the systematic assessment of the degree to which the intended transformations have occurred. If measured carefully, variations in the levels of implementation can provide the opportunity to answer important questions about the degree of change and the interactions between changes that are necessary to obtain the desired results. Given the resource constraints under which schools operate, it is critical to clarify the complex relationships among the Turning Points recommendations, as well as the point at which additional resources or changes yield diminishing returns. Hence, in our measurement model and instruments, we included direct assessments of changes in the teaching/learning process and context at multiple levels (e.g., school, grade/team, classroom). We also started with an open stance concerning the potential importance of the structural changes that have been at the core of the recommendations of middle-grades reformers. That is, we sought to evaluate whether, and if so how, these structural changes by themselves produced important changes in student outcomes or changes in the teaching and learning processes that we and others posit may be central to improved student performance. We also sought to define our implementation elements in ways that were more conceptually driven than was typical of prior work. Thus, as we assessed the impact of both structural and implementation variables, our focus was on the degree to which each added to the manifestation of one or more of the eight overarching conceptual goals of Turning Points. For example, rather than simply looking at whether or not one school is "teaming" in a checklist fashion, we sought to understand the defining features of teams that related to differential levels of the attainment of "small communities for learning" -- one of the core recommendations of Turning Points. Furthermore, we sought to explicate the influence of such structural variables and operational norms as team size (both numbers of teachers and students), student/teacher ratios, stability of teams, frequency and duration of common planning periods, and length of time spent in teaming as they related to the experience of the school as a smaller, more orderly, more predictable, and more supportive community for learning; to the attainment of other sought-after reforms (e.g., changes in instruction); and, ultimately, to student achievement. The core of the evaluation is a "compressed longitudinal" design. Such a design relies on obtaining observations of sets of schools that are at different phases in the transformation process and following them over time. This procedure enables us to evaluate the impact of shifts across levels of implementation in less time than would be required if all schools in the study were just starting the process. Hence, across a period of six to seven years, we can evaluate the impact of all levels of changes that might take 10 years to assess if all schools began the process at the same (low) level of implementation. Further, such a design allows for several replications of each key analysis. That is, since we are admitting successive cohorts of schools into the analyses, we can test whether subsequent cohorts show patterns of change that are similar to those exhibited by the preceding cohorts. Inherent in our measurement model are the three primary domains for assessment that any adequate evaluation of an educational program or policy reform needs to contain.12 These include:
Before proceeding, it is worth noting that one of the reasons that the IMGN and the Project on High Performance Learning Communities selected Turning Points as the focal set of recommendations for our reform efforts is that it is one of the very few reform proposals that yields the kinds of testable, replicable, systematic, and comprehensive recommendations for change, and accompanying derivable hypotheses, that are necessary for both effective school reform and sound research. Our conceptual and measurement model considers the way in which such structural and demographic variables as the size of the school, the per-pupil expenditures, the student/teacher ratio, and the social and ethnic composition of the student body shape the nature and impact of the implementation of each recommendation, separately and interactively, on such school-level conditions as teachers' quality of life and job satisfaction, school and community climate, supports and resources available to students, and student outcomes (academic and affective). For example, implementing structural changes -- such as the adoption of governance teams, interdisciplinary teaming, and teacher-led advisories -- will have an impact on a variety of outcomes for the school, the teachers, and the students, with the specific impact shaped by the extant school structure and demographic conditions. Of critical concern for understanding the importance of each of the Turning Points recommendations are 1) the degree to which the implementation of a recommendation contributes to the attainment of changes in school and student outcomes and 2) the degree to which, as schools implement increasing numbers of the recommendations, combined effects are evident. Each of the recommendations in Turning Points, as it is adopted, will have both direct and mediated influences on a variety of outcomes. Moreover, the longitudinal nature of the current study allows us to consider the reciprocal relationships that may develop over time. For example, as teachers become more invested in students, student adjustment should improve. In turn, this improvement should affect the feelings of teachers about students. We have found that the most successful sequencing for the implementation of school transformation efforts involves bringing about key changes in leadership processes and in staff attitudes about the importance of the practices that are recommended (what Jeannie Oakes and her colleagues would call "normative" and "political" changes13), as well as achieving critical shifts in the operational norms and structures of the school.14 Schools that undertake such changes first will obtain larger and more rapid changes in the implementation elements relating to the teaching/learning practices and school characteristics recommended in Turning Points. Samples Overall, schools participating in the IMGN network represent the full range of geographic, demographic, and size characteristics of all schools in Illinois, including urban, suburban, and rural schools. For example, the sample includes schools that have populations approaching 2,000 students and schools with fewer than 200 students. The full range of student characteristics is also represented, with schools whose populations are primarily economically disadvantaged and/or from African American, Hispanic, or other racial and ethnic minority backgrounds; schools that have highly heterogeneous student populations; and those that have highly affluent populations or whose students are predominantly of European American background. Data Analyses and Report of Baseline Findings Before we turn to any further discussion of findings and other accomplishments of the project during its current period, we ask readers to consider our results as preliminary, for a number of reasons. The results discussed here are all based on only data that were collected through our third year of large-scale data collection. Jeannie Oakes and Gretchen Guiton and others have noted that three years or less is not an adequate period for schools to accomplish the full range of changes required to successfully implement the recommendations of Turning Points or other school reform efforts.15 Our more recent results are highly congruent with those reported in this article and add weight to the current findings. But analyses of these more recently obtained data are not complete. Once schools enter the IMGN, they typically spend the first year in the project planning for the actual start of the restructuring process and participating in inservice and AIMS/IMGN institute activities. During the second year of participation, the vast majority of partnership schools move forward rapidly in their efforts to implement teaming, common planning time (most with five days per week), and teacher-based advisory structures. Our preliminary data indicate that the first year of implementation may be one in which there is considerable disruption. It may take an additional year or two for the process to settle down, for the organizational changes to become institutionalized, for teacher norms to move strongly toward acceptance of the assumptions that underlie the restructuring, and for changes in actual teaming and classroom practices to begin to emerge. It must also be recognized that if, for example, a school has grades 6-8 in its middle-grades program, it may be five years or more before a cohort of students has been exposed to a smoothly and fully realized middle-grades program. Given these issues, it is critical to understand in the analyses below that those schools in the "most fully implemented" group are typically some distance from actually being "fully implemented." Indeed, we -- and the schools -- know that they still have a long way to go. This is one reason that we are continuing to follow our cohorts over a much longer period of time. Hence, in the following analyses, when we compare schools at varying levels of implementation it must be clear that the comparisons are among schools at relative levels of implementation. To anticipate the findings somewhat, however, it does appear that our most fully implemented schools are dramatically different -- and better -- places for students to learn and teachers to teach than those at lower levels of implementation. Findings and Issues for the Future The analyses of the data thus far have taken a variety of forms, ranging from correlational, hierarchical regression, and structural modeling approaches to those that deal with the data in a less linear fashion. As we explore our data in these different ways, we are learning a great deal not only about our own data, but also about issues that may require attention in future studies of school restructuring. One of the major overarching findings is the importance of considering the comprehensiveness and level of implementation of the Turning Points recommendations. There are a number of ways in which the effects of scope and intensity are manifested. For example, the presence or absence of a particular element of the program may affect the levels of implementation of other components. We can also look at the degree to which each element individually, and in combination with others, exerts an impact on student achievement, health, and mental health. Consideration of a few of the findings that illustrate these points may be helpful here. School- and cross-school-level analyses. One key set of analyses involves cross-sectional comparisons of schools that have attained different levels of implementation of overall middle-grades practices, including some schools that have not attempted to change and thus serve as "traditional" comparison schools. The central question of these analyses is the extent to which schools that have attained different levels of implementation show concomitant differences in student achievement, behavior, health practices, and socio/emotional adjustment for all students and for targeted subgroups (e.g., students who are economically disadvantaged or students during the transition year coming from differing types of feeder school configurations). A related set of analyses is concerned with changes within schools (and sets of schools) in their practices over time and the way that these changes affect the subsequent functioning of the teachers, students, administrators, and parents. Let us briefly turn to a representative set of the findings of these analyses.16 In the initial year of the study (1990-91) there were 11 schools in the sample. During the 1991-92 school year, the number of schools was increased to 31, and we obtained second-year data on the 11 original schools. While the number of schools now in the sample has reached 97, the results presented here pertain to the 31 schools that were a part of the IMGN in the 1991-92 school year. Analyses of the larger samples (Table 1) are ongoing.
Employing data we had obtained concerning key structures and resources, decision-making patterns, teacher norms, and instructional patterns, we classified each of the 31 schools into one of three levels of implementation (LOI), based on what we have labeled the "Carnegie Index of Middle School Transformation." Briefly, the nine schools classified as being in the highest LOI were those that had accomplished the majority of structural changes "at high levels," that is, in ways that most reflected the constructs they were intended to manifest rather than simply being present in a checklist sense. For example, schools in which teams had four to five common teacher planning periods per week, in which there were relatively small numbers of students (not more than 120) on the team, in which there were relatively low teacher/student ratios (one teacher per 20-25 students), in which advisories occurred with relatively high frequency (e.g., four or five times per week), and in which teacher/student ratios in advisories were approximately one to 22 or less were weighted as having more fully implemented the Turning Points goal of creating "small communities for learning." In addition, those schools showed critical changes in the school context and in the teaching/learning process. Schools that showed patterns of instruction, decision making, and teacher norms consistent with the educational practices that attended to the developmental issues of adolescents were also generally included in the highest group. Schools in the initial "partial" group were those schools that had implemented at least some of the key structural changes at high levels but were not yet showing the levels of instructional and contextual changes that were typical of the high group. Generally, the 12 schools in the "partial" group had made the structural changes either more recently or at lower levels than those in the most fully implemented group. Finally, the "low-implementation" group included both those schools that were not making changes and schools that had recently joined the IMGN but had yet to make significant progress on any implementation front. In considering the findings that follow, we remind readers that the assignment of schools to LOI groups was done on the basis of their relative similarity (within groups) and relative difference (across groups), not on the basis of some absolute scale. Moreover, in assigning schools to groups and, more specifically, in establishing "boundaries" between groups, we also considered socio/demographic characteristics of the schools to maximize comparability of the groups. As a result, there are three sets of schools that, although clearly differing in level of implementation, are demographically comparable in terms of size, percentage of students eligible for free/reduced-priced lunch, and per-pupil expenditures. It is not the case, as some might suspect, that the highly implemented group is made up entirely of affluent suburban schools and that the least implemented group consists of poor urban schools; rather, each group contains a representative mix of schools, reflecting the diversity of schools in the sample. Student outcomes. Figure 1 shows the average achievement scores in reading, mathematics, and language arts that were obtained by schools in each of the three groups. In these schools are more than 15,000 students and nearly 900 teachers. The state mean score on each of these achievement dimensions is 250 with a standard deviation of 50.
The data show that, across subject areas, adolescents in highly implemented schools achieved at much higher levels than those in nonimplemented schools and substantially better than those in partially implemented schools. Average achievement scores shown are a composite of sixth- and eighth-grade scores. (The states' achievement tests are constructed so that scores across grade levels are comparable and can therefore be averaged to create a single schoolwide composite, as we have done here.) It is important to note, however, that combining sixth- and eighth-grade scores into a single index is a more conservative test than if only eighth-grade scores were used, which some would argue represents a truer assessment of the power of the conditions that appear to influence achievement. Indeed, when we considered only students who have had such longer exposure to these conditions (i.e., our eighth-grade scores), then the differences in achievement scores between the groups are substantially larger than with the combined sixth-/eighth-grade index. A critical feature of our design is that we have attempted to obtain multiple convergent measures on aspects of the implementation of reforms as well as on outcomes across related dimensions. Hence, for these initial LOI analyses there were a number of other student outcomes that were considered. Additional indicators of achievement included the percentage of students who were performing at grade level and scores in subsets of schools that administered the Iowa Tests of Basic Skills and the California Tests of Basic Skills. Generally, these additional indicators showed strong association with the state-level scores. For example, in our sample, the state achievement scores typically correlate (.88 or greater) at the individual student level with these and other national achievement tests. We also examined different domains of student outcomes as they related to the level of implementation that schools had obtained. These included teacher ratings of student behaviors, as well as student self-reports of depression, fear, worry, anxiety, and self-esteem. The patterns of teacher reports of student behavioral problems, including aggressive or moody/anxious behavior and learning-related behavior problems, are highly correlated with the patterns noted earlier within achievement data, but in the desired opposite direction (see Figure 2). That is, in the most fully implemented schools, teachers report far lower levels of student behavior problems than do teachers in less implemented and nonimplemented schools. Similarly, teachers in the partially implemented schools still perceive students as showing fewer behavioral problems than those in the least-implemented schools.
Similar analyses for student self-reports of a representative set of the domains of socio/emotional function were converted so that the scales could be displayed in equivalent ways. These data show patterns that are highly convergent with the achievement and behavioral data. Specifically, students in the more fully implemented schools are less fearful of being victimized, are less worried about something bad happening at school and about the future, and have higher levels of self-esteem (see Figure 3). As before, students in the partially implemented schools show better outcomes than those in the nonimplemented schools, but not as good as those in the most fully implemented schools. Recall, too, that because the groups of schools are comparable with regard to important variables of school and community contexts, it is not the case that, for example, the "fearfulness" of youths in nonimplemented schools might be justified because of unsafe conditions in the neighborhood.
Clearly, across quite different types and sources of data (e.g., achievement tests, teacher reports, student self-reports), there are distinct differences between schools that have attained different levels of implementation of the Turning Points recommendations. Such patterns are important indicators of the reliability and validity of the joint outcomes. Nevertheless, these data are limited by their cross-sectional nature. The core of the current evaluation is a long-term longitudinal study in which we are following schools as they move through different levels of implementation. We will then consider how such changes in implementation within schools are associated with shifts in contextual conditions and, ultimately, with student achievement and related outcomes. The focal question here is, Do student performance and adjustment improve as the level and quality of implementation increase over time? As before, schools in the longitudinal analyses are categorized according to level of implementation. These categorizations, however, have been expanded to consider both the level of implementation obtained -- as in the cross-sectional analyses -- and the degree of change over the past year. Consequently, a "level 5" school is one that was nonimplemented or only marginally implemented in the previous year and had made no changes during the current one. "Level 4" would include those schools that were nonimplemented or only marginally implemented in the previous year but that had over the intervening year initiated planning processes and begun to make some structural changes. These changes, while important, will require further refining to be truly effective. For example, level 4 schools might have moved to a system in which there are teams of 130 to 150 or more students, teachers have perhaps one to two planning periods, and the planning does not yet reflect any instructional changes. By contrast, "level 1" schools include those schools that have attained the highest levels of structural changes, have implemented key changes in instruction and decision making, and are exhibiting continuing refinements in the critical areas of teaching and learning processes and practices. That such refinements are ongoing, again, shows that even our most fully implemented schools have considerable room to improve, particularly in areas of instructional change and in the extent to which Turning Points recommendations are embraced by all teachers within the school. A first set of analyses considered the simple correlations between changes in level of implementation across one- and two-year periods and changes in reading and mathematics scores. As schools move up in their level of implementation of Turning Points recommendations from 1991-92 to 1992-93, the one-year correlation of such changes with increases in eighth-grade reading scores was .51 (p<.001) and with increases in eighth-grade math scores was .30 (p.<.001). Similar patterns were found for two-year changes in implementation level and achievement scores, with correlations of .53 and .35 respectively (both p<.001). It is encouraging to note that longer-term analyses, if anything, tended to yield findings that were as strong as or stronger than those yielded by shorter-term analyses. Similar patterns to those found with achievement score gains emerged when we examined indicators of student adjustment, students' experiences of school climate, and health indices. These data complement the cross-sectional data described earlier, showing that whatever the preexisting levels of student outcomes in these areas, as schools move through levels of implementation of the Turning Points recommendations there appear to be associated gains in key areas of student behavior and socio/emotional adjustment. We also examined, in a comparison group fashion, the relative magnitude of the gains that were associated with differences in levels of implementation. We have four years of data on changes in achievement test scores (from 1990-91 through 1993-94; these data are available even for schools that joined after 1990-91). For changes in LOI, we have data from 1991-92 onward. We considered both one- and two-year changes in math and reading achievement scores (these are the most consistently available data for all schools) across LOI change and attainment categories. In all analyses of both one- and two-year data, there were large and meaningful differences between schools that had reached the highest levels of implementation or those that had made the most progress toward high levels of implementation and schools in which little implementation or relatively smaller LOI changes had occurred. The average gains in math/reading achievement scores across two, two-year periods are displayed in Figure 4. In the most fully implemented (level 1) schools the average gain was nearly 21 points (recall that 25 points is a full half standard deviation on these scales). Schools that had attained high levels of implementation structurally but had done so most recently and thus had rather moderate levels of change in the core teaching and learning processes showed average achievement gains of more than 15 points. Those schools that were not yet highly implemented but that had shown several degrees of LOI gain had average gain scores of nearly 12 points. By contrast, schools that exhibited "intermediate levels of implementation, little refinement" (i.e., little improvement had recently occurred) showed average gains of less than three points, and those schools that had made little or no movement toward implementation showed "negative" average gain scores -- that is, achievement in these schools actually declined.
Taken together, these preliminary findings are extremely encouraging and show what kind of an impact the implementation of the Turning Points recommendations could have on adolescents' achievement and adjustment. Yet as teachers and administrators in our level 1 schools would quickly point out, these highly implemented schools are far from fully transformed, particularly in terms of actual changes in instruction at the classroom level. Hence, if we consider that our most fully implemented schools are only part way there, then the potential positive impact of a Turning Points-style transformation of a school appears to be well beyond the results we have already obtained. This is an issue we will explore further in our ongoing efforts. What will happen if schools fully implement the Turning Points recommendations? How do we get there, and what have we learned about the current process that can help? It is with a brief consideration of these issues that we close. The Case for Comprehensive Implementation Perhaps the most important lesson about implementation we have learned from this ongoing research is that successful reform must be comprehensive and integrative, with careful attention to sequencing and the establishment of some Turning Points building blocks on which other elements can be mounted. That is, there are clear patterns of interdependence among the implementation elements that may require additional consideration by those involved in school reform efforts if we are to fully realize the benefits of middle-grade restructuring. One of the clearest patterns that has emerged from our data is the difference between a "checklist"-based implementation of structural changes and implementation that is "idea-driven" -- that attempts to reflect the underlying constructs and issues in the Turning Points recommendations. Take the principle of "creating small communities for learning" as an example. If one employs a checklist approach here, as typified in many sets of recommendations for middle-grades reform, a school might simply ask itself whether or not it has teacher teaming or interdisciplinary instruction. Unfortunately, being able to "check off" these practices becomes an end unto itself, with little regard as to why these practices should be implemented or in what forms and at what levels they need to be present to contribute to a more effective teaching and learning process. Across our Illinois, MGSSPI, and other samples, schools that say they are "teaming" may have team sizes that range from 60 to 70 students and two to three teachers to more than 240 students and nine to 12 teachers. Student-to-teacher ratios on a team may range from below 20 students per teacher to more than 40 students per teacher. Additionally, the amount of common planning time -- which appears to be a critical element of small learning communities -- varies from no common planning time to the shared use of individual planning times to daily common planning time that is provided in addition to individual planning time for every teacher. There are considerable differences in the costs of these implementation options, and policy makers and school administrators who must make decisions about expending resources need to take these differences into account. Our findings reveal a number of patterns that, if they hold, make a strong case against attempting middle-grades school restructuring "on the cheap." We have found that each of the dimensions of teaming noted above (size, student/teacher ratio, amount of common planning time) appears to have a significant effect on the degree to which other elements of the Turning Points reforms and goals may be accomplished. Our findings indicate that, when these variables are inadequately implemented, they are associated with a variety of problems.
There seem to be critical levels of implementation, below or above which changes make little difference. Going from one day per week of common planning time to two or moving from 200-student teams to 180-student teams seems to matter little. Teams that exceed approximately 120 students, that have fewer than four common planning periods per week, and that have student/teacher ratios beyond the middle 20s tend to show little impact on instructional practices or student well-being. It should also be clear that deficits in any of these elements place severe limits on what is yielded by the others. Thus small teams of students may somewhat improve students' reports of their levels of support and feelings of connectedness to schools, but unless teachers have adequate allotments of common planning times, actual instruction does not appear to change. Indeed, changes in classroom practices that are in keeping with the Turning Points recommendations (e.g., use of small-group instruction, delivery of integrated instruction) appear, not surprisingly, to be significantly associated with changes in team activities. Thus one can begin to see the way in which the desired changes are nested within one another. These findings underscore Seymour Sarason's arguments that, if we are to avoid the "predictable failure of educational reform," we must understand that schools are complex integrated systems, and we must address the full set of operational norms, regularities, and behaviors that may affect or undermine efforts at change in specific areas.17 The interdependence of the elements of reform is also suggested by the point in the implementation process at which the outcomes obtained with students move from the prevention of failure to the enhancement of achievement. In a recent paper on the transition to middle-grades schools, Robert Felner and his colleagues reported that the patterns of relationships between more positive levels of implementation and student achievement, mental health, and behaviors, result -- at least in part -- from the preventive effects of middle-grades structures.18 Small teams and teacher-based advisory programming in particular, especially when the teams are kept in their own areas of the building and away from older students, appear to enable students to make the transition into middle-grades schools without the pronounced declines in socio/emotional well-being and academic achievement that have been reported in some studies of students moving into middle-grades schools and junior high schools.19 Hence, the greater well-being and higher achievement scores attained by students in middle-grades schools that have created small communities for learning appear to result more from declines in these indicators among students in more traditional schools than from gains by students in environments that have only these restructuring elements. This is not a trivial finding. These predictable declines have, in the past, been linked to a full range of socio/emotional problems among youths (e.g., school failure, dropping out, crime, depression, substance abuse, and teenage pregnancy and parenthood). Still, middle-grades reformers have sought not only to avoid the onset of new difficulties but also to enhance the developmental course of students. In our current sample, we have a small but growing set of schools that have shown significant levels of implementation across virtually all Turning Points areas. Our findings suggest that it is in such schools that the desired enhancement effects will accrue. The relationship between a school's level of implementation and its ability to prevent decline or enhance outcomes is even more pronounced in the important case of "at-risk" students. (The general trend pattern relating implementation levels to achievement and adjustment gains is displayed in Figure 5.) Consistent with much prior work, students in more traditionally structured schools show declines in achievement/adjustment indicators. It is not until substantial transformation has been accomplished that preventive effects (that is, the absence of declines that would otherwise appear) are found for these at-risk students. And broad-range enhancements in achievement and adjustment are not obtained until implementation is quite mature, comprehensive, and being conducted with a high degree of fidelity.
Assuming that these patterns hold over time, our findings should encourage policy makers to move forward rather than to stop or move away from restructuring because the results in the early stages are not all that was hoped for. It should not be surprising that it takes fairly comprehensive and intensive levels of implementation for the suggested changes to produce major gains in all spheres of functioning of high-risk students. Often these students live in community environments that may be high in stress and low in opportunity and resources. However, our findings to date strongly support the view that high-quality schooling, well implemented, can make profound contributions to the achievement, mental health, and socio/behavioral functioning of students who are often left behind and for whom there is often a sense that school cannot make a difference in their lives. These data also argue for resources to be used effectively in schools with high concentrations of at-risk students, and, in some instances, for resources to be increased significantly in order to create the necessary conditions for all children to be successful. 1. Task Force on Education of Young Adolescents, Turning Points: Preparing American Youth for the 21st Century (Washington, D.C.: Carnegie Council on Adolescent Development, 1989). ROBERT D. FELNER is director of the National Center on Public Education and Social Policy and chair-elect of the Department of Education, University of Rhode Island, Kingston; ANTHONY W. JACKSON is a program officer at the Carnegie Corporation of New York and the primary author of Turning Points; DEBORAH KASAK is executive director of the Association of Illinois Middle Level Schools, Urbana; PETER MULHALL is a research scientist at the University of Illinois, Urbana; STEPHEN BRAND is a research assistant professor of education at the University of Rhode Island, Kingston; and NANCY FLOWERS is coordinator of research programs at the University of Illinois, Urbana.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
