 |
What Did the Massachusetts Teacher Tests Say About American Education? By R. Clarke Fowler Will the mechanisms that policy makers are using to improve schools of education -- more certification tests, higher cut scores, and severe penalties for institutions that fail to meet specific pass rates -- actually deliver the increased accountability and better teachers that policy makers have promised? Mr. Fowler's experience in Massachusetts suggests that the answer is no. |
DURING the summer of 1998, magazines, newspapers, and the electronic media widely reported the results of the first administration of Massachusetts' new tests for aspiring teachers. The results were notable because 59% of those tested failed an exam that state officials described as a test of eighth- to 10th-grade skills.
This story was also notable because of the remarkable series of events that accompanied the announcement of the test scores. The state board of education sparked outrage when it initially set the cut scores at one standard deviation below the levels recommended by its own panels of experts. It sparked controversy when it reversed this decision just one week later, in a vote preceded by the unexpected departure of Frank Haydu, then commissioner of education, who resigned with the comment that "the political forces have been unleashed."1
This story also made headlines because of the heated and highly quotable rhetoric that surrounded the affair. Thomas Finneran, the speaker of the Massachusetts House of Representatives, declared: "I'll tell you who won't be a teacher. The idiots who took that test and flunked so miserably -- and, of course, the idiots who passed them."2
At the heart of this summerlong brouhaha, though, was the oft-repeated statement that nearly 60% of aspiring teachers had failed a test of "basic skills." As this new "fact" spread across the country, U.S. Sen. Jeff Bingaman (D-N.M.) declared the results of the Massachusetts tests an "unflattering snapshot of the state of teacher preparation in America."3
This snapshot has had a powerful impact. It launched scores of op-ed articles, helped shape the Higher Education Act of 1998, and pushed teacher quality to the top of the nation's agenda. This picture had such impact because many perceived in it the principal reason for the purported "failure" of American education: our students aren't learning enough because our teachers don't know enough. For example, when editorialists in Georgia, writing in the Augusta Chronicle, asked, "Wonder why Johnny can't read, write, or add?" they answered by pointing to the results of the Massachusetts teacher tests.
But how accurate is this snapshot that led policy makers and commentators to draw such sweeping conclusions? According to official data and documents, this now three-year-old picture was not at all accurate. Rather, it presented a grossly distorted view of aspiring teachers and of teacher preparation programs. In fact, much higher percentages of candidates have since passed the Massachusetts teacher tests, now called the Massachusetts Educator Certification Tests (MECT). What's more, passing the MECT reflects a higher level of knowledge and skills than politicians have admitted. Finally, because there are serious questions about the quality of the tests that produced the original snapshot, it is impossible to know exactly what the results of the Massachusetts tests have to say about teacher preparation in Massachusetts -- let alone in America.
How Many Have Passed the MECT?
To qualify for certification, aspiring Massachusetts teachers must pass two separate four-hour exams: a subject-matter exam and a two-part literacy exam (composed of a reading test and a writing test). When reporters mentioned the passing rate on the first administration of the MECT, they were referring to candidates who had taken and passed, in one eight-hour sitting, three distinct examinations in reading, writing, and a subject area. And when they referred to the "failure rate," they were referring to candidates who had taken all three tests in one day but had not passed them all.
When the state board of education finally settled on a cut score in July 1998 (for an exam that had been administered in April 1998), 70% of the 1,800-plus test-takers passed the reading exam, 59% passed the writing exam, and 62% passed their respective subject exams. Just 41% of those who took all three exams actually passed all three.
These results differ substantially, however, from the cumulative results that the state department reported for the 1998-99 year, and they differ from the results that I calculated for the entire testing program. As of June 2000, 89% of test-takers had passed the reading section (up 19% from the first administration), 85% had passed the writing section (up 26%), and 79% had passed the subject exam (up 17%). Consequently, I estimate that approximately 73% of all test-takers who have taken all three parts have passed the MECT, an increase of more than 30% over the April 1998 results.4
A number of factors appear to have contributed to the enormous disparity between the results of the first test and the current cumulative results. First, as with any new test, it is common to find candidates earning comparatively low scores on the first few administrations and higher scores on later administrations. However, Massachusetts surely depressed scores more than usual because it committed such unusual and egregious errors when it first administered these tests.
Initially, the state department informed test-takers that the first two administrations of the MECT would not be used for certification decisions. Less than two weeks before the first administration in April 1998, the state department reversed itself and announced that the test would count. By the time test-takers learned that the MECT would count, no study materials were available. The state had withdrawn the study guide because it did not describe, among other items, an unusual dictation exercise that had been added to the test at the last minute. Two members of the state board of education, John Silber and Edward DeLattre, had prevailed upon the test contractor, National Evaluation Systems (NES), to add this item.5
Second, almost all the candidates (97%) who sat for the first administration of the MECT took all three parts (reading, writing, and subject area) on the same day. Not surprisingly, the percentage of candidates opting to take all three parts in a single day has dropped precipitously, from 97% on the first administration, to 36% for the sixth administration, to approximately 27% for the 10th administration.
Third, a relatively high percentage of test-takers have passed different parts of the MECT after failing on their first attempt. Over the life of this testing program, 59% of repeat test-takers have passed the reading exam, 44% have passed the writing exam, and 40% have passed the subject exams. Indeed, my calculations indicate that repeat test-takers are two to three times more likely to pass the MECT's reading and writing exams than to pass comparable exams that Educational Testing Service gives to preservice teachers.6
There is insufficient space to discuss other factors that might contribute to this disparity, such as institutional test-preparation efforts and the availability of some study materials. It is clear, though, that the difference between the April 1998 and the more recent cumulative pass rates is enormous, more than 32%.
What Level of Skills Is Tested by MECT?
To many, this difference between pass rates may seem unimportant. After all, as one editorial writer put it, "It was astounding enough that so many failed, but it was made worse by how spectacularly they failed. Some of the tests were simpler than the high school graduation exams teachers are supposed to prepare students to pass."7
When Massachusetts officials reported the results of this exam, they described it as remarkably easy, a test of basic skills and knowledge. How basic? The first and most widely circulated estimate appeared in an Associated Press report that newspapers across the country carried: "The department of education released a sample of the exam, which board of education Chairman John Silber said seemed to be at about an eighth-grade level."8
Other, less widely circulated estimates were higher, but not by much. Speaker of the House Thomas Finneran called the MECT a ninth-grade exam: "I got chills up and down my spine because it was described to me, quite accurately, as a test that a reasonably educated ninth-grader could pass."9 The highest published estimate appeared in John Silber's New York Times op-ed article, in which he described the MECT as "a test that a bright 10th-grader could pass without difficulty."10
However, a series of official materials, all produced before the summer of 1998, contradict the preceding statements. In February 1997, when the state department first asked contractors to submit bids for the MECT, it stated:
The level of communication and literacy skills targeted for PreK-12 students in all grades and subject areas entails a correspondingly high level of competence on the part of teachers, going well beyond basic skills. The test of communication and literacy skills will assess the proficiency of candidates at a level normally required by the junior year of college.11
In December 1997, when Robert Antonucci, then commissioner of education, selected NES as the test contractor, he told the state board that he needed to work out two issues before signing a contract. The first issue was relatively minor, but the second was important:
The second issue, which will require a lot more work on our part -- and I have been in contact with the Chair [John Silber], who also has very serious concerns about the quality of the test itself -- is that the vendor ensure that the tests do focus on the college level of the specific content being tested. It's the strong belief that I have, it's the strong belief this Board has, when we sent out the RFP.12
In January 1998, when the state department distributed "Questions and Answers," a document describing the MECT to prospective test-takers, it wrote:
What level of knowledge will be tested? The test of communication and literacy skills will assess the proficiency of candidates at a level required for a bachelor's degree. Subject tests for specific certificates will assess proficiency and depth of understanding of the subject required for a baccalaureate major.13
Clearly, there is an enormous gulf between the way state officials described the MECT before and after its first administration. Which description is accurate? Two independent reports speak to this issue. The first report is a content analysis of teacher licensing examinations commissioned by the Education Trust. The authors wrote that, based on their inspection of sample questions, many of the questions on the Massachusetts tests assessed college-level content. The authors said they were
impressed by the sample items for the Massachusetts literacy and communication skills exam. . . . These questions, in the words of Don Jones, were "of a higher degree of complexity and expectations than any of the others we looked at. . . . In contrast, . . . the majority of tests we examined were dominated by high school level materials.14
The authors also praised the complexity of sample items from a number of the MECT's subject exams, including the elementary exam, which they suggested tested knowledge equivalent to "the general knowledge acquired in a four-year liberal arts degree program."15
The Advanced Literacy Skills Study Group (ALSSG), composed of professors at the University of Massachusetts, issued a report on the MECT's literacy exams. The ALSSG concluded that these exams, which ask candidates, among other things, to take a dictation and to define both words and parts of speech, are neither complex nor basic but archaic. They are archaic in that preservice teachers "are being tested in part for literacies that belong to the eighteenth and nineteenth centuries. They are being assessed on their ability to act as scribes, on their ability to recite memorized definitions of parts of speech or vocabulary words out of context."16 From this perspective, the MECT literacy tests are challenging to the extent that they assess candidates on a narrow set of archaic "literacies for which their education, rightly, has not prepared them."17
Absent a more rigorous independent study of the MECT, one that would allow investigators to study complete copies of these exams as well as to compare candidates' performance on the MECT with their performance on other tests, it is impossible to say what level of skills the MECT is testing. It is indisputable, though, that the state department stated at the outset that it wanted tests that would assess college-level skills, that the commissioner of education stated that he would not sign a contract without assurances that the tests would assess college-level skills, and that official documents informed test-takers that the MECT did indeed assess college-level skills.
When the national media assembled to hear Massachusetts policy makers report the results of these tests, however, the same officials who had earlier insisted that the MECT test college-level skills now stated that the MECT tested eighth- to 10th-grade skills. And when the state department issued an updated version of "Questions and Answers" two months later, it no longer discussed the level of skills assessed by the MECT.
If the exams that make up the MECT are valid and reliable, and if they assess the level of skills that Massachusetts officials insisted that they test, then the public should disregard the snapshot of teacher preparation that the media distributed three years ago and replace it with a more accurate picture. In this revised picture, many more aspiring teachers (approximately 32%) have passed a test that is more rigorous than state officials led them to believe. Such a picture would indicate that teacher preparation in Massachusetts, while perhaps not as good as it should be, is not as bad as state officials made it out to be.
However, it is still not clear whether this more recent picture is accurate. Incredibly, even though Massachusetts has now administered the MECT to more than 60,000 aspiring educators over the last three years, there are still serious questions about the reliability and validity of these tests.
Are the MECT Exams Reliable and Valid?
Even before the state released the results of the first administration of the MECT, teacher educators raised questions about the validity and reliability of the exams because NES disregarded national standards for testing and did not issue a technical report.18 Teacher educators expressed even more concern once the results were released and they learned that many skilled candidates had failed the exams. The most compelling case is that of Nancy Schmeing. When she initially took the MECT in July 1998, she passed the physics content test and the writing exam but failed the reading exam (by 11 points) with a score of 59. This was puzzling because Schmeing had earned her Ph.D. from MIT, was a Fulbright fellow, had published more than 20 articles, and currently translates complex technical articles (from German to her native English) for Fortune 500 companies. Somehow, though, the MECT initially determined that Schmeing could not read well enough to teach in Massachusetts. When she took it again, she passed with a score of 93.
Alarmed by such anecdotal evidence and by the absence of a technical report, several colleagues and I conducted a study of the psychometric qualities of the MECT.19 Our analysis of a sample of scores from the April and July 1998 administrations indicated that the reading and writing tests were unreliable and of doubtful validity. Specifically, we concluded that the reading and writing tests 1) are unreliable measures as indicated by our calculations of test-retest reliability; 2) contain almost two to three times the degree of error as well-developed tests; 3) have high false-pass and false-failure rates, as indicated by huge fluctuations in scores for those candidates retaking portions of the tests; and 4) contain questionable content and are poorly designed, as indicated by the lack of relationship between candidates' reading and writing scores. We recommended that the state immediately suspend using the MECT while independent experts audited these tests.
The authors of the ALSSG report also questioned the validity of the MECT's literacy exams, noting discrepancies between what these exams are supposed to test and what they actually test. For example, they noted, "While the test objectives [for the writing test] suggest application to a range of genres and purposes, the test items include only expository and persuasive texts and samples addressed to teacher audiences."20 They also doubted whether the dictation exercise accurately assesses what it is supposed to test -- language mechanics: "Experts in language and literacy . . . point out that this exercise actually calls for a very complex system of skills -- many unrelated to knowledge of writing mechanics -- and thus is a peculiar and inefficient means of measuring that particular body of knowledge."21
NES issued a five-volume technical report in the summer of 1999 -- 15 months after the first administration of the MECT and after four test administrations.22 The company wrote that the state department and NES took careful steps "to establish and verify the validity of the tests in the program" and assured the public that it can have confidence in this testing program. However, the data NES presents to support the preceding claims are not persuasive.
First, the company neither responded to nor acknowledged the problems that my colleagues and I had noted in our study. This is odd because NES could have seriously challenged these findings, which were based on analyses of the scores of a sample of test-takers, had it produced more acceptable findings based on analyses of the scores of the entire population of test-takers.
Second, the company presented insufficient evidence to support the claim that the MECT exams are valid. For example, NES could have correlated test-takers' scores on the reading and writing exams with their scores on comparable literacy exams, such as the SAT. A high correlation on this kind of analysis, which psychometricians refer to as concurrent validity, would have supported the NES claims. No such analysis was conducted, however.
Third, Larry Ludlow analyzed the NES technical report and uncovered statistics that suggest that some of the content exams (e.g., special education and early childhood education) contain test items that may be flawed or may have been scored incorrectly.23 The presence of such flaws in the NES certification tests in Alabama led a federal judge to order the state to stop administering them, ruling that NES "violated the minimum requirements for professional test development" when it developed Alabama's tests.24
Commissioner Driscoll announced in July 1999 that he would ask the National Academy of Sciences to nominate a panel of experts to review the MECT. Exactly one year later, when the state department had still not produced such a review, the legislature passed budget language requiring the department to hire independent experts to audit the MECT. The governor vetoed this legislation, saying that the state department was "just commencing" the review that Commissioner Driscoll had promised to undertake one year before.25 When the department announced in March 2001 that it had finally hired outside experts, however, it stated not that they would audit the current tests but that they would "advise the Commissioner on technical and implementation issues" of the next generation of tests.26
What Have the MECT Exams Revealed?
Until independent experts conduct a full audit of the MECT, it is impossible to say exactly what these tests reveal about the state of teacher preparation in Massachusetts or anywhere else. However, the reaction to the MECT has revealed some important features of American education that I discuss next, beginning with inadequate press coverage.
Inadequate press coverage. Media coverage of events surrounding the MECT has been inadequate in a number of ways. Although the media reported some facts incorrectly, more typically they simply failed either to separate facts from state officials' interpretations or failed to examine facts that officials had overlooked or ignored.
When the state released the results of the second round of testing in July 1998, first-time test-takers' pass rates increased sharply, rising from 59% to 78% on the writing exam and from 51% to 70% on the communications and literacy exam. Many teacher educators perceived these increases to be evidence that the first administration of the tests had dramatically depressed test-takers' scores. National media outlets, however, did not report such perceptions. Instead, they reported state officials' interpretations of this second round of testing.
Why did first-time test-takers' pass rates increase sharply? The Washington Post accepted Commissioner Driscoll's explanation that "the 'extraordinary public attention' paid to the previous high failure rate may have led July candidates to take the test more seriously while dissuading others, who could not pass. Their educational backgrounds may also have differed."27 In other words, the explanation for the marked increase in pass rates lay not with the test but with the test-takers.
The national media also relied heavily, if not exclusively, on the interpretations of state officials as to what it meant to fail these tests. When state officials showed reporters samples of particularly weak answers, the Associated Press reported that these samples "showed some test-takers, when trying to rewrite sentences, misspelled words a 9-year-old could spell -- even though the words were right in front of them. Some wrote at a fifth- or sixth-grade level."28 Ronald Hambleton wrote at the time, "I suspect that the work of the very lowest performers was used to highlight poor writing skills; but it would be wrong to generalize from this sample of work to the population of failing candidates."29 Such admonitions were not noted in the media, which strongly implied that anyone who failed the MECT was practically illiterate.
However, since a public official had stated that the MECT assessed eighth-grade skills, reporters had reason to believe that anyone who failed these tests was barely literate. After all, logic dictates that a person who fails an eighth-grade test must be operating on a sixth- or seventh-grade level.
On the other hand, when some media outlets reported that the Education Trust had praised the MECT exams for their complexity, they failed to mention that such praise contradicted their previous reporting. Specifically, both the Boston Globe and the New York Times repeatedly described the MECT exams as tests of "basic skills" during the summer of 1998. One year later, the New York Times reported that "the Education Trust praised Massachusetts for its rigorous teacher exam."30 And the Boston Globe reported at the same time that "the panel of educators who reviewed the tests nationwide were impressed by the rigor and complexity of the Massachusetts test."31
It is curious to note that the Education Trust also has spoken on both sides of this issue. Here is how one reporter wrote about his conversation with Amy Wilkins, one of the Trust's senior associates: "The Massachusetts test wasn't any harder than other teacher certification tests, she said. 'As far as I can tell, a bright ninth- or 10th-grader could pass that test,' said Wilkins."32 Even more curious, this conversation occurred on the same day that the Education Trust issued a report that stated, "Massachusetts has devised new and much more rigorous [certification] examinations, especially in the content areas."33
The myth of immaculate test construction. One factor that may have contributed to the media's apparent reluctance to go beyond the sound bite on this story is the apparently widespread belief that tests, even brand-new tests, invariably tell the truth. For example, Sen. John Kerry (D-Mass.) wrote in an op-ed article that "the scores on those tests don't lie."34
The belief that tests are created perfect is also evident in the outrage that journalists and commentators expressed when test-takers or professors suggested that the high failure rate might indicate a problem with the tests. Steve Forbes fumed on his magazine's editorial page: "After these students in Massachusetts scored so poorly on their certification test, one would-be teacher made the absurd comment that there must obviously be something wrong with the test -- and not with those taking it."35 And the editorial writers for the Boston Globe scolded the deans of 10 schools of education for asking that NES provide a "plan for conducting validity and reliability studies." The Globe stated that "rather than concentrating on minor testing flaws, the deans should identify and analyze major weaknesses in their own instructional programs."36
The most intriguing expression of belief in the inerrancy of the tests is found in the Education Trust's report on certification exams. The authors wrote:
With public support and political will, policy makers and educators can loosen the stranglehold that litigation and psychometrics have on developing licensing exams. They can make them into instruments that signify high professional standards and tests that teachers will be proud to pass.37 (Emphasis added.)
The authors do not indicate, however, how it is possible to give birth to excellent tests without psychometrics.
The best and brightest are born, not made. When Massachusetts released the results of the MECT in the summer of 1998, numerous commentators solemnly urged the country to locate and recruit to teaching the "best and the brightest." The sensible desire to attract skilled people into the teaching profession was often marred, however, by the disturbing number of writers whose language implied that the best and the brightest are born, not made. A college president lamented, "As long as other fields offer financial rewards in excess of teaching, the best brains will continue to be lost."38 A Boston Globe reporter wrote that the relatively high numbers of students enrolled in remedial programs in the state's public colleges and universities "reflect poorly [on these institutions] for accepting students who are not cut out to do college work."39 Editorial writers at the Boston Herald noted that the purpose of the MECT was not to identify who would be a good teacher but "to find out who lacks the raw material to be a good teacher."40 And John Silber stated that the major problem with teacher preparation is that students with "superior native intelligence" usually attend law schools and medical schools but rarely attend schools of education.41
This is not to say that all the preceding writers actually believe, as Silber apparently does, that failure to pass a preservice teacher test should be attributed to biological inadequacy as opposed to inadequate instruction or lack of individual effort. It is to say, though, that the notion that intelligence is inborn is deeply embedded in our culture and our language. Even people who would presumably object to explicit expressions of this idea often accept it implicitly. Such language contradicts the most laudable tenet of education reform -- the notion that all people can learn.
It is perversely fitting, therefore, that Speaker Finneran called people who failed the MECT "idiots," because this term is more than a simple insult. Rather, it is a technical term with a specific meaning in our country's long and often ignoble tradition of intelligence testing. In the early years of this tradition, researchers contended that individuals are divided into "three classes according to their intelligence: feeble-minded (I.Q. below 70), normal (I.Q. 70 to 130), and gifted (I.Q. above 130)."42 Residing among the feeble-minded are morons (I.Q. 50 to 70), imbeciles (I.Q. 20 to 50), and -- last and clearly least -- idiots (I.Q. below 20).
Improving Teacher Preparation
Although I have argued that the Massachusetts teacher tests have little to say about teacher preparation and much to say about public perceptions of American education, I do not wish to imply that teacher preparation is without problems or that policy makers should not work to address these problems. Teacher preparation clearly belongs at the top of the national agenda, for research clearly indicates that teacher expertise is a critical factor in student achievement.43 It can and should be improved, but our public officials need to understand the limits of what can be accomplished by reforming schools of education. As Jeremiah Murphy wrote, "Singling out schools of education as the chief cause of our educational woes oversimplifies the problem. Trying to revitalize our schools just by fixing schools of education is like trying to repair a wornout pair of shoes by replacing the laces."44
Thus the question is not whether current efforts to improve schools of education will resolve our educational ills -- they won't. Rather, the question is whether the mechanisms that policy makers are using to improve schools of education -- more certification tests, higher cut scores, and severe penalties for institutions that fail to meet specific pass rates -- will deliver the increased accountability and better teachers that policy makers have promised.
My experience in Massachusetts indicates that this test-and-punish approach will not deliver on either promise. It will not deliver accountability, because many programs will undoubtedly react the same way that state colleges in Massachusetts have responded to the state's threat to decertify programs that do not achieve specific pass rates on certification exams. All of Massachusetts' state colleges now require their students to take and pass all certification tests (i.e., the literacy and the subject tests) before entering student teaching. Consequently, the state colleges will soon be able to achieve 100% pass rates for all program completers on all required examinations.
For those of us who teach in the state colleges, this is a great solution. While we get a 100% pass rate, our (unclaimed) students pay the price for not passing tests on material that we may not have covered in our classes. We have effectively shifted the burden of program accountability from our own backs to the backs of the students.
Does it really matter, though, whether this approach allows institutions to avoid accountability, as long as it bears the purported fruits of accountability (i.e., better teachers)? Probably not to pragmatically minded policy makers. However, much research on the impact of certification testing indicates that this approach will not deliver better teachers.45 While scores on paper-and-pencil tests are not irrelevant to teaching quality, they do not correlate well enough with teacher quality to serve as the chief -- much less the only -- tool for improving teacher quality.
Moreover, other research indicates that some certification tests fail to capture what some writers have identified as one of the few (marginally effective) predictors of classroom effectiveness -- verbal ability.46 Brian Cobb and his colleagues correlated preservice teachers' scores on Colorado's basic skills tests of reading and writing (since discontinued) with their scores on the ACT, the SAT, and the Graduate Records Examination. These researchers write that such analyses should produce "correlations in the .75 to .80 range for solid evidence of concurrent validity."47 They found, however, that the basic skills reading tests did not achieve correlations of .50 with any of the other tests, "suggesting little concurrent validity with those tests." Similarly, they found that Colorado's writing test correlated poorly with every one of the other tests. Such results strongly suggest that the Colorado tests, which were produced by NES, the same company that produced the MECT, were not accurately measuring literacy.
Such research will not deter policy makers from claiming, however, that they have improved teacher preparation when schools of education begin to report higher pass rates on state certification tests. Indeed, this is already happening in Massachusetts. When the state department announced the results of the April 2000 administration of the MECT, Commissioner Driscoll stated:
I continue to be encouraged by the steady improvement of scores, and I am delighted by the increasing numbers of test-takers. This suggests to me that a greater number of qualified candidates are seeking to enter teaching careers. Also notable is the improvement in the pass rate for writing, which I believe reflects the improved preparation of candidates that is taking place in our institutions of higher education.48
Still, it is hard to believe that these higher pass rates are the result of the entry of more qualified candidates into the profession or of improved institutional preparation. Why? Because the candidates achieving these pass rates were already juniors and seniors when the testing flap first began in April 1998. Although most teacher preparation programs now offer test-preparation workshops, it is highly unlikely that any institutions imposed any new course requirements on students who were so far along in their programs. Consequently, the higher pass rates that Driscoll noted are measures not of how much programs have "improved" but of how much the state depressed scores -- on whatever it is that the tests measure -- when it first administered the MECT exams three years ago.
Nevertheless, the day is coming when state officials will proclaim, more formally and more loudly, that they have used certification tests and institutional pass rates to reform and improve schools of education. When they do so, the national media will return to Massachusetts to report the state's tale about the power of tests and accountability to improve teacher quality. What's worrisome, though, is that many teacher educators will be tempted to play along with this tale too. After all, we have spent the last three years cast in the role of Idle Fritz, a disobedient and stupid child who appeared in a book of stories that my mother read when she was a girl. The story begins thus: "Fritz was an idle boy, indeed; he would not learn to write or read."49 His parents, "finding they could bear no more . . . whipped and drove him from their door" -- out into the cold, where a wolf ate him.
Of course, this is a cautionary tale, a story about bad things that deservedly happen to bad people. State officials are undoubtedly eager to have some institutions disappear, just as Idle Fritz did, so that they may serve as a caution to others. However, the state will also be looking to cast some institutional actors in the newly developed and far more enviable role of Reformed Fritz, a lad who, thanks almost entirely to his wise elders' high expectations, stern reprimands, and severe sanctions, has turned his life around.
This is a role that very few teacher educators will disavow. We want to claim that we put our shoulders to the wheel, our noses to the grindstone, and our students to the test. This is a great opportunity, a chance to act out a tale of psychometric redemption, written for the golden age of testing.
We will not mention, of course, that we are now claiming to be cured of ailments that we previously denied we had. Nor will we mention that we have achieved higher pass rates by disclaiming our students and abandoning our missions. And we certainly won't mention that these "improved" pass rates will not result in better instruction in the schools. Instead, we will preserve this new fiction about teacher preparation in the hope that it will restore some of the respect we lost when Massachusetts officials led the nation to believe the older fiction that nearly 60% of the state's prospective teachers had not acquired eighth- to 10th-grade skills.
2. Darrell Pressley, "Dumb Struck: Finneran Slams 'Idiots' Who Failed Teacher Tests," Boston Herald, 28 June 1998, pp. 1, 28.
3. Richard Whitmire, "High-Quality Teachers Can Have Huge Impact," Gannett News Service, 10 August 1998.
4. I have kept a running tally of the pass rates as each new set of results is released and posted on the state department's website. Readers may request a copy free of charge from me (clarke.fowler@salemstate.edu).
5. Jordana Hart, "From Madison, via Silber," Boston Globe, 5 August 1998, pp. A-1, A-20.
6. The Educational Testing Service produced a report that included, among other statistics, the percentage of test-takers who passed different sections of the Praxis exam after failing on their first attempt. (See "The Use of Praxis Pass Rates to Evaluate Teacher Education Programs: An ETS Background Report," Educational Testing Service, May 1998, Princeton, N.J.) The estimates were that, during the 1996-97 program year, 2.0% of all test-takers who passed the reading and 3.0% of those who passed the writing sections of the Praxis I Academic Skills Assessments were repeat test-takers. Using ETS' methodology, I calculated the same statistics for the MECT's 1998-99 program year. This analysis indicated that 6.4% of all test-takers who passed the reading exams and 5.9% who passed the writing exams were repeat test-takers.
7. "Shocking Failure," Augusta Chronicle, 14 July 1998, p. A-4.
8. Jean MacMillan, "Massachusetts Grades Aspiring Teachers on a Curve," Washington Post, 28 June 1998, p. A-10.
9. "Grading the Graders: Is Teacher Testing the Best Way to Improve Academic Achievement?," The NewsHour with Jim Lehrer, Public Broadcasting System, 15 September 1998.
10. John Silber, "Those Who Can't Teach," New York Times, 7 July 1998, p. A-15.
11. "Request for Responses Regarding Massachusetts Certification Tests of Communication/Literacy Skills and Subject Matter Knowledge," Massachusetts Department of Education, 24 February 1997, p. 2.
12. "Massachusetts Board of Education Meeting, December 15, 1997," videotape of the board meeting.
13. "Massachusetts Teacher Tests: Questions and Answers," Massachusetts Department of Education, January 1998, p. 5.
14. Ruth Mitchell and Patte Barth, "How Teacher Licensing Tests Fall Short," Thinking K-16, Spring 1999, p. 7.
15. Ibid., p. 17.
16. "Improving the Assessment of Communication and Literacy Skills for Prospective Teachers: Report on the Massachusetts Educator Test of Communication and Literacy Skills," Advanced Literacy Skills Study Group, July 1999, p. 7. This report is available free of charge from Anne Herrington, Department of English, Box 3-0515, University of Massachusetts, Amherst, MA 01003-0515.
17. Ibid.
18. American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education, Standards for Educational and Psychological Testing (Washington, D.C.: American Psychological Association, 1985).
19. Walt Haney, Clarke Fowler, Anne Wheelock, Damian Bebell, and Nicole Malec, "Less Truth Than Error? An Independent Study of the Massachusetts Teacher Tests," Education Policy Analysis Archives, vol. 7, 1999, available at http://epaa.asu.edu/epaa/v7n4/.
20. "Improving the Assessment of Communication," p. 10.
21. Ibid.
22. "Massachusetts Educator Certification Tests: Technical Report Summary," Massachusetts Department of Education, 27 July 1999, p. 8; available at www.doe.mass.edu/teachertest/072799c/techsumm.html. The entire five-volume technical report may be purchased from the state department for $340.
23. Larry H. Ludlow, "Teacher Test Accountability: From Alabama to Massachusetts," Education Policy Analysis Archives, vol. 9, 2001, available at http://epaa.asu.edu/epaa/v9n6.html.
24. Richardson v. Lamar County Board of Education et al., 729 F. Supp. 806, 820-21 (M.D. Ala. 1989), aff'd 935 F. 2d 1240 (5th Cir.)
25. A. Paul Cellucci, Veto Items: Fiscal Year 2001 General Appropriations Act, Outside Sections (Boston: Commonwealth of Massachusetts), Sec. 414, p. 16.
26. "Board in Brief," Massachusetts Department of Education, 27 March 2001.
27. Pamela Ferdinand, "Nearly Half of Aspiring Teachers Fail Latest Massachusetts Test," Washington Post, 13 August 1998, p. A-6.
28. MacMillan, p. A-10.
29. Ronald Hambleton, "Politicians Fail, Not the Teachers," Education Connection, Winter 1999, p. 19.
30. Kate Zernike, "Union Is Urging a National Test for New Teachers," New York Times, 14 April 2000, p. P-1.
31. Beth Daley, Doreen Vigue, and Kate Zernike, "Survey Says Massachusetts Test Is Best in US," Boston Globe, 22 June 1999, p. B-2.
32. Whitmire, op. cit.
33. Kati Haycock, "Good Teaching Matters . . . A Lot," Thinking K-16, Summer 1998, p. 11.
34. John Kerry, "Board of Ed Errs on Teacher Testing," Boston Herald, 26 June 1998, p. 31.
35. Steve Forbes, "The Alarming News," Forbes Magazine, 21 September 1998, p. 32.
36. "A Test That Makes the Grade," Boston Globe, 7 October 1998, p. A-24.
37. Mitchell and Barth, p. 16.
38. Richard Freeland, "Answering a Wake-up Call at Northeastern," Boston Globe, 27 July 1998, p. A-11.
39. Kate Zernike, "Report Finds a Third of Students at State Schools Need Basics," Boston Globe, 11 November 1998, p. B-5.
40. "Why Our Teachers Should Be Tested," Boston Herald, 7 December 1998, p. 26.
41. I heard Silber make these comments during a special public meeting held at the Massachusetts Department of Education, Malden, that preceded the state board's regular meeting on 15 May 1998.
42. Walter C. Varnum, Psychology in Everyday Life, 2nd ed. (New York: McGraw-Hill, 1942), p. 271.
43. Linda Darling-Hammond, "Teacher Quality and Student Achievement: A Review of State Policy Evidence," Education Policy Analysis Archives, vol. 8, 2000; available at http://epaa.asu.edu/epaa/v8n1/.
44. Jeremiah Murphy, "Debate on Schools Heads Down the Wrong Road," Boston Globe, 25 July 1998, p. A-15.
45. Walter Haney, George Madaus, and Amelia Kreitzer, "Charms Talismanic: Testing Teachers for the Improvement of American Education," Review of Research in Education, vol. 14, 1987, pp. 169-238; and George Madaus and Diana Pullin, "Teacher Certification Tests: What Do They Tell Us?," Phi Delta Kappan, September 1987, pp. 31-38.
46. The Teachers We Need and How to Get More of Them: A Manifesto (Washington, D.C.: Thomas B. Fordham Foundation, 2000).
47. R. Brian Cobb et al., "An Examination of Colorado's Teacher Licensure Testing," Journal of Educational Research, January/February 1999, p. 167.
48. "Record Number of Candidates Take and Pass April Massachusetts Educator Certification Tests," news release, Massachusetts Department of Education, 12 May 2000, p. 1.
49. Heinrich Hoffman, Slovenly Peter; or, Cheerful Stories and Funny Pictures for Good Little Folk (Philadelphia: Porter and Coates, 1873).
R. CLARKE FOWLER is an associate professor
in the School of Education, Salem State College, Salem, Mass.
