Art based on a photograph © 2002 by David Grossman

Babes in the Woods: The Wanderings of the National Reading Panel By Joanne Yatvin From the choice of participants to serve on the National Reading Panel to the hasty release of an uncorrected, undeliberated, and unapproved subcommittee report, the procedure used by the NRP was flawed, Ms. Yatvin -- a member of the panel -- charges. Now government agencies at all levels are using the "science" of the NRP report to support their calls for changes in school instruction and teacher education.

WHEN THEY heard that I had been appointed to the National Reading Panel (NRP), my friends predicted, "They'll eat you alive." But it was never like that. When we panelists began our journey to discover what "research says about the best methods for teaching children to read," we were all searchers after truth, each knowledgeable and respected in his or her professional domain and each dedicated to working together toward our joint goal. Along the trail, pressured by isolation, time limits, lack of support, and the political aims of others, we lost our way -- and our integrity.1

To begin with, Congress, which had commissioned our journey, was naive to believe that a panel of 15 people, all employed full time elsewhere and working without a support staff, could in six months' time sift through a mountain of research studies and draw from them conclusions about the best ways of teaching reading. And the National Institute of Child Health and Human Development (NICHD), designated as our guide and provisioner on the journey, was irresponsible both in advising Congress that the task could be done in that way and in selecting the wrong combination of people to do it.


In late 1997 Congress passed legislation authorizing the "Director of the National Institute of Child Health and Human Development (NICHD), in consultation with the Secretary of Education," to select the members of the panel from more than 300 nominations by individuals and organizations involved in reading education. The bill specified that the panel was to be made up of "15 individuals, who are not officers or employees of the Federal Government and include leading scientists in reading research, representatives of colleges of education, reading teachers, education administrators, and parents."

NICHD stretched that definition to its limits by appointing 12 university professors. Eight of them were reading researchers, two were administrators without backgrounds in reading or teacher education, one was a teacher educator, and one was a medical doctor. Other categories were represented by one parent, one elementary school principal, and one middle school language arts teacher.2 There was no reading teacher in the sense I believe Congress intended. When, shortly after the initial panel meeting, one of the university researchers resigned, I suggested that it made sense to replace him with a primary-level teacher of reading. A month later, at our second meeting, the panel chair announced that, "after considerable discussion, we concluded that at this stage in the game we might just as well not replace him."3 The panel was not told who the "we" were. And since the work of the panel had scarcely begun, the explanation offered was scarcely credible. Why wouldn't NICHD officials want someone on the panel who actually taught young children how to read?

The appointment of the medical doctor was also troubling. Although, technically, she was a reading researcher who worked in the controversial area of brain activity in reading, she had no knowledge or experience in reading instruction. What really made her an inappropriate choice, however, was her close professional association with NICHD. In a videotape later produced under the direction of NICHD, this doctor appears five times, hailing the breakthrough accomplishments of the panel, while other members who were far more involved in the panel's research appear once or not at all.

At the first meeting of the panel in April 1998, another troubling fact about NICHD's appointments became apparent. All the scientist members held the same general view of the reading process. With no powerful voices from other philosophical camps on the panel, it was easy for this majority to believe that theirs was the only legitimate view.

Without debate, the panel accepted as the basis for its investigations a model composed of a three-part hierarchy: decoding, fluency, and comprehension. Theoretically, the components of the model are both discrete and sequential. This skills model posits that learners begin to read by separating out the individual sounds of language and matching them to written letters and combinations of letters. Learners then move on to decoding words and stringing them together into sentences. Since most words in grade-appropriate texts are already in learners' spoken vocabularies, understanding emerges from correct pronunciation. For sentences to be understood, rapid, conversational verbalization is required; this is called fluency. The understanding of texts was seen to depend on building a larger vocabulary and using strategies to uncover ideas and the structures that bind them together in written discourse. Despite minor differences of opinion that surfaced in discussion from time to time, this hierarchy-of-skills model was always the official view of the panel.

For scientists to take such a quick and unequivocal stance was disturbing, since there are two other models of reading that currently claim legitimacy, each with numerous adherents. In one of them, a holistic or constructivist view, readers must do many things at once, right from the beginning. They identify words by visual memory, match sounds to letters, pull word meanings from context, understand sentences as complex structures, figure out how the whole system of written language works, obtain information about content, and predict both the words and the content to come. Of course, the texts young learners attempt to read are short and simple in the beginning and grow more challenging as their facility grows.

The other dominant model among conservative thinkers and in the public mind is a simple decoding model. It posits that learners begin in the same way as in the skills model -- by separating oral language into sounds and matching those sounds to written letters. With increasing mastery of this one skill, learners can read anything. Understanding the meaning of what one reads and acquiring new words and ideas are seen as separate from learning to read. These processes are facilitated by the teaching of school subject matter, by life experiences, and by reading more advanced material.

The decision to use only one model for all its investigations was critical in sending the NRP down a particular path in its journey. It excluded any lines of research that were not part of this model, among them how children's knowledge of oral language, literature and its conventions, and the world apart from print affects their ability to learn to read. It also excluded any investigation of the interdependence between reading and writing and of the effects of the types, quality, or amounts of material children read.4 Contrary to interpretations made by many politicians, members of the press, and ordinary citizens, the NRP report does not -- and cannot -- repudiate instructional practices that make use of any of these components because the research studies on them were never examined.

Despite the choice of a single research path, a large number and a wide range of topics were proposed and discussed by panel members at our second meeting in July 1998. Several of those topics were in fact outside the boundaries of the accepted skills model -- such as writing and literature -- but the panel members were then in an optimistic frame of mind, thinking that those topics could be worked into the narrow structure we had decided upon. At that time, we were roaming free.

By October of that year, as the reality of the limits of our time and energy and the vastness of the body of research on reading were beginning to sink in, the panel created a list of 32 relevant topics and voted to investigate 13 of them, including oral language, home influences, print awareness, instructional materials, and assessment instruments. This occasion, incidentally, was the only time that the panel took a formal vote on anything. Our usual manner of making decisions was to talk an issue to death until the chair decided that one position was more solid than others. From my perspective, it appeared that he was more favorably disposed toward the contributions of the scientists than those of other panel members. I began to realize who was leading this expedition.

A second critical decision, urged by NICHD at the first panel meeting and later accepted by the panel and codified in a lengthy and detailed methodology, was that only experimental and quasi-experimental studies would be included in the review of research. NICHD's premise was that a great deal of published research is of poor quality. It exhorted the panel to set higher standards, comparable to those used in medical research. No one discussed the fact that the type of medical research referred to is applied to the treatment of disease or deficiency, not to the processes of normal, healthy development, which is what learning to read is for most children. Moreover, medical research differs in two important ways from educational research: experimental subjects are randomly selected from homogeneous populations, and most treatments are given under a "double-blind" protocol, in which neither the subjects nor the experimenters know who is getting the treatment and who is getting a placebo. Such conditions are impossible to re-create in educational settings.

Two nondecisions by the NRP are also worth mentioning: not to use a compass and not to consult knowledgeable guides. Despite several discussions about formulating our own definition of reading, we never did so. And despite my repeated requests that subcommittee reports be reviewed by outside practitioners as well as by researchers before the panel accepted them, the panel never said yes or no. In the end, the reports were submitted only to other researchers. With regard to definitions, although reading has been defined often and well in the past, it was important for the NRP to make clear its own use of the term. In the various subcommittee reports, "reading" is used to represent many different kinds of operations, from accurate pronunciation of nonsense words to a thorough understanding of a written text. When a subcommittee report asserts that a particular instructional technique "improves children's reading," the public deserves to know whether the authors mean word calling, speed, smoothness, literal comprehension, or the ability to assimilate a subtle and complex set of ideas.

With regard to review by practitioners, it was also important to get reactions from teachers, who are at the heart of the instructional process. One component of the charge from Congress to the NRP was that it determine "the readiness for application in the classroom of the results of this research." How could a group that included only one classroom teacher make such a determination without consulting a number of teachers?

ONCE THE PANEL began digging into research studies in the summer of 1998, the members realized that, even with a limited number of topics and strict selection criteria in place, the tasks of analysis and synthesis were overwhelming. Clearly, more time was needed. Late in the fall, as the original deadline approached, NICHD asked for and received a year's extension from Congress. But even that was far from enough time. Three years might have allowed the panel to investigate thoroughly all the topics it had originally identified.

The huge volume of work to be done brought to light another adverse pressure on the panel. Outside of a research librarian who would do electronic searches on request from panel members, NICHD supplied no support staff. Although the organization was willing to pay assistants employed by panel members to screen, analyze, and code the relevant studies, enough hands were simply not available. The only members who had assistants qualified to do such work were the university researchers. And most of their assistants were graduate students, already deeply immersed in their own research projects and reluctant to take on a new line of inquiry that would not benefit them directly.

As time wound down, the effects of insufficient time and support were all too apparent. In October 1999, with a January 31 deadline looming, investigations of many of the priority topics identified by the panel a year earlier had not even begun. One of those topics was phonics, clearly the one of most interest to educational decision makers and to the public. Although the panel felt that such a study should be done, the alphabetics subcommittee, which had not quite finished its review of phonemic awareness, could not take it on at this late date. And so, contrary to the guidelines specified by NICHD at the outset, an outside researcher who had not shared in the panel's journey was commissioned to do the review.

In the end, only 428 studies were included in the NRP subcommittee reports. Thousands of studies were rejected without analysis because their titles, publishing circumstances, or abstracts revealed that they did not meet the panel's criteria. Since the release of the report, outside reading experts have charged that the panel missed many qualified studies. I cannot say if that charge is true, but it certainly seems possible that the shortage of time and support staff could have led to errors of omission.

At the October 1999 meeting, subcommittee chairs summarized their findings before the whole panel for the first time. Although there was general satisfaction with the content of the reports presented, the panel members were worried. There was no time to give the reports careful and critical scrutiny. In fact, even then, not all the reports were in finished, written form. Moreover, individual members were more interested in finishing their own reports than in scrutinizing the work of others. In that respect, we had reached a point where it was "every man for himself."

Panel members were also dismayed to realize that only eight topics had been covered. Somehow, each subcommittee thought -- perhaps hoped -- that the others were covering more ground. It also became apparent that different subcommittees had used different approaches to their topics. Although the agreed-upon plan had been for all subcommittees to use common procedures for search, selection, analysis, and reporting, this turned out to be impossible for most of the topics. Often there were too few studies, or the studies were too diverse to do the meta-analyses originally intended. Most discussion at that October meeting focused on how to present these facts honestly and clearly to prospective audiences. Ultimately, the panel decided to explain its difficulties in the full report in the belief that the various audiences for the report would understand and respect the panel's decisions.

It was at this meeting that I formed the intention of submitting a "minority report." Shortly thereafter, I informed the panel chair in writing and sent a copy to the director of NICHD. I felt that we had done an incomplete, flawed, and narrowly focused job and that our explanations would not make up for it, even if the public read them, which was unlikely, given the fact that they would be buried in a more than 500-page report. Receiving no response to my letter, I drafted a minority report expressing my dissatisfaction with our work and submitted it to the panel. For the most part, the panel members received my report without comment, although the chair and the executive director tried to persuade me that my points could be incorporated into the body of the full report.5 Right up to the deadline for publication, I was ready to withdraw my report if I could be shown that my concerns were met in some other way.

The NRP's last bad decision was to call its report finished and submit it for publication. Members convinced themselves that, because they had worked hard under adverse conditions, the report was satisfactory. Most of the scientists also seemed to believe that the standards they had set and the methodology they had developed were accomplishments important enough to compensate for the shortcomings in their work. To justify themselves, they added a special section titled "Next Steps" that explained the small number of topics investigated and suggested areas for future investigation. Another special section called "Reflections" was also added to summarize and emphasize the panel's accomplishments. These last-ditch efforts were to no avail. The panel's claim to scientific objectivity and comprehensiveness was lost.

Still, the panel's trials were not over. The situation worsened when the phonics report was not finished by the January 31 deadline. NICHD officials, who wanted it badly, gave that subcommittee more time without informing the other subcommittees of this special dispensation. The phonics report in its completed form was not seen, even by the whole subcommittee, of which I was a member, until February 25, four days before the full report was to go to press. By that time, not even all the small technical errors could be corrected, much less the logical contradictions and imprecise language. Although a few changes were made before time ran out, most of the report was submitted "as is." Thus the phonics report became part of the full report of the NRP uncorrected, undeliberated, and unapproved. For me, that was the last straw, and I informed my fellow panel members that I wanted my minority report to be included.

As I feared, since April 2000, when the report of the National Reading Panel was released, it has been carelessly read and misinterpreted on a grand scale. Many journalists, politicians, and spokespersons for special interests have declared, for example, that 100,000 studies were analyzed by the panel and that we now know all we need to know about teaching reading. Government agencies at all levels are calling for changes in school instruction and teacher education derived from the "science" of the NRP report. NICHD has done its part to misinform the public by disseminating a summary booklet and the aforementioned video, which, in addition to being inaccurate about the actual findings, tout the panel's work in a manner more akin to commercial advertising than to scientific reporting. Neither includes any mention of a minority report.

I said above that the NRP's last bad decision was to publish its findings as if they were complete and definitive. Unfortunately, that has proved to be untrue. Individually, members of the panel have made the decision not to speak out against the misrepresentations and misinterpretations of their work. A few have even jumped on the NICHD bandwagon for reasons I can only imagine. Most have simply remained silent.

Although NICHD will not provide all-expenses-paid trips6 for panel members who might say anything critical -- or even altogether accurate -- about the NRP report, those who wish to speak out are not without access to professional and public audiences. Why not write letters to editors, speak at professional conferences, seek meetings with legislators? Perhaps the silent ones have convinced themselves that the NRP report really is all that NICHD claims it to be or that, whatever its flaws, it is doing more good than ill. Unquestionably, it would be difficult for them to admit that the panel lost its bearings and let guides who had other goals lead it in the wrong direction. Or perhaps they have more selfish reasons. As one researcher on the panel told me in private conversation, "I agree with you on many points, but I depend on NICHD for funding my research."

PDK Home | Site Map
Kappan Professional Journal
Last updated 17 January 2002
URL: http://www.pdkintl.org/kappan/k0201yat.htm
Copyright 2002 Phi Delta Kappa International