At a time when information spreads like wildfire through the media and across the Internet, researchers around the country are arguing that the standards for reporting and interpreting educational research should be raised. In particular, the need for a higher standard is urgent in fields around which there is intense public interest, because “findings can quickly become distorted or misinterpreted and then enshrined through misinformed policy decisions,” write Barbara M. Taylor, University of Minnesota; Richard C. Anderson, University of Illinois; Kathryn H. Au, University of Hawaii and Taffy E. Raphael, Oakland University.
A case in point is the debate about beginning reading instruction. Policymakers and the public have become centrally involved in the debate over instructional methods and cite research to support their points of view. Taylor et al. believe that one of the major reasons why controversy exists about reading instruction is that reading is a very complex skill. Taylor et al. evaluate a study of beginning reading instruction in urban schools: Foorman, Francis et al. (1998), which has been reported widely in the press. They analyze the study and the responses to it by the press, the professional community and the general public to see what can be learned about potential uses and misuses of research when translated into policy and educational practice. Foorman’s findings have been described as “proof that phonics is the best way to teach children to read.” Taylor et al. assert that this misinterprets the study’s findings.
Taylor and colleagues believe that while research can and should inform decisions about policy and practice, literacy research cited to justify policy should reflect a broad understanding of literacy, meet high standards of quality, and have the potential to improve student achievement. Their analysis suggests that the Foorman study, although it has been widely used for policy decisions, does not meet these standards. Taylor et al. agree that while phonics instruction is important, they say, it is only one of many factors that need to be addressed to improve reading achievement, and it can not prevent reading failure all by itself as the Foorman study suggests. In their opinion, a literacy curriculum should also include strategies and skills for constructing meaning from a variety of text types and instruction in written expression that takes advantage of the oral language skills a child brings to school.
Critique of the Foorman Study
Foorman et al. studied first- and second-graders’ reading development in three different reading programs: “direct instruction in letter-sound correspondences with decodable text” (direct code); “less direct instruction in spelling-sound patterns embedded in connected text” (embedded code); and “implicit instruction in letter-sound correspondences during the reading of connected text” (implicit code). Taylor et al. say these descriptions are too vague for understanding exactly what occurred in these classrooms. For example, how many minutes were spent on direct phonics instruction versus reading decodable texts in the direct-code classes? Taylor and colleagues also say that the researchers’ biases about reading instruction are reflected in their choice of tests used to measure results and the way in which they were interpreted. For example, the analyses highlight only measures that involve letters, sounds, and words in isolation. Although the test battery purported to measure reading comprehension as well as decoding skills, the comprehension tests used did not measure comprehension of extended passages of text. None of the measures involved asking students to read connected text from books likely to be found in primary-grade classrooms, despite the fact that several reliable tests exist that do this. One measure of comprehension, the Formal Reading Inventory, was judged to be too difficult by Foorman and colleagues and was not administered to 21 percent of the children in the study. The word-list reading was so difficult that the first graders averaged only 4 to 25 percent correct. Taylor et al. conclude that important aspects of reading were not assessed at all, or not assessed well.
Direct-code group does better
Approximately 45 percent of students not receiving direct-code instruction showed no demonstrable growth in word reading compared to only 16 percent of the students in the direct-code group. Foorman et al. concluded from this that direct phonics instruction is the best method for teaching beginning reading. This conclusion is based on students’ pronunciation of isolated words on a test on which students across all instructional groups scored poorly. And this measure was biased in the favor of the direct-instruction group, because they were the only group that spent considerable time working with words in isolation. It is the opinion of Taylor et al. that if direct instruction of letter-sound correspondences by itself is a superior method for teaching children to read, then direct instruction must be able to show that it improves student reading and comprehension of connected text, which Foorman’s study did not do.
Taylor and colleagues criticize the Foorman study’s general approach to beginning reading instruction because it attributes the reading difficulties of low-income urban children to a “lack of home preparation in understanding the alphabetic principle.” Schools that adopt this model assume they have to help students overcome the deficits of their home experience. A more appropriate model, these researchers believe, is one of cultural difference rather than cultural deprivation. Instead of blaming families for inadequate preparation, they believe that children come to school with a variety of valuable oral literacy skills that schools need to recognize and to build on in beginning reading instruction.
Judging research conclusions
Decades of research have documented that systematic word-recognition instruction is necessary to teach many children to read. However, direct phonics instruction was only one part of the instructional program in the direct-code classes in the Foorman study. Taylor et al. say that one could reasonably conclude that Foorman’s “results show a small advantage for at-risk children on restricted measures for a reading instructional program, that along with an emphasis on explicit instruction in the alphabetic principle, also included reading practice, literature, writing, spelling and one-on-one or small-group Title 1 instruction.”
Foorman’s conclusions emphasize certain findings while barely mentioning others. Taylor et al. report that Foorman failed to find significant differences between the groups on more than half the tests, including all measures involving reading beyond the letter and word levels. The IQ results were not reported at all and no significant differences were found on the Peabody Picture Vocabulary Test, Woodcock-Johnson passage comprehension, Formal Reading Inventory and Kaufman spelling dictation test. Foorman et al. failed to acknowledge the narrow and highly specific nature of the findings in their study. Results on any but the phonics tests are downplayed.
Children in the implicit-code group were found to have significantly more positive attitudes toward reading but this group also had more academic and behavior problems that may have inhibited learning in these classes. Grade-level differences among students were also downplayed. At the second-grade level, there were negligible differences between groups in word reading. This is important because the significant differences occurred only at the first-grade level, and there were significantly more first-graders in the direct-code classrooms. In addition, there were fewer low-income students in the direct-code schools.
At the end of the year, the direct-code group performed significantly better on two measures – letter-word identification and pseudo-word decoding. But when these results are adjusted for differences in students’ scores at the beginning of the study, the advantage of the direct-code group shrinks and in many cases is no longer significant. Taylor et al. conclude that even when one considers only word-level measures, the results provide little credible evidence for the superiority of the direct-code method.
Taylor et al. chose the Foorman, Francis et al (1998) study as an example of research that has been overpromoted by the media and misused by some policymakers and educational leaders in the search for a relatively simple answer to the complex problem of improving children’s reading achievement. While Taylor et al. agree that an extensive body of research exists to support the importance of teachers’ helping students learn phonics, extensive research also supports the importance of many other factors that need to be addressed to improve reading achievement in disadvantaged children. Literacy research documents several practices important for struggling beginning readers, including systematic instruction in word recognition, carefully selected texts, repeated reading, guided writing, regular assessment of student progress, extra time in reading, one-on-one tutoring, strong connections between home and school, and ongoing staff development.
Despite serious methodological weaknesses, this study has been widely cited by the press and is being used across the United States to support the trend toward mandated instruction in phonemic awareness and phonics for beginning readers of all levels and abilities. Taylor et al. believe that “claims made on the basis of this flawed study feed the false hopes of many Americans that we can find a single, simple solution, such as directly teaching phonics, to the real and complex problem of improving reading of young children in high-poverty schools.” Educational researchers should assume responsibility for alerting policymakers to the breadth of relevant reading research and the importance of using forms of evaluation suitable for assessing complex processes such as reading. Finally, researchers must explain the limitations of their study and how their work on reading instruction fits into the larger picture of literacy.
“Discretion in the Translation of Research to Policy: A Case from Beginning Reading” Educational Researcher Volume 29, Number 6, September 2000 Pp. 16-26.
Published in ERN October 2000 Volume 13 Number 7