One of the first tests students encounter in their school careers is the widely used measure of early literacy skills, DIBELS (Dynamic Indicators of Basic Early Literacy Skills). But how predictive is this test of students’ abilities in reading comprehension?
“One of the more common criticisms of DIBELS is that it is not an adequate indicator of reading comprehension,” writes researcher Brant Riedel, who conducted a study of 1,518 1st-grade students to see how well their DIBELS scores predicted reading comprehension in 1st grade and 2nd grade.
ORF is the exception
With the exception of Oral Reading Fluency (ORF), the other DIBELS subtests are not very predictive of reading comprehension at all, concludes his recent study in Reading Research Quarterly. While ORF correctly classified 80% of the students as scoring as poor or satisfactory in reading comprehension in 1st grade (71% in 2nd grade), other DIBELS subtests, Phoneme Segmentation Fluency (PSF), Nonsense Word Fluency (NWF), Letter Naming Fluency (LNF) and Retell Fluency (RF) were poor predictors of comprehension, with PSF being the weakest predictor, reports the researcher.
He recommends that schools use an abbreviated DIBELS protocol that excludes PSF and NWF. The subtests, LNF and NWF, which can be administered early in 1st grade were better predictors of reading comprehension. In the middle of 1st grade, when ORF can be administered, testing can be limited to ORF which has the best predictive power, he says.
“This revised protocol would minimize the amount of instructional time lost and still preserve predictive power,” he says. He adds that his results also do not support intervention instruction in phoneme segmentation or decoding for those who score poorly on PSF or NWF. In the study, PSF and NWF scores at the end of 1st grade misjudged comprehension status for 47% and 32% of students, respectively.
Study of 1,518 Memphis 1st graders
Students were 1,518 1st-graders from the Memphis City Schools district during the 2003-2004 school year. They attended 26 schools with a Reading Excellence Act (REA) grant. The students were predominantly African American (92%). The poverty rate was high, with 85% of the students qualifying for free or reduced lunch. There were only a few English Language Learners (ELLs) and no students receiving special education included in the sample.
DIBELS tests were administered at the beginning, middle and end of 1st grade. The researcher analyzed students’ DIBELS scores on the 5 subtests (ORF, PSF, NWF, RF, LNF) and their scores on tests measuring comprehension, the Group Reading Assessment and Diagnostic Evaluation (GRA + DE), in 1st grade and on TerraNova in 2nd grade.
The researcher examined each DIBELS measure in each period to see if it predicted whether there was a poor or satisfactory outcome in comprehension. The cut score was a Normal Curve Equivalent of 40; satisfactory reading comprehension was >40 NCE and poor comprehension was >40 NCE. A cut score of 40 was chosen rather than 50 or 40th percentile, the researcher explains, because it provides a stricter definition of poor comprehension.
“If the emphasis is on identifying those most at risk for comprehension difficulties, then the lower cut point (40 NCE) is preferable to higher cut points (e.g., 50 NCE),” the researcher says.
Understanding the relationship between DIBELS subtests and reading comprehension is important, he says, for wisely allocating resources. Reading comprehension is, after all, the goal of developing reading skills, he says. Students with good comprehension skills but low DIBELS scores may be getting unnecessary intervention services, while students with high DIBELS scores but poor comprehension might not be receiving needed services, he notes.
While a great deal of research has examined the predictive value of oral reading fluency tests, the researcher says he wanted to look at the predictive value of ORF specifically and of the other DIBELS subtests.
Retell fluency weaker predictor than ORF
Retell fluency (RF) is intended to be an indicator of comprehension, but RF was a weaker predictor of comprehension than ORF, he says.. Even considering RF results in combination with ORF results did not substantially improve predictive accuracy over using RF alone, he adds.
Although there is no empirical evidence to support it, Riedel believes it is still useful to have students perform the RF task because it is an indicator that they need to read for understanding rather than speed and thus could strengthen the relation between ORF scores and comprehension.
About 15% of students with satisfactory ORF scores had poor comprehension. A striking characteristic of those students, says Riedel, is that they had poor vocabulary skills. Students with poor comprehension scored over 20 NCE points lower on the vocabulary subtest. Being aware of students’ vocabulary abilities may help interpret ORF scores. For these students, vocabulary needs to be the focus of intervention, he writes.
There have been concerns about using DIBELS ORF with ELLs who may be able to decode words rapidly, but do not comprehend text because of vocabulary problems. But in this study, the ORF scores were strongly correlated to comprehension, although the researcher notes that the number of ELLs in this study was small.
While this study supports the use of DIBELS ORF as a screening measure in the middle of 1st grade and as an outcome measure at the end of 1st grade, its value as a diagnostic assessment is less clear, writes the researcher.
“Although DIBELS ORF usually correctly predicts current and future comprehension difficulties, it may not provide any details regarding the student’s reading difficulties or the interventions needed to remedy them.”
“The Relation Between DIBELS, Reading Comprehension, and Vocabulary in Urban Frst-grade Students” by Brant Riedel, Reading Research Quarterly, Volume 42, Number 4, October/November/December 2007, pp. 546-567.