Reliability of informal reading inventories

iStock_000014316766XSmallHow reliable are Informal Reading Inventories?

Informal Reading Inventories (IRIs) assess a student’s instructional reading level using sets of passages that are selected to be representative of texts at different grade levels. Most reading inventories provide a choice of three to six passages at each grade level.

Estimates of a student’s reading level may be based on a combination of oral reading accuracy (percentage of words read correctly) and comprehension, or on comprehension alone. IRI results are commonly used to identify students’ present level of reading for Individualized Educational Plans, to evaluate year-to-year growth in reading, or to target students for Title 1 services. IRIs also can be used for diagnostic purposes such as observation of a student’s approach to decoding unfamiliar words.

Reliability of inventories

Previous research has been critical of these inventories for their lack of attention to reliability. When a test does not have adequate reliability, it is inadvisable to use it, even if the tasks and materials appear to be well aligned with classroom instruction.

In this study at the University of Maine, recent editions of IRIs were studied to determine whether the concerns about reliability had been addressed. Nine recently revised IRI manuals were examined for evidence of reliability. Only four IRI manuals provided any reliability data. The IRI that provided the most reliability data revealed relatively large standard errors of measure, indicating low reliability.

This investigation found that many of the earlier criticisms of IRIs are still true. IRI authors ignore widely accepted professional standards of test quality. Even among IRIs that have investigated reliability, much of the evidence is poorly documented or undermined by weak research methodology.

Because many of the studies described in the manual are small scale, the generalizability of their results is questionable. Some tests have established reliability across grades rather than within grades, so the results are likely to be overestimates. Most IRI authors have investigated a single type of reliability and only one has evidence of interrater reliability. This is a major shortcoming that needs to be addressed.

Examine reliability data of inventories

In conclusion, IRIs with no evidence of reliability should not be used to estimate a student’s reading level, regardless of how the results will be used. Results have the potential for harm if the information they provide is imprecise or misleading. Five IRI manuals–the Analytical Reading Inventory, Classroom Reading Inventory, Reading Inventory for the Classroom, Burns Roe-IRI and Basic Reading Inventory–provide no reliability data.

IRIs that provide some reliability data should be evaluated in light of their intended uses. The amount of error that is tolerable depends on the way the results will be used. Reliability of .90 is desirable, even when the results will be used in combination with other measures. A single test should never be used as the only source of evidence in educational decision-making.

With the exception of the Qualitative Reading Inventory-3, none of the IRIs in this study reached this level of reliability. Even so, the authors of QRI-3 recommend that decisions be based on at least two passages at the same grade level because of the poor internal-consistency reliability of their comprehension scores.

These researchers suggest that a reliability of .70 may be used for tentative in-class curriculum decisions that can be readily reversed if the child exhibits a different level of skill during instruction. This study concludes that IRI authors need to give much greater attention to reliability standards.

While these inventories are intuitively appealing, almost all of them need more reliability data than currently reported. Without reliability, test utility is severely limited. Teachers and other school personnel should be informed of the limitations of most IRIs and to guard against inappropriate uses by selecting measures with adequate reliability.
“How Reliable Are Informal Reading Inventories?,” Psychology in the Schools, Volume 42, Number 6, 2005, pp. 593-604.

Published in ERN September 2005 Volume 18 Number 6

 

Leave a Reply

  • (will not be published)