A new study in the Journal of Educational Psychology tests several strategies to reduce the number of false positives and false negatives in universal screening and improve their efficiency.
One measure serves as gate
One strategy is a two-stage screening process. Under this proposal, a single brief measure serves as a gate to the longer screening battery. Only children who score within the risk range on the initial measure complete the screening battery.
“The majority of screening studies used a one-stage approach and reported classification accuracy well beyond the acceptable range, with false positives ranging from 20% to 60% and false negatives running from 10% to 50%,” the researchers write.
The measure of phonemic decoding efficiency eliminated the greatest number of true negatives (43.4% of the sample) from screening, the researchers write, significantly outperforming measures of sight word efficiency, word identification, and word attack .
Adding one measure to battery
To further refine the screening process, the study of 355 children assessed the impact of adding one additional measure to the battery of tests to refine results. The researchers evaluated the impact of adding the following measures to the base screening model to improve accuracy: Short-term Word Identification Fluency (WIF) progress monitoring (intercept and slope), dynamic assessment, running records, and oral reading fluency.
“Results indicated that the addition of WIF progress monitoring and dynamic assessment, but not running records or oral reading fluency, significantly decreased false positives,” the authors write.
Below are short descriptions of the candidates for the additional measure:
Progress monitoring–WIF_N level, WIF_N slope, WIF_B level, WIF_B slope. Children were administered forms of WIF_N and WIF_B each week for 5 weeks (forms were alternated each week). With the WIF_N, children have one minute to read a page of 50 high-frequency words randomly sampled from 100 high-frequency words from preprimer, primer and 1st-grade level lists. The WIF_B had 500 words. WIF_B slopes were a better predictor of RD status in this study.
Dynamic assessment(DA)–Pseudowords were used to teach decoding skills and also to measure mastery of the decoding skills.
Running records (RR)–Originally introduced to identify the needs of struggling 1st-graders enrolled in Reading Recovery, RRs are now used to estimate students’ reading levels and identify children at risk. Students read leveled passages as a test of contextual reading accuracy and strategy use.
Oral reading fluency (ORF)–ORF was measured with 1st-grade passages and children were scored for the number of words they read correctly.
“A clear message emerged from our attempts to identify additional measures for limiting false positives within the context of a second stage of screening,” the researchers write. “Measures designed to directly assess (progress monitoring) or forecast (DA) children’s response to classroom instruction added significantly to prediction accuracy by reducing false positives. By contrast, measures designed to assess children’s ability to read passages, whether focused on accuracy (RRs) or fluency (ORF), assed little to the prediction models by way of limiting false positives.”
The researchers warn that the failure of RRs to improve classification accuracy should be interpreted cautiously.
The base screening model predicted future reading disability risk with precision, but was too long for a universal screen with all 1st-grade children. So researchers decided to test the 2-stage model.
As part of this battery, the following tests were administered to children in the fall of 1st grade; the goal was to determine risk for reading disability.
Rapid digit naming–The Comprehensive Test of Phonological Processing: Rapid Digit Naming (Wagner, Toregsen, & Rashotte, 1999) measures the speed at which an individual can name 36 digits.
Phonemic awareness–The Compre-hensive Test of Phonological Processing: Sound Matching (Wagner et al., 1999) assesses matching of first and last sounds of illustrated words
Oral Vocabulary–Woodcock-Johnson Psychoeducational Battery–Revised: Oral Vocabulary (Woodcock, McGrew, & Mather, 2001) assesses the ability to provide synonyms and antonyms in response to spoken words.
To determine reading disability status, the following measures were administered in the fall of 1st grade and also at the end of 2nd grade:
Untimed decoding skill–The Woodcock Reading Mastery Test–Revised/Normative Update: Word Attack (Woodcock, 1998) evaluates children’s ability to pronounce pseudowords presented in list form.
Untimed word identification skill–The Woodcock Reading Mastery Test–Revised/Normative Update: Word Identification (Woodcock, 1998) asks children to read words in a list form ordered by difficulty.
Sight word reading efficiency–The Test of Word Reading Efficiency: Sight Word Efficiency (Torgesen, Wagner, & Rashotte, 1997) assesses the number of real words accurately.
Phonemic decoding efficiency–The Test of Word Reading Efficiency: Phonemic Decoding Efficiency (Torgesen, 1997) measures the number of nonsense words accurately decoded in 45 seconds.
Reading comprehension–Woodcock Reading Mastery Test–Revised/Normative Update: Passage Comprehension (Woodcock, 1998) is a modified cloze procedure. The tester uses a rebus or symbol, asks the child to point to a picture corresponding to the rebus. Next, the child points to a picture representing the words.
Participating in the study were 1st-graders from 56 classrooms in 14 schools in urban and suburban districts in middle Tennessee; 7 schools were Title I institutions.
“Selecting At-Risk First-Grade Readers for Early Intervention: Eliminating False Positives and Exploring the Promise of a Two-Stage Gated Screening Process,” by Donald Compton, Douglas Fuchs, Lynn Fuchs, et al., Journal of Educational Psychology, 2010, Volume 102, Number 2, pps. 327-340.