The No Child Left Behind (NCLB) Act of 2001 is motivated by a widely shared desire to improve the education of our children. It relies on assessment and accountability requirements as the major mechanism for bringing about these improvements. It demands extensive testing (yearly in grades three through eight) and sets ambitious objectives for rapid increases in student achievement.
Its demands for the various subgroups — minority, poor, disabled and second-language students — goes well beyond what most state laws currently require. The law’s requirements pose challenges for schools, districts and states, report Robert L. Linn, University of Colorado/Boulder, and Eva L. Baker, University of California/Los Angeles, codirectors of the National Center for Research on Evaluation, Standards, and Student Testing (CRESST); and Damian W. Betebenner, University of Colorado/Boulder.
This legislation requires that states demonstrate steady gains in student achievement and close the gap in achievement between groups. The objective is to have all students demonstrate proficiency on state exams within 12 years. To achieve this, states will have to make sufficient gains each year to reach full proficiency by the 2013-2014 school year.
Alternative instructional approaches
Schools that fail to meet improvement targets must adopt alternative instructional approaches that research studies have shown to be effective. States may average up to three years of data in demonstrating progress. At least 95 percent of each subgroup of students must participate in state assessments.
These researchers point out that states are not starting on a level playing field. Currently, states’ content standards, the rigor of their tests, and the stringency of their performance standards vary greatly. The levels set for passing or proficient vary greatly from state to state. Since states will be held accountable on the basis of their individual standards and tests, the improvement demands on states will be highly unequal. Students in states that have set high cutoff points for proficiency will appear to be making less progress and to be performing more poorly than students in states that set lower standards. This could result in pressure for states to set lower standards and this would defeat the objective of NCLB.
Currently, the only data comparable across states is from the National Assessment of Educational Progress (NAEP), used by all states in grades four and eight. The standard for proficient performance on the NAEP, however, is set so high that no state has yet reached 50 percent proficiency.
Linn et al. contend that even with the considerable pressure, 12 years is not enough time for all students to reach the proficient level as defined by the NAEP or by many state tests. Although goals must pose a challenge, they need not be set so high that they are unachievable. These researchers believe that having a goal that is unobtainable, no matter how hard teachers try, will do more to demoralize them than to motivate greater effort.
Linn et al. caution that the progress standards set by the NCLB present particularly difficult challenges at the school level. School-level achievement scores are often volatile from year to year because of differences in the groups of students taking the test or changes in teaching staff. Scores for individual students can vary so greatly as to be unreliable. Both measurement and sampling errors contribute to this variability. This volatility poses significant problems for evaluating schools and groups of students on a yearly basis.
Averaging progress across three years
Linn and fellow researchers suggest ways that the volatility of individual school results can be reduced. Aggregating data from several years by using rolling three-year averages to define yearly progress will achieve more dependable data. The stability of scores can be improved by tracking individual students from year to year, or by using composite scores across subject areas and grades.
Since schools with the most ambitious test and performance standards will have much larger yearly progress requirements, these researchers suggest that the U.S. Department of Education offer guidance in defining yearly progress objectives that are challenging but feasible, given sufficient effort and concentration of resources. Interpretations of the law need to minimize the likelihood of unintended negative consequences such as an incentive for states to adopt less-challenging content standards or developing tests aimed more at minimum than at high standards.
Since the NCLB requires that states participate in the NAEP reading and math tests at fourth and eighth grade, ways can be found to use this test as a benchmark for comparing state tests and performance standards. Making NAEP the controlling factor would be fair to states; all would be operating under the same rules.
There is a problem, however, with using the NAEP to determine performance levels and progress objectives. The proficient level on NAEP has been determined to be unreasonably high by the National Academy of Education and the National Research Council. The annual gain in the percentage of students reaching the proficient level in mathematics averages 1.5 percent per year at grade four and 1.2 percent at grade eight. Gains in reading are smaller. At this rate, since schools have less than 50 percent proficiency currently, it is not feasible that they could reach 100 percent proficiency in 12 years.
Use of the NAEP basic level
Linn et al. suggest that an alternative that is more attainable but still challenging is the NAEP basic level. They believe that bringing all grades and subjects close to 100 percent by 2014 would be a major educational accomplishment. And if the percentages of students within each state who achieved at the basic level or higher on NAEP were used as a benchmark against which state standards of performance could be compared, it would ensure that state standards were less disparate than they are now.
Index scores offer another possibility for monitoring progress in achievement. For example, students who score at the proficient level could be given a score of 1.0, those who score in the high end of the basic range a score of 0.8, those in the mid part of the range 0.6, and those in the low end of the basic range 0.4. Students below basic would receive scores of 0, and those scoring at the advanced level might receive 1.2. The objective to be reached by 2014 could be an average index score of 1.0. Yearly expected progress could then be set equal to an effect size of .05, for example. The use of effect-size statistics would preclude the need to set performance standards and therefore would avoid the problems of judging the comparability of performance standards in different states.
Linn et al. believe that the challenge is to implement NCLB in a way that will provide the information needed to assess and improve educational quality. The information must be understandable to teachers, administrators, policy makers, students and parents. Interpretations of the law should recognize the variability in state standards and the volatility in school-level results from year to year. States must have ways to report progress that minimize volatility. Possibilities worth consideration include the use of index scores, composites across grades and rolling averages. Use of the NAEP as a benchmark could reconcile differences in state standards and level the playing field.
“Accountability Systems: Implications of Requirements of the No Child Left Behind Act of 2001”, Educational Researcher, Volume 31, Number 6, September 2002, pp. 3-16.
Published in ERN November 2002 Volume 15 Number 8