Federal and state regulations have led to the widespread use of standardized testing programs. Increasingly, these programs are determining our educational policies. Forty-five states now have some form of accountability program. Eighteen of these states hold schools or teachers directly accountable for student test scores. In a recent issue of the Peabody Journal of Education, George F. Madaus, Professor of Education and Director of the Center for Study of Testing Evaluation and Educational Policy at Boston College, warns of the dangers of what he labels “high-stakes” testing.
The results of high-stakes tests, Madaus reports, determine who gets certified or recertified, who gets award or merit pay, and which schools or districts get funds. Since teachers are held accountable for results, Madaus explains, student tests “directly and powerfully influence how teachers teach and students learn.” As the stakes increase, teachers are put under increasing pressure to teach to the test and, as a result, instruction is driven more and more by the testing process. This measurement-driven instruction, Madaus believes, is detrimental to the education of our students.
Proponents of measurement-driven instruction argue that teaching to the test is beneficial. They believe that if a test measures basic skills, then preparing students to take that test can only improve those skills. Madaus maintains, however, that while teaching to the test may result in higher scores, it does not mean that students’ skills will necessarily improve.
The validity of a test
Madaus defines a test as a set of questions which sample the knowledge, skills or abilities of a particular content area or ‘domain’. What test-writers, and presumably test administrators, are interested in obtaining is an accurate measure of the student’s knowledge of the entire content area. The extent to which a test can accurately reflect a student’s knowledge or skill in a general domain is referred to as its validity.
This is the most critical concept in designing a test, for if a test is valid, the assumption can be made that the better a student does on it, the better he/she knows the entire content area. Madaus warns, however, that when tests become so important that they begin to influence curriculum, the validity of the test is jeopardized since it is at that point that students and teachers focus on learning only the kinds of material that are likely to appear on the test. When curricula are designed around tests, it can no longer be assumed that good performance on the test necessarily means that a student knows the entire content area.
For example, competency tests for high school graduation were developed to help determine if students are prepared to be “competent” members of society. The tests were designed to sample the skills and knowledge deemed necessary to function in our society. However, when students are taught only the specific material needed to pass the test, the test can no longer function as it was intended to. The test, Madaus believes, is invalidated because the test results only indicate that the student has learned those specific skills, and not that he/she has learned the whole domain of knowledge and skills necessary to function successfully in society.
Teaching to the test
The more that decisions are based on test results, or even thought to be based on test results, the more likely these tests are to distort the process they were intended to measure. The more important the test is thought to be, the more students and teachers will prepare for the specifics of the test. As a consequence, the test’s ability to accurately measure an entire domain is undermined.
Since teaching to a test tends to narrow the range of instruction, this, in turn, narrows the range of skills that are learned. As past tests are used to redefine curricula, those areas of a subject which have not appeared on past tests tend to remain untaught. Also, entire subject areas not included in standardized testing programs are gradually regarded as less important with the result that, over time, less attention and fewer funds are devoted to them.
The types of questions children are asked in the classroom are also heavily influenced by past tests. Multiple choice, for example, is the most common form of question on stadardized achievement tests. Since, in teaching to this test format, children are trained to answer this specific type of question, the kind of thinking they learn to do is limited. In teaching to the test, students are also trained to arrive at the correct answer as quickly as possible because this is what they must do on the test. Although teaching in this way can improve test performance, it does not mean that students have learned more or that their education has improved. Test performance improves because measurement-driven instruction ensures that students learn the narrow range of skills and facts most likely to appear on the test.
Test results have come to be regarded as the major goal of schooling rather than as one of many indicators of achievement. We have mistakenly, says Madaus, overemphasized that which is measurable. At the same time, we have ignored knowledge, skills or abilities which are not as easily measured, even when such knowledge, skills and abilities are vital for success outside of school.
School-based management an antidote
Most educators are saddled with high-stakes testing programs, which, directly and indirectly, pressure them to teach to the test in order to ensure good results for their students, themselves and their school. Madaus recommends that educators resist this trend by lobbying their professional organizations and state and district policy makers to reduce the emphasis on tests. Teacher organizations, Madaus believes, should work to convince parents, the public and the educational administration that measurement-driven instruction is detrimental to our students and, in the long run, to society.
Madaus believes that the current movement toward school-based management could facilitate the reduction of educational bureaucracy and lead to a reemphasis on curriculum designed by educators rather than driven by testing programs.
Multiple measures of learning
Madaus does not call for an end to testing or to accountability. He believes schools should be accountable to the public. But he recommends that to avoid the deleterious effects of measurement-driven instruction:
1. Tests for accountability, including traditional, standardized, multiple-choice achievement tests should be given unannounced, on a sampling basis (i.e., a random selection of classes in the grade levels normally tested) in order to assess curricular impact and not individual student or teacher performance.
2. More than one indicator of student achievement should be used as the basis for important decisions. Standardized test scores should be interpreted in light of other direct indicators of achievement (student portfolios, writing samples, oral reading, teacher evaluations).
3. Teachers need to be closely involved in the development of these direct indicators of achievement.
Madaus reminds us that tests are a tool of instruction and not a measure of it. Instruction should be shaped by the judgment of educators and not by standardized test scores alone.
“The Distortion of Teaching and Testing: High-Stakes Testing and Instruction” Peabody Journal of Education Volume 65, p. 29-46
Published in ERN January/February 1991 Volume 4 Number 1