Understanding the TIMSS ‘final year’ study

Results of the Third International Mathematics and Science Study (TIMSS) of fourth- and eighth-graders reveal that United States students score about average compared to students in other industrialized nations around the world. At grade 4, U.S. students were above average in both math and science. At grade 8 they were slightly above average in science and slightly below in math.

However, the “Final Year” report paints a very different picture of how U.S. students compare by the time they graduate from high school in their “final year”. At the end of secondary school, our students scored far below students from 20 other countries in math and science literacy and below students from 15 other countries in physics and advanced mathematics.

Assessments compare systems, not students

TIMSS may be rich in important information, Gerald Bracey, an independent educational researcher, disputes much that has been written about these results. Quoting TIMSS research coordinator, William Schmidt, he points out that the Final Year assessments compare systems, not students.

Students who take the test do not represent comparable samples of students across countries. Bracey refutes published interpretations of the data, including reports from the U.S. Department of Education’s National Center for Educational Statistics, which, according to him, include misleading information and factual errors. Bracey points out that the data do not show that the teaching of math and science courses in the U.S. is inferior to the same teaching in other countries. Results demonstrate, rather, that students who take more advanced math and science courses during high school score higher.

American students, on average, are younger than students taking the test in other countries. While American students had just turned 18, other countries
tested students who were mostly 19 or even 20 years old and who had completed up to 14 years of schooling. The reason for this discrepancy is that the test is administered in the “final year of secondary school,” which means different things in different countries.

While U.S. students were all tested in 12th grade, in other countries it varied from the 10th to the 14th grade, depending on a student’s course of study. The percentage of students enrolled in school for these various “final years,” and therefore eligible to take the test, was dramatically different from country to country. In many countries, only those in highly academic or technical courses of study attend through the end of secondary school.

In Canada, for example, 74% of the original class was enrolled in school when the TIMSS Final Year tests were given to Canadian students in their 12th, 13th
or 14th year of school. Another factor, the diversity of curricula across countries, makes it even harder to compare results. Only in the United States did students enrolled in pre-calculus take the advanced math test. Twenty-three percent of the items on the test presumed students had taken calculus. American
students in pre-calculus scored 100 points lower than American students who had taken calculus. More than half the American students taking the test had not had calculus, which significantly lowered the U.S. mean score. American students who had taken calculus scored at the international average.

The physics test reflects similar comparability problems. The two nations scoring the highest, Norway and Sweden, tested students who were in the final year of a three-year physics course. In several other countries, students tested were enrolled in technical programs with curricula heavy with math and science
courses. American students taking this test almost invariably were in their first year of physics.

Heavy outside workload affects American students performance

Bracey also notes that the heavy outside workload of American students appears to affect their performance. Students in other countries rarely work while going to school. In the U.S. the majority of students work. The seven percent of American students working up to 15 hours a week scored above the international average on the math and science literacy test. However, approximately 55 percent of American students work more than 21 hours a week.
For the 28 percent working 21 to 35 hours a week, the mean score was 474 (compared to the international mean of 500); and for the 27 percent working more
than 35 hours a week, the mean was 448.

While Bracey’s comments are critical of the interpretations of the TIMSS “Final Year” Study, he emphasizes that as a comparison of educational systems,
the report has much to say to U.S. educators. A higher percentage of students in other countries take more advanced math and science courses during high school than American students do. U.S. educators and policy makers should consider offering more of our students these classes if they wish to create a more level playing field for international comparisons.

“The TIMSS ‘Final Year’ Study and Report: A Critique” Educational Researcher Volume 29, Number 4, May 2000 Pp. 4-10.

Published in ERN September 2000 Volume 13 Number 6.

Leave a Reply

  • (will not be published)