Readers' reactions to the article ("What educators can learn from Wal-Mart about data-driven decision-making" ERN January 2008) was quick and emotional considering the dry topic, better data in education. The article described the work of James Guthrie in a recent issue of the Peabody Journal of Education where he made the case that to get more specific information about education spending and student outcomes, educators need to work with school-by-school budgets rather than district-wide budgets. To illustrate his point, Guthrie notes that Wal-Mart managers routinely know more about how teddy bears from China are selling in Wal-Mart stores than school administrators know about the progress of their enrolled students. "Wal-Mart knows more, and within minutes, what items and services it is selling, and not selling, in its stores and can compare stores across a region and over time. Most U.S. schools cannot easily tell which of its students have attended which classes in an individual day," Guthrie wrote.
The comparison of students to teddy bears (and perhaps educators to Wal-Mart managers) drew a sharp response. One reader wrote:
There is so little that can be measured accurately on our students that it is irresponsible to try to compare education with retail. Teddy bears do not have emotions and change daily, they don't have supportive or dysfunctional families, they don't get hungry, they don't get stressed, they don't speak a second language, they don't have special needs, they don't go through puberty, they don't date, they don't get bullied or harassed, they don't struggle with negative news of the day, they don't worry about their future, they don't worry about tomorrow.
Schools do much more than academics. Daily, our teachers promote responsible behavior, citizenship, health, productivity, civility, coping skills, tolerance, self confidence, the arts, life skills, technical skills among other things not measured mathematically by any assessment. Even the academic growth mentioned in your article is hard to measure, are we using norm referenced standardized test, criterion referenced tests, diagnostic test, formative assessments or summative assessments. Schools can spend a lot of time testing which does not increase learning.
Point well taken. Data about student performance fail to take into account all the emotional baggage of families and stresses of growing up. These are all the emotional uncertainties that impact the learning process. Our reader provided an important reminder of all that is emotionally unmeasurable in student performance.
Guthrie talks about many of the technical uncertainties educators face (class size, school size, etc) in trying to raise achievement.
But do uncertainties, be they emotional or technical, mean we should give up measuring learning altogether?
Hurricanes, Guthrie writes, are a good example of what is both measurable and unmeasurable and predictable and unknown.
On the one hand, he says "hurricanes are explainable meteorological phenomena comprising known consequential interactions between oceanic temperatures, wind dynamics, and climatic conditions. The presence, force, and trajectory of these know elements, within explainable parameters, are a matter of scientific probability."
While we can know a great deal about hurricanes, we will never know, for instance, when they will strike. There are similarly many unknowns in education. For instance, he writes, "much that is intuitively important regarding the education production function is unknown, has little or no empirically verified underpinning, is not amenable to mathematical modeling, and is unpredictable."
Should educators stop trying to measure learning? As difficult as the process may be, this may be tantamount to asking, should scientists stop trying to measure hurricanes?
What educators can learn from Wal-Mart about data-driven decision-making
Go to research briefs.
