By now, many educators have come to appreciate the role that data can play in instructional decision making. But as with any good habit, it’s a long way from knowing what you should do to actually doing it on a regular basis.
In a recent What Works Clearinghouse Institute of Education Sciences practice guide on using student achievement data to support instructional decision making, researchers discuss the obstacles to the use of data at the classroom, school and district levels.
Ideally, use of data is part of an ongoing cycle of instructional improvement. After collecting multiple sources data about student learning, educators interpret data and develop hypotheses about how to improve student learning then modifying instruction to test those hypotheses.
Below are some common obstacles to the use of data in guiding learning and instruction at the teacher level, district level and student level with recommendations on how to address the obstacles:
Obstacle: Teachers have so much data that they are not sure what to focus on to raise student achievement
Formulate specific questions that you want answered and identify the data that will answer those questions. Guide this process by setting schoolwide goals that help clarify the kinds of data teachers should be examining.
Using data should be part of an ongoing cycle of instructional improvement based on developing hypotheses about how to improve student learning. Educators should collect and prepare a variety of data about student learning, interpret the data and develop hypotheses for improving learning, then modify instruction to test those hypotheses.
Obstacle: Teachers are making inappropriate use of state test results
Results from any single test are imprecise and always should be considered in conjunction with other sources of data. One common misuse of data is for teachers to devote a disproportionate amount of resources to “bubble kids”, students who score immediately below the proficiency cut-off on high-stakes assessments.
Students scoring further from the cut score in either direction may have just as many if not more distinctive instructional needs as those scoring near the cut score. Educators should “triangulate” data or use multiple sources of data to get a more accurate picture of the instructional needs of all children.
Collaborative interpretation of data by teachers promotes a collective understanding of the needs of individual students in their school so that they can work as an organization to provide support for all students.
Another misuse of state test results is to assign students to courses.
Tests should be used for the purposes for which they have been validated, the report says. State assessments have not been validated for the purpose of making decisions about course placement. Other sources of data such as curriculum-based assessments, chapter tests and classroom projects can all provide data for instructional decision making. The student’s prior performance should also be taken into account.
Obstacle: Concerns that if students monitor their own own data, they will become discouraged rather than motivated
Having students self-monitor their achievement data can help motivate students to perform better. One concern, however, is that students will view the feedback as a reflection of their ability rather than an opportunity for improvement.
Students are best prepared to learn from their own achievement data when they understand the learning objectives and when they receive data in a user-friendly format. Tools such as rubrics provide students with a clear sense of learning objectives.
Encourage goal setting so that students view the feedback as useful information in reaching for their goals. Emphasize a student’s performance on a task in relation to a specific learning goal. Avoid making global statements about the student’s ability. Help students understand the state standards they are expected to meet by spending a few minutes at the beginning of an instructional unit, for example, by explaining that certain essential concepts in the lesson may appear on the annual test.
Obstacle: The feedback students receive from teachers on their data will be highly variable
Teachers should engage with students in ways they find genuine and effective, but many teachers could benefit from professional development on how to provide concrete and constructive feedback. Teachers should collaborate with peers to develop a shared understanding about what constitutes formative feedback. Students also can be invited to take part in these conversations and to provide input on how they use and respond to feedback.
Feedback should be timely, appropriately formatted, specific and constructive. Consider setting aside 10-15 minutes of classroom time to allow students to interpret and learn from the data.
Obstacle: Teachers feel they don’t have enough time to explain rubrics and help students analyze feedback
Instruction time is limited, yet explaining assessment tools and strategies for analyzing feedback should be a natural, integral part of the teaching process, not an add-on activity.
These activities, when built into the classroom routine, help students develop a habit of learning from feedback, making them more independent as the year progresses. Some helpful computer-based or paper-based tools include a template that lists strengths, weaknesses and areas to focus on for a given task, a list of questions for students to consider and respond, teacher-generated graphs that track student progress over time, etc.
Obstacle: No one is qualified (or wants) to be on the data team
Schools should establish data teams to ensure that data activities are not imposed on educators but rather are shaped by them. Principals should invite individuals who have knowledge–or have a desire to gain knowledge–of data analysis and interpretation.
New teachers or those who recently completed continuing education programs may have just completed training on the use of data. Staff with statistics training or special education certification may have experience with data analysis and interpretation.
Consider the strengths and leadership skills of individuals in your school. Many have related training and skills that make them strong team members. Others may be able to provide enthusiasm and leadership that inspire others to support the data use process. Once qualified and interested staff are identified, consider offering s a small stipend to be on the team.
Obstacle: Data-savvy staff are overwhelmed by questions and request for assistance
It is important for principals and district leaders to protect people’s time by clearly defining roles and responsibilities in job descriptions. Principals should encourage members of the data team to train other educators to use and interpret data. Phasing data use into the entire school also can help prevent staff burnout and widen and deepen staff data literacy.
Obstacle: It’s difficult to find professional development that is specific to the needs of the school
Discuss data needs with a professional development provider to see if the provider can customize a training session that meets the needs of school staff. If a session cannot be tailored to the needs of the school or district, schools should consider using a “train-the-trainers” model. Schools should identify potential trainers for the school, such as professional development staff within the district office, who can receive broad training on data-based decision-making and then adapt the training to fit the needs of the school or district.
Obstacle: Resources budgeted for creating staff capacity to use data are needed for other school priorities
Data-based decision making is not an isolated issue but rather one that benefits all subject areas and grades. Dedicating resources to data literacy will help support and enforce a culture of data use that generally will assist educators in helping students meet their learning goals.
Obstacle: Using a data system is daunting for staff who are technophobes or technologically challenged
The district should not implement the data system without professional development and technology training sessions for a variety of skill levels. Principals and data facilitators should be available to support teachers’ use of data within the school building. Mechanisms for providing assistance on an as-needed basis (technology help desk) should be in place as soon as educators start using the system.
Data facilitators should not only have expertise with data analysis but also have an ability to train and encourage other staff. They should meet at least monthly with grade- and subject level teacher teams and should help staff obtain the knowledge and skills they need so that they do not become too dependent on assistance.
Be very careful that educational goals are front and center–technological requirements and considerations should never be put before the educational goals that the system supports. If the implementation plan clearly articulates how the system relates to learning goals, staff will better understand how the system is to be used.
Obstacle: In these challenging economic times, a data system is a luxury given the other pressing educational priorities
For districts that prioritize and indicate that the use of student data is a priority, a data system must equally be a priority. The district plan should clearly explain and illustrate how a data system supports its goals. A central message of this practice guide is that effective data practices at the classroom, school and district levels are interdependent.
“Using Student Achievement Data to Support Instructional Decision Making,” by Laura Hamilton et al., National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, September 2009.