Teachers have made informal use of formative assessments for years, but more and more districts want to take a more formal approach to formative assessments to ensure that their students are on track to meet statewide assessments. Before adopting a formative assessment system, districts must make many preliminary decisions, reports this study in the American Journal of Education.
Conclusion: Before choosing a formative assessment system, one of the most important decisions a district must ask is: Who will be using the formative assessment results–administrators or teachers? A district also must assess its own infrastructure and set of competences. Schools where educators have already learned to work collaboratively to analyze student work and design effective instructional improvement strategies have different needs for implementing such a system than do schools that have little capacity to do this work.
Main research question: What are the key decisions to make before developing or choosing a formative assessment system?
Participants: 42 individuals from one large urban district that was implementing reform based on the increased use of data. Participants included the superintendent, district-level administrators, teachers, school-based administrators and the vendor that provided the computer-based assessment program to the district.
Method: Interviews with participants that focused on how teachers and administrators use student assessment results. Researchers also reviewed documents related to district’s recent history and current challenges.
| LESSONS LEARNED:|
• If a computer-based system is chosen, additional technology investment such as data storage and software may be needed by the district, depending on its existing infrastructure.
•Teachers and administrators may need professional development to help them analyze student assessment results and translate the knowledge effectively.
•It’s important to consider the existing competences of educators for working collaboratively on designing instructional improvement.
Findings: What is the purpose of the assessment is a key question that should guide the choice of a formative assessment system. The superintendent in the school district at the center of this study saw four roles. Formative assessments could assist:
- teachers in planning instructional interventions;
- principals in developing improvement plans for students with lagging skills;
- the superintendent in identifying schools that are not making adequate progress;
- and updating the school board on the likely performance of students on the next round of the state’s math tests.
Among the key decisions districts must make in selecting a system are:
Compulsory or voluntary? If the assessments are to be used by the central office to identify struggling schools, then assessments will have to be scheduled at the same time with the same test administration and security methods, the researchers note.
One disadvantage of mandatory assessments is that faculty may see the process as one more externally imposed mandate rather than as a source of information for them. If the primary goal of formative assessments is to provide educators with information, the design choices would be different than if the goal is to hold principals accountable.
Make or buy? Buying a system is faster than developing a home-grown system, researchers report. Vendors have done the work of aligning the assessments with state standards and assessments. However, the cost of purchasing a commercial system is usually higher (cost can range from $5 to $75 per student per year) Teacher buy-in is usually higher with a system that is developed internally.
Computer-based or paper-based? One major advantage of computer-based assessments is quick and easy scoring as well as graphic summaries of results. Another advantage is that students like taking tests on computers.
An important disadvantage of computer-based testing is that all items are multiple-choice questions so that educators do not get information about students’ abilities to answer open-ended questions. Computer-based testing can also place a strain on a district’s computer infrastructure.
Computer-adaptive assessments? Computer-adaptive testing allows for testing of students with widely divergent skills. A short test can test the abilities of students with low skills while a longer test is needed to test students with higher skills.
One disadvantage of computer-adaptive assessments is the considerable amount of time that is needed to test the higher-ability students–an hour in one of the pilots tested in this school district. Another disadvantage, as with any computer-based testing, is the demands it places on the infrastructure.
Because of bandwidth issues at the district in this study, there was also a long lag between the student answering a question and the next question appearing on the screen, causing the student to lose interest in the process.
Study: “Tough Choices in Designing a Formative Assessment System”, by Nancy Sharkey and Richard Murnane, American Journal of Education, August 2006, Volume 112, pp. 572-588.
From Assessment for Learning: 12 recent studies on formative assessment and aligning assessments with learning goals, published by Educational Research Newsletter August 2007