Tuesday, December 06, 2011

WHAT ‘MULTIPLE MEASURES’ REALLY MEANS IN EVALUATION

 

By Lisa Guisbond and Monty NeillValerie Strauss’ Answer Sheet/Washington Post | http://wapo.st/rK74cI

This was written by Lisa Guisbond and Monty Neill of FairTest, the nonprofit National Center for Fair and Open Testing, which is dedicated to eliminating the misuse of standardized testing. Guisbond is a policy analyst and Neill is the executive director.

05:00 AM ET, 12/06/2011 :: Like the same old laundry detergent with a “new and improved” sticker, sometimes the only real change in U.S. education policy is the language used to describe it.

The federal No Child Left Behind (NCLB) law is badly tarnished. Even Education Secretary Arne Duncan is running away from it, offering waivers from NCLB’s demand for “adequate yearly progress” on state test scores. To get the waivers, states must propose how they will measure student progress.

Some states, such as Minnesota and Colorado, have said they will use “multiple measures.” This would represent significant progress if state politicians were talking about the real thing.

Real multiple measures use a variety of different ways to measure student learning over time. They are the only way to ensure that children are getting a well-rounded, multifaceted education — not just being trained to bubble in the best choice among four options or writing a few paragraphs in response to a generic question.

Unfortunately, Minnesota, Colorado and other states mostly propose nothing more than just looking at the same old test results in different ways: the actual score, year-to-year growth in scores that compare students to their previous scores or to the state’s goals, or whether gaps between whites and other student subgroups are narrowing.

These are merely multiple ways to slice and dice the same information – they are not different kinds of measures. Worse, teachers will still be pressured to focus narrowly on preparing kids for standardized tests. The problems with NCLB — narrowed curricula, teaching to the test, and temptation to cheat — will remain.

Examples of real multiple measures abound. Among many possibilities, they include science labs and field work, from short tasks to extended projects; oral presentations in any subject; extended math problems that require application to real world uses; and in-depth history reports, presented orally, in an essay, a PowerPoint, etc.

The complex question is how to put these measures together plausibly and defensibly, but this has been done in the United States and other nations. (For more discussion and examples, see FairTest’s Multiple Measures fact sheet.)

Freed from the strictures of high-stakes testing, Finland has achieved great success using true multiple measures. Finnish education authorities periodically evaluate samples of students’ classroom work to determine the quality of teaching and learning in each school. The nation often ranks at or near first in various international comparisons. From Australia to Singapore to England, nations have found ways to use performance tasks and classroom-based evidence to evaluate how well students are doing and to inform school improvement efforts.

Despite the heavy hand of high-stakes standardized tests, there are schools and networks of schools in the United States that use multiple measures. For example, the New York Performance Standards Consortium schools use complex performance-based assessments instead of all but one of the state’s high school tests. The Primary Learning Record is a structured means of evaluating student progress in reading and writing based on assembling examples of each student’s work across the year. Both of these networks have shown strong validity and reliability.

In comparison, the ersatz “multiple measures” offered in exchange for NCLB waivers tell us little about students or our schools, and their overuse damages curriculum, instruction and student learning.

For more information about states misuse of the term “multiple measures,” see articles on Minnesota and Colorado.

No comments: