EXPERTS SAY SCHOOLS NEED TO SCREEN FOR CHEATING

 

By SHAILA DEWAN | New York Times

February 13, 2010 -- ATLANTA — This week, Georgia officials said they had found evidence that cheating might have occurred on standardized tests at one in five public elementary and middle schools around the state. What was extraordinary, however, was not so much the extent of the problem, but the decision of the state to screen for cheating at all.

Using a computer scanner, the state used a simple, quick analysis to flag classes where an unusually high number of wrong answers were erased and corrected. The testing company generated the data at no charge.

Yet even as test scores carry greater stakes for students, schools and districts, testing experts say most states fail to use even this most elementary means to monitor for cheating.

“No one is doing it, and when you ask people why they’re not doing it, they shrug their shoulders,” said Jennifer Jennings, a sociologist at New York University who studies school accountability.

Ms. Jennings suggested that the federal government should require states to check their test results. “It’s absolutely scandalous that we have no audit system in place to address any of this,” she said.

Cheating on tests used to be thought of as primarily the domain of students, but as standardized test results have taken on an increasing importance as a way to measure schools, the culprits have increasingly turned out to be educators, experts said.

Under the federal No Child Left Behind law, schools are required to meet improvement goals or face penalties including, in the worst cases, the loss of jobs. Cities like New York and Houston have recently threatened the tenure of teachers whose students do not meet goals.

As the consequences have grown more serious, reports of cheating have exploded, said Robert Schaeffer, the president of FairTest, an organization that opposes the emphasis on standardized testing. “They’ve gone from a handful a year to a handful a month,” he said.

Because parents, students and administrators all like to see higher scores, said Gregory J. Cizek, a testing expert at the University of North Carolina, “There’s really no incentive to vigorously pursue cheaters.”

He said some states did not ask their testing contractors to generate an erasure analysis, while others did receive them but did not use them.

One problem, experts said, was asking school systems to police themselves, which often requires the kind of independent oversight set up in Georgia. The state Department of Education is led by an elected superintendent, Kathy Cox, but the governor, Sonny Perdue, controls a separate Office of Student Achievement, which has auditing powers.

It was the Office of Student Achievement that conducted the erasure study, not the Education Department.

Even states that have weathered widespread cheating scandals do not necessarily follow up with regular statistical monitoring. In 2005, after an investigation by The Dallas Morning News pointed to extensive cheating in Texas, the state hired Caveon Test Security, a Utah company that improves testing procedures, to conduct what the company calls “forensics analyses” of answer forms. But the company was not retained to do yearly monitoring, said John Fremer, Caveon’s president.

Caveon’s forensics analyses use several methods of detecting cheating, screening not only for erasures but improbable increases or decreases in scores, individual students whose performance swings widely from year to year, patterns where multiple students share the same wrong answers and other anomalies.

Erasures alone only indicate certain types of misconduct, as when answers are changed after a test. Other methods, Mr. Fremer said, flag other types of cheating, like filling in the remaining answers on an incomplete form.

States that are not checking answers with such forensic measures cannot use the excuse that they are new, said Walt Haney, a senior researcher at the Center for the Study of Testing, Evaluation and Educational Policy at Boston College. Using statistics to detect cheating on standardized tests dates back to the 1920s, and erasure analyses are practically as old as filling in bubbles on answer forms with a No. 2 pencil.

Of about 16 state public education clients of his company, Mr. Fremer said, fewer than 10 conduct such analysis regularly. A few other states use their own testing vendors, as Georgia did, to provide similar data. Mr. Fremer said he thought more states would move toward statistical analysis in order to maintain public confidence in test scores and school ratings.

“I don’t think they can avoid doing it,” he said. “There’s too much riding on the test results.”

Southern states, which have embraced the accountability movement in education, have also been quicker to adopt statistical methods to combat cheating.

South Carolina has been quietly using an erasure analysis since the 1980s, said Elizabeth Jones, the director of the state Education Department’s Office of Assessment. If a class is flagged for suspicious activity, the state sends testing monitors the following year, and sometimes educators are criminally prosecuted or lose their teaching certificates.

Principals and teachers are well aware that the state can detect erasures, and only a handful of classes are flagged each year, Ms. Jones said.

In Washington, the superintendent of education has recently conducted the first of what is to be an annual statistical analysis of test results. Twelve of about 230 schools were flagged and asked to conduct investigations, said Chad Colby, a spokesman for the Office of the State Superintendent of Education. (The state superintendent oversees the District of Columbia Public Schools and the city’s charter schools.)

In Mississippi, Caveon does forensics analyses each time a test is administered, and the state withholds questionable scores until an investigation is completed.

“Initially, it was a new thing and folks were a little skeptical — could we really reach these kind of conclusions just by looking at the data?” said Kristopher Kaase, Mississippi’s deputy superintendent for instructional programs. But investigations bore out the statistical findings. “That’s made believers out of the school districts,” he said.

Robbie Brown contributed reporting.

This article has been revised to reflect the following correction:

Correction: February 16, 2010
An article on Saturday about the failure of states to use computerized scanning to check for possible cheating on standardized education tests misspelled the given name of the Georgia superintendent of schools and misidentified her political affiliation. She is Kathy Cox, a Republican — not Cathy Cox, a Democrat, who was Georgia’s secretary of state from 1999 through 2007.