Tuesday, March 08, 2011

Campbell's law redux: Systemic cheating edition

Kudos to the researchers at USA Today newspaper, who investigated standardized test scores in six states, Arizona, California, Colorado, Florida, Michigan and Ohio. In each case the research found substantial numbers of statistical "anomalies" (read: cheating):
The newspaper identified 1,610 examples of anomalies in which public school classes — a school's entire fifth grade, for example — boasted what analysts regard as statistically rare, perhaps suspect, gains on state tests...There were another 317 examples of equally large, year-to-year declines in an entire grade's scores. 
To people used to the standard narrative this story may seem shocking, but to people familiar with Campbell's Law these results are to be expected. Cheating is the natural result of a process that elevates testing from a diagnostic to a policy tool.

Some of the cheating is openly defended by the practitioners. One superintendent told the newspaper about how his district's strategy of education revolved around teaching kids how to pass tests:
"Yeah, we do that. Everyone does that in America," he says. "There's no secret about the test. It's a standards-based test. We teach to the test."
In California, where hundreds of regular public schools are closing while hundreds of charters are opening the scandal is particularly severe. One hundred twelve such "incidents" were investigated and reported in just the past two years.  In Los Angeles the founder and executive director of a six school charter system "had orchestrated widespread cheating on 2010 tests."

Despite these scandals
"California dropped the audits two years ago because of record budget deficits. And the state no longer collects data on which schools show unusually high rates of erasures on answer sheets — sometimes a clue, experts say, that either students or school officials might be cheating. Total savings: $105,000."
So in a time when test scores are being used to judge teachers, principals and superintendents to an unprecedented degree, scrutiny of those scores is being reduced. Meanwhile cheating has become systemic. California saved $105,000 by discontinuing efforts to ensure there wasn't widespread fraud. That same year the Gates foundation spent $45 million on a  bogus study that aimed to prove that teacher quality could be determined by the test scores of their students. The longer the education deform movement goes on the more absurd it becomes.

No comments: