Blog: The Educated Reporter

High-Stakes Testing Errors Nationwide Revealed in New Atlanta Journal-Constitution Series

High-Stakes Testing Errors Nationwide Revealed in New Atlanta Journal-Constitution Series

We spend quite a bit of time talking about cheating on standardized tests — who does it, how to prevent it, the pressures that might be contributing to it. What we don’t talk much about are the errors that occur before a student is ever handed a bubble sheet and the mistakes that can occur after the completed exams are turned in.

Investigative reporter Heather Vogell of the Atlanta Journal-Constitution has spent the past year examining testing errors nationally and the impact on students, teachers, schools, and states. The problems revealed in her reporting range from poorly worded assessments to wide-scale errors by testing companies. In some cases, students were wrongly denied high school diplomas based on faulty scores.

The new series is a followup to the newspaper’s award-winning “Cheating Our Children” project, which found suspicious test scores both in Atlanta and in districts nationwide. The team’s dogged reporting lead to the early retirement of Beverly Hall as Atlanta’s schools superintendent. Since then 36 people — Hall included — have been indicted on criminal charges.

I asked Vogell if she was prepared for her work to be used as ammunition by opponents of standardized testing, a fight which has been heating up in recent months. She told me that she “knew people who oppose these tests would use this as more evidence of their flaws.” However, many of the people she met during the course of her work “don’t say they never want their kids tested. They’re concerned about the quality and implementation. That makes me think there’s some common ground that can be reached on the use of measurement in schools.”

There are plenty of lessons to be learned from the series, including how even a seemingly well-designed system of checks and balances can’t always keep well-intentioned people from making mistakes. Just one example: a question on Georgia’s social studies test for sixth graders asked students to identify the professional occupation of Andrew Lloyd Webber, the British composer behind hit musicals including “Phantom of the Opera.” But instead of listing “composer” as one of the multiple choices, the “correct” response was supposed to be “playwright.” Once the error was discovered, undoing it became a logistical nightmare – and, of course, raised questions over whether this was a piece of knowledge Georgia students should really be expected to know (state officials later decided it wasn’t).

Testing is “sold as a scientific enterprise. But we have all these human actors making judgment calls all the time,” Vogell said. (You can read more of my interview with her here, and also watch our Google Hangout above.)

Vogell did much of the research for the project during her year as a Spencer Education Fellow at Columbia University’s journalism school. This is just the latest example of the remarkable work that reporters are able to conceive and carry out with the support of the fellowship. Another example: Divided We Fail, a terrific new book on the landmark fight against school segregation in Louisville by The Hechinger Report’s Sarah Garland.

As for the AJC’s latest series, the second part comes out Sunday and the third and final installment will run next week. Robert Schaeffer, public education director for the advocacy group FairTest, which opposes overuse and misuse of standardized assessments, said it’s “an important step toward bringing accountability to an industry whose products are increasingly used to assess students, teachers and schools … It is high time for the U.S. to begin examining the examiners.”