City Pages has this engrossing piece about the business of scoring essays for standardized tests. According to the piece, the No Child Left Behind legislation tripled the business for scoring standardized tests: “‘The amount of testing that was being done mushroomed,’ says Kathy Mickey, a senior education analyst at Simba Information. ‘Every state had new contracts. There was a lot of spending.’ The companies that create and score tests saw profits skyrocket. In 2009, K-12 testing was estimated to be a $2.7 billion industry.”
For scoring the essays, the companies hire phalanxes of graders, most of them temps. Typically, each essay is scored by two graders, and each grader scores some 200 essays per day.
Although DiMaggio had been through a training process, he found himself tripped up as he began scoring the essays. What made the organization “good” as opposed to “excellent”? What happens when the kid doesn’t answer the question at all, but writes with excellent organization about whatever the hell he wants? Did it matter that it was insane for seventh-graders to think they’d be benching 200 pounds?
DiMaggio had good reason to worry. His score could determine whether the school was deemed adequate or failing—whether it received government funding or got shut down.
Do read the whole thing. Essays are especially difficult to score in a standardized manner. (Let’s not think about that too much as thousands of law school graduates around the country are getting ready to take bar exams in about three weeks.)