Posted by: Patrick Allen Foster | July 8, 2013

The Essay-Scoring Industry

City Pages has this engrossing piece about the business of scoring essays for standardized tests. According to the piece, the No Child Left Behind legislation tripled the business for scoring standardized tests: “‘The amount of testing that was being done mushroomed,’ says Kathy Mickey, a senior education analyst at Simba Information. ‘Every state had new contracts. There was a lot of spending.’ The companies that create and score tests saw profits skyrocket. In 2009, K-12 testing was estimated to be a $2.7 billion industry.”

For scoring the essays, the companies hire phalanxes of graders, most of them temps. Typically, each essay is scored by two graders, and each grader scores some 200 essays per day.

An excerpt:

Although DiMaggio had been through a training process, he found himself tripped up as he began scoring the essays. What made the organization “good” as opposed to “excellent”? What happens when the kid doesn’t answer the question at all, but writes with excellent organization about whatever the hell he wants? Did it matter that it was insane for seventh-graders to think they’d be benching 200 pounds?

DiMaggio had good reason to worry. His score could determine whether the school was deemed adequate or failing—whether it received government funding or got shut down.

Do read the whole thing. Essays are especially difficult to score in a standardized manner. (Let’s not think about that too much as thousands of law school graduates around the country are getting ready to take bar exams in about three weeks.)

Advertisements

Responses

  1. I find the topic too depressing to actually read the linked article, but I recall the dismal experience of grading freshman comp essays. To preserve my sanity, I had to work with a system more or less like this:

    A = huh, hey, that’s actually pretty good
    B = if someone I worked with wrote this, I would not think him an idiot
    C = typically bad but gets the idea across
    D = whoa, maybe this co-worker *is* an idiot
    F = I have no idea what point this is even trying to make. Communication FAIL.

    • Interestingly, your system sounds remarkably similar to the scoring rubric described in the article (only they use a scale of 0-6, with a 6 being “excellent” and a 0 being…the other end of the scale).

    • Pretty much all you can do, though I’m skeptical how precise one can be about 7 gradients … A-F is stretching it.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: