Posted by: Patrick Allen Foster | September 23, 2014

Scottish Independence Referendum After-Action Analysis, Part I

Nate Silver has written a post exploring why, in his view, the polling on the Scottish Independence Referendum last week was off the mark — that is, even the polls predicting a “No” victory predicted that the margin of victory would be substantively narrower than the actual 10-point margin that was the actual result.


I have only a theory: Scotland’s results may have had something to do with the “Shy Tory Factor.” This was the tendency of conservatives (Tories) to outperform their polls during a number of U.K. elections in the 1990s and especially in the U.K. general election of 1992. The idea is that conservatives were less enthusiastic than Labour voters and therefore less likely to declare their support for a conservative government to pollsters. Nevertheless, they turned out to vote.

U.K. pollsters responded to these elections in a variety of ways, including by weighting their results based on voters’ party preference in prior elections. But this may have been a patch that failed to address the underlying issue: Voters are not equally likely to respond to polls; those who are more enthusiastic about an upcoming election are more likely to do so.

This potentially leads to a double-counting of enthusiastic voters if turnout models are not applied carefully.

The problem could become worse as response rates to polls decline. Furthermore, many polls of the Scottish independence referendum were Internet-based, and some of those polls did not use probability sampling, historically the bedrock for demographic weighting. …

At Slate, Dave Weigel thinks that Silver is missing something:

Now, it’s true that if you locked yourself in cryo-freeze nine months ago and expected a “No” win, you’d have been right. Just as you’d have been right if you said “I bet the frontrunner, Mitt Romney, wins the GOP nomination” in 2011, and ignored everything else that happened. If you want to shake your head at the reporters who wrote that “to walk through virtually any Scottish town this week was to be confronted by an apparently unassailable army of people voting Yes,” you can!

But you would have missed a panic that changed the future of 5.4 million Scots. The “No” coalition, composed of the three main British parties, responded to the “Yes” surge by pushing up or amending their own promises for further Scottish devolution. They had been expected to offer some of this stuff in six-odd months, in party manifestos. Instead, they tried to head off a “Yes” win by assuring wavering voters that unemployment programs, some housing programs, and significant taxing powers be devolved to the Scottish parliament. Banks and investors threatened to bolt the country. Ironically, this decision of “yes” or “no” was less binary than the standard U.S. presidential election. When one of our candidates wins, he wins. When the “Yes” campaign got close to winning, the “No” campaign made huge concessions, and this simply wouldn’t have happened had the polls stayed at 60/40 for “No.”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: