2014 was, as you have probably heard by now, not the best year for polling; even Nate Silver concluded not only that the polls were skewed to overstate the performance of Democrats, but that it was, on average, tied with 2002 as the second-worst polling year for Senate races since 1990 (the worst was 1998). That couldn’t be predicted before the election, but a major reason why it might be true was: that too many pollsters were still using a 2012 turnout model, and were afraid that departing from that would open them to criticism of “unskewing” the electorate, undercounting non-white voters and underestimating the Democrats’ vaunted turnout operation. And there was a particular operational reason why, as Erick explained about 10 days before the election:

[A]ll the states I have noticed that have wildly fluctuating polls are also states that had no exit polling in 2012. The national consortium decided they did not need to poll those states and could save money.

Alaska, Arkansas, Delaware, District of Columbia, Georgia, Hawaii, Idaho, Kentucky, Louisiana, Nebraska, North Dakota, Oklahoma, Rhode Island, South Carolina, South Dakota, Tennessee, Texas, Utah, West Virginia and Wyoming had no exit polling.

…I remember enough of the problems from 2012 and the analysis thereof to wonder if not having a 2012 baseline from the exit polling is making a difference. Pollsters are having a harder time figuring out the composition of the electorate. And, if they use outside surveys from 2012, the only good ones were from left-leaning partisan groups. Their turnout models were well done, but also agenda driven showing more Democrats than actually showed up.

…[N]o one knows who will show up and the most cautious pollsters are desperate to not turn into a Rasmussen or Gallup 2012.

In retrospect, it seems that this may be precisely what happened:

A senior Republican strategist who helped direct the GOP’s midterm strategy speculated that media outlets weren’t willing to spend enough on their polls to produce a product that was as detailed as the private, more accurate partisan surveys. This Republican also believes that public pollsters in general were “vulnerable and susceptible” to the arguments Democrats were making about expanding the electorate, given what the party achieved in 2012.

“The Democrats kept telling us they had a gold-plated ground game,” added Republican pollster Wes Anderson, who worked on Sen.-Elect [mc_name name=’Rep. Tom Cotton (R-AR)’ chamber=’house’ mcid=’C001095′ ]’s race. “People kept weighting up Democratic turnout.”

I’ll come back to some of the theories about the 2014 polling another day, but I thought it would be useful to specifically examine Erick’s pre-election thesis: that the polls were struggling worse in states that had no 2012 exit polls. We have a RealClearPolitics final polling average for 34 of the 36 Senate races (polling was too sparse for an average in New Mexico, and [mc_name name=’Sen. Jeff Sessions (R-AL)’ chamber=’senate’ mcid=’S001141′ ] was unopposed), 20 in states that had no exit polls (including 2 each in Oklahoma and South Carolina) and 14 in states that had exit polls. While this isn’t a large enough sample to conclude as a statistical matter that there was a difference, it’s enough states that if there was a difference, we should expect to see some evidence of that.

And we do. Here’s the RCP average compared to the final results in the 20 Senate races in states with no 2012 exit polls:

For ease of comparison and because it’s the one thing consumers of polls most want to know, I used the Republican’s percentage of the 2-party vote among the top 2 candidates in the race as the basis for comparison. On an unweighted average basis across these 20 Senate races in states that had no 2012 exit polls, measured by the GOP’s share of the 2-party vote, the polls were off an average of 3.4 points, and a net average of 2.8 points in the Democrats’ direction. In 18 of the 20 races they were skewed more towards the Democrat, in only three ([mc_name name=’Sen. Tim Scott (R-SC)’ chamber=’senate’ mcid=’S001184′ ] and [mc_name name=’Sen. Lindsey Graham (R-SC)’ chamber=’senate’ mcid=’G000359′ ] in South Carolina and the hapless Cam Cavasso in Hawaii) did they overestimate the Republican. And the errors were not trivial: in only 3 of the 20 was the final RCP average within 1 percentage point of the final GOP share of the 2-party vote share, in 5 of the 20 was it within 2 points.

Now, the 14 races where there were 2012 exit polls (this is a bluer group of states and includes more races won by Democrats, so there’s that as well):

Here, we see a different story. Across these 14 Senate races, polls were off an average of 1.6 points overall, and a net average of 1.0 points in the Democrats’ direction – an average error half as large, and a skew barely a third as large. In 10 of the 14 states they were skewed more towards the Democrats, but in 5 of the 14, the final RCP average was within 1 point; in 11 of the 14, within 2 points. The polls were still off quite badly in Kansas and off in Iowa and Maine, but on the whole, they performed much worse in the states that had no 2012 exit polls.

ADVANTAGE: Erick.