We come at last to the home stretch, and that means it’s time to consider again what the polls mean, what they can tell us, and what they can’t. The short answer is that we still do not know what is going to happen – but we still have a good deal of evidence to work with, and most of it suggests that Republicans have a strong chance to pick up anywhere from 8 to 10 Democrat-held Senate seats, but are still in serious danger of losing one seat of their own and some danger of losing a second.

I last looked at the Senate races on October 22. As always, my method – explained at the outset – is to look specifically at the question of, assuming the accuracy of the RCP poll averages, what percentage of the remaining undecided voters would need to break in the GOP’s direction to win? My underlying optimistic assumption is that the national environment should favor Republicans in winning more than half of those voters in most states. The news, for the most part, continues to be consistent with that assumption, but only the election itself will prove or disprove it.

Trust, But Verify

Before we get to the numbers, a few observations on the return of some of the perennial polling debates. First, Sean Trende – whose work I’ve relied on a good deal in developing his original thesis about the gravitational pull of Obama’s low approval ratings and the GOP edge in the generic ballot, and how those factors would be reflected in the movement of late-breaking undecideds – lays out the two most likely paths to the Senate endgame: that the remaining large numbers of undecideds will mostly break heavily to the GOP, and that they will mostly stay home. Obviously, if the polls are otherwise correct, both results will lead to a good night for Republicans, but the first scenario will lead to a much better one, probably a wave big enough to save [mc_name name=’Sen. Pat Roberts (R-KS)’ chamber=’senate’ mcid=’R000307′ ] and tip the North Carolina and New Hampshire races to the GOP.

Then we have Nate Cohn, who at this point in 2012 was reassuring us that “Friendly Reminder: The Polls Are Usually Right”, writing an extensive look in the NY Times at “Why Polls Tend to Undercount Democrats.” Now, the initial instinct of a lot of us on the Right is to point and laugh at this, given how heavily liberal/progressive bloggers, op-ed writers and White House flacks overinvested in 2012-13 in the idea that polls were infallible and anyone who questioned them were a bunch of science deniers on par with Holocaust deniers (if you think I’m kidding, try Googling phrases like “poll denier” or searching for them and “unskewed” on Twitter). But those of us who did actual, serious poll analysis and criticism in 2012 and thereafter should instead welcome people like Cohn to the ranks of those who openly admit to treating polls with informed skepticism rather than religious awe (to be fair to Cohn, he has done some great work over the past two years on the foibles of PPP in particular). My own view of poll analysis – like most fields of inquiry, and maybe moreso – is that it’s a field that benefits greatly from the adversarial give-and-take of analysts with different points of view and different partisan loyalties, in which you lay your assumptions on the table, show your work, back up your arguments and subject them to the test of reality.

Cohn’s article is worth a read if you’re interested in the nuts and bolts of how polls work, because he gets into the social science of why declining poll response rates and the demographics of the Democratic coalition make it harder for pollsters to be confident that they have enough Democratic voters in their samples. But in connecting that to Democratic arguments that the poll toplines in 2014 may be systematically favoring Republicans, Cohn makes a simple bait-and-switch error. He asserts that “The polls have generally underestimated Democrats in recent years, and there are reasons to think it could happen again.” But because pollsters are already adjusting their polls for the undercounting issue, he is basically double-counting by implying we should further upgrade the Democrats’ standing in the polls that have been subjected to that adjustment – exactly the sort of thing I described as “alchemy” when Dean Chambers was doing it.

First of all, for his assertion of bias in the final results, Cohn uses data from just two election cycles, 2010 and 2012, which is hardly a large sample, and switches between Senate races in one year and presidential polls in another:

In 2010, the polls underestimated the Democrats in every competitive Senate race by an average of 3.1 percentage points, based on data from The Huffington Post’s Pollster model. In 2012, pre-election polls underestimated President Obama in nine of the 10 battleground states by an average of 2 percentage points.

I’ve gone through the final Senate polling at RCP for every competitive race from 2002 through 2012, and I’ll publish my results of that study in the days to come, but measured against the two-party share of the vote in the final polling, I found a smaller effect in those two years, and different effects in others:

2012: D-1.8 (i.e., Democrats did 1.8 points better than the polls)
2010: D-1.0
2008: R-0.9
2006: D-1.0
2004: D-0.4
2002: R-1.3

But even in years when there was an overall bias, it was hardly consistent across races. Overall, yes, the Senate polling shows more races (40) in which the Democrat beat his or her share of the 2-party vote in the final polls by 1 point or more than races (23) in which the Republican did so – and the number of such misses should give us pause about over-trusting the polls. Here’s the list of races where the pollsters underestimated the Republican – you’ll notice that repeat offenders here include Alaska, Colorado, and New Hampshire, all 2014 battlegrounds, and that the polls specifically overrated [mc_name name=’Sen. Mark Begich (D-AK)’ chamber=’senate’ mcid=’B001265′ ], [mc_name name=’Sen. Mark Udall (D-CO)’ chamber=’senate’ mcid=’U000038′ ] and [mc_name name=’Sen. Jeanne Shaheen (D-NH)’ chamber=’senate’ mcid=’S001181′ ] in 2008 and Shaheen in 2002 as well:

The 2012 and 2010 races had the largest tilt – 23 races where the Democrats overperformed the polls, compared to 5 for the GOP – but there were races going both ways each year. In fact, as both Trende and Nate Silver have demonstrated, there’s no entirely predictable pattern in poll misses from one year to the next.

The real problem is that Cohn elides the difference between raw poll results and published toplines. He argues that polls undercount Democrats, so they may be wrong. But pollsters know this already, and so they try to fix the issue by modeling the electorate, as he acknowledges:

The most highly regarded pollsters, like those at Pew, have made many adjustments to compensate. But other polls, including many of those informing polling averages and Senate forecasting models, are not nearly as high in quality. Another highly regarded pollster, who requested anonymity, put it more bluntly, calling the new challenges “scary.” Many pollsters are reluctant to say things on the record that might undermine confidence in their own polling; others are unwilling to say anything at all, even to offer basic methodological information.

As you may recall, the argument about the inconsistent quality of some pollsters was a common one made by Trende and others in 2012, but did not show up in the presidential polling as a tilt in the averages so much as just random noise. But what Cohn misses is that he’s doing exactly what those of us on the Right did in 2012: taking issue not with the counting, but with the assumptions behind the modeling of the electorate, with what pollsters assumed about turnout. In the end, if Cohn is right about counting but wrong about modeling, he’ll be wrong about outcomes, which after all is what consumers of polls care about: do they reliably predict outcomes? We will know the answer to that (at least for this year) after Tuesday.

The reason why this is never settled is that the electorate changes, the responses to polls change, and the pollsters change their models, constantly evolving towards a moving target. You never get a big enough sample size of poll results derived from a single methodology applied to a single electorate to form scientific conclusions before the world changes again. That’s why every kind of analysis – historical, demographic, mathematical, etc. – is a guide, not holy writ.

Back To 2014

Now, on to the main chart, which once again includes a section on what Republicans need to force [mc_name name=’Sen. Mary Landrieu (D-LA)’ chamber=’senate’ mcid=’L000550′ ] and Michelle Nunn into runoffs, and also includes an alternative figure for North Carolina 2-way polling (which is now actually more favorable to [mc_name name=’Sen. Kay Hagan (D-NC)’ chamber=’senate’ mcid=’H001049′ ] than 3-way polling, even though the latest polls are 2-way polls that look up for Thom Tillis).

Thoughts as we go, race-by-race.

In South Dakota, it now looks like the brief Democratic enthusiasm for toppling Mike Rounds has completely collapsed. In fact, nearly every poll in the race has showed Rounds getting 55-60% of the two-party vote (the one Harper poll that got everyone agitated still had him at 52.9%), and the only real variable was the third party vote for Larry Pressler, which has since receded.

The last few polls in Arkansas are essentially the death rattle of [mc_name name=’Sen. Mark Pryor (D-AR)’ chamber=’senate’ mcid=’P000590′ ], the first Democratic incumbent we can legitimately declare to be finished. Incumbents at 41% and down 7 points in the polls five days before the election in an intensively polled race just don’t come back.

Iowa has seen [mc_name name=’Rep. Bruce Braley (D-IA)’ chamber=’house’ mcid=’B001259′ ] stabilize his poll position, but in the last six public polls he’s been at 45, 45, 46, 44, 43, and 45 – he shows no sign of being able to pull much ahead of Obama’s 40.4% approval rating. And Joni Ernst is at 49, 49, 47 and 48 in four of those six polls, the other two showing her at 44. In other words, Braley is stuck, and the optimistic case for him is that the polls showing Ernst close to 50 are wrong.

Alaska looks tighter thanks entirely to a single poll by something called “Ivan Moore Research,” which has Begich up by 6 points; the RCP average had Sullivan up by 4.2 before this one poll. Ivan Moore Research is, so far as I can tell, a pollster paid by some undisclosed and possibly partisan client, that has never published a poll in an Alaska Senate or Governor’s race, so it’s impossible to gauge its credibility.

Alaska is notoriously hard to poll but also notorious for its polls underestimating Republicans. If there’s an optimistic case for Begich, it’s that (1) some of that is due to Native American groups that have supported Republicans in the past, some of which now support Begich, and (2) Begich is reported to have put more per capita resources into his GOTV operation than possibly any candidate in the history of modern Senate races. But that’s just another way of saying that, Ivan Moore Research or no, the polls could be wrong.

David Perdue has pulled back ahead in Georgia, leading Republicans to breathe a sigh of relief, and as Erick has noted, the winds shifted back in his favor after Obama went on the radio to talk up Michelle Nunn. Perdue has probably long since blown his chance to win this race without a runoff, which would be the fourth time he faces the voters in this election. But he now looks in good shape to at least head off Nunn beating him on Election Day, and a runoff probably favors Perdue.

I’ve said for a while that if Pat Roberts pulled ahead of Orman in Kansas, he’d stay there. He did pull into a tie for a while, but has not led since Chad Taylor was in the race. Orman’s poll standing has deteriorated, a sign that the pounding he’s taken has removed some of the shine from him, but both Roberts and Sam Brownback, despite being alienated from different wings of the GOP, are suffering in common from the confluence of that dissatisfaction.

The cheese has slid completely off the cracker for Terri Lynn Land in Michigan, where [mc_name name=’Rep. Gary Peters (D-MI)’ chamber=’house’ mcid=’P000595′ ] now has a larger lead than Dick Durbin. The vast numbers of undecided voters in that race a few weeks ago have now evaporated, nearly all of them going to Peters. Michigan was always going to be tough sledding, but Peters is nothing inspiring, and I still think in a year when the GOP is leading Senate races in Iowa and Colorado and favored to re-elect governors in Michigan, Wisconsin and Ohio, this could have been a winnable seat. Instead we got a candidate so bad her handlers were afraid to let her take questions or even publicize her appearances.

The hottest races of all are the two seats where the GOP is still behind but has been closing: North Carolina and New Hampshire. Particularly with some new polling dropping in New Hampshire (not yet reflected in the RCP averages here), I’ll hold off until next time on a deeper dive into those polls.