Polls have acquired a life of their own in political commentary. They are quoted relentlessly to make cases about what will happen in the future. So I thought it would be fun to take a quick tour of Super Tuesday polls, particularly the polling averages (for some reason a lot of people believe that if you average a lot of bulls*** numbers you get a real number, go figure), and compare them to actual results. The results don’t give one great confidence in political polling at the state level. And, quite honestly, if you can’t get the number right is a state, particularly a smallish state, you can’t get the number right nationally.

Alabama

Polls

Actual result

Verdict

Solid polling. It got the order of finish correct and was close to the actual numbers of the top three. What is impressive is that the numbers of Kasich and Carson were really, really close. Especially in Kasich’s case where his support was virtually within margin of error.

Alaska

Polls

Actual result

Verdict

So much for Sarah Palin’s home state influence, right? This poll got the winner wrong, and not by a little. It was off by 12 points. It missed Rubio by over 100%.

Arkansas

Polls

Actual result

Verdict

Not all that bad on the winner and it got the order of finishing correct. But it did miss Cruz+Rubio by a combined 10 points, which is pretty significant.

Georgia

Polls

Actual result

Verdict

Not bad. The averages shorted Cruz by 5 points and Rubio by 3 points.

Massachusetts

Polls

Actual result

Verdict

On the money. It understated both Kasich and Carson, within the margin of error, but was otherwise accurate.

Minnesota

Polls

Actual result

Verdict

Garbage. There is only one poll here worth considering, the Mason-Dixon/Star Tribune poll. It gets the order of finishing correct, but simply by luck. All three of the top finishers are clustered within the margin of error. In reality, the race wasn’t close at all among the top three.

Okalahoma

Polls

Actual result

Verdict

This might qualify as the worst set of polls of the evening… or maybe ever. It got the winner wrong. It got the order of finishing wrong. It missed Ted Cruz’s win and was 15 points off. It doesn’t get much more wrong than that.

Tennessee

Polls

Actual result

Verdict

There weren’t enough polls for real averaging. Of the two most recent polls, you can see that NBC/WSJ/Marist was damned close, missing Cruz and Rubio at the upward end of the margin of error. SurveyMonkey could have saved a lot of money by simply making up random numbers and calling it a poll.

Texas

Polls

Actual result

Verdict

You would have gotten the same or better results interviewing an collection of random vagrants, hobos, and skid-row denizens. The poll averages understated Ted Cruz’s support by nearly 10 points and over stated Kasich and Carson support by nearly a factor of two. They got the order of finishing correct, but that was a given. It also hints that the polls artificially created the narrative of Trump catching Cruz. For instance, the Emerson College poll from Sunday shows a 3 point Cruz lead, you can’t reconcile that result with any answer other than deliberately massaging the numbers to create buzz.

Vermont

Polls

Actual result

Verdict

Well, they got the winner right. Beyond that it is hard to understand how they missed a 16 point Kasich surge and got Rubio wrong outside the margin of error.

Virginia

Polls

Actual result

Verdict

Meh. Not all that great. The poll averages overstated Trump’s support by 4 points and understated Rubio’s by 7. It got the order of finish correct and shows that the Rubio campaign’s enthusiasm about taking Virginia was well-founded. It got Cruz, Kasich, and Carson right but it failed to see how close the race was.

Summary

Not all the polls are bad. A handful are quite good. Collectively, though, they show that polling averages cannot fix bad polls and that there are many, many more bad polls than good ones in circulation. What most have in common is this: they overstate Donald Trump’s support by about 4-5 points and they understate Ted Cruz’s support by about 3 points. The way things are going that may not matter, but, knowing that is something that we all need to keep in mind going forward so we have a better understanding of what the actual state of play is.