The polls this year are just plain messed up

I went through the polls from 2004. I looked at the RCP averages (and if they didn’t have one up, I averaged the polls from about 10/15 on to get an average). I then compared those averages to the actual results. Here is what I found…

The averages will be listed first then the actual results:

Florida: Bush by .6% > Bush by 5%

Ohio: Bush by 2.1% > Bush by 2.1%

Pennsylvania: Kerry by .9% > Kerry by 2.5%

Wisconsin: Bush by .9% > Kerry by .38%

Iowa: Bush by .3% > Bush by .67%

Minnesota: Kerry by 3.2% > Kerry by 3.48%

Michigan: Kerry by 3.5% > Kerry by 3.42%

Missouri: Bush by 4.2% > Bush by 7.2%

New Mexico: Bush by 1.4% > Bush by .79%

Nevada: Bush by 6.3% > Bush by 2.59%

Colorado: Bush by 5.2% > Bush by 4.67%

New Hampshire: Kerry by 1% > Kerry by 1.37%

Maine: Kerry by 9.5% > Kerry by 8.99%

West Virginia: Bush by 8.5% > Bush by 12.86%

Oregon: Kerry by 4.8% > Kerry by 4.16%

New Jersey: Kerry by 7% > Kerry by 6.68%

Arkansas: Bush by 6.5% > Bush by 9.76%

California: Kerry by 10.6% > Kerry by 9.95%

Alabama: Bush by 21.5% > Bush by 25.62%

Connecticut: Kerry by 10% > Kerry by 10.36%

Georgia: Bush by 13.3% > Bush by 16.6%

Illinois: Kerry by 11.5% > Kerry by 10.34%

Indiana: Bush by 19% > Bush by 20.68%

Louisiana: Bush by 13% > Bush by 14.5%

Kansas: Bush by 23.6% > Bush by 25.38%

Kentucky: Bush by 20% > Bush by 19.86%

Maryland: Kerry by 13% > Kerry by 12.98%

Massachusetts: Kerry by 25.5% > Kerry by 25.1%

Montana: Bush by 21% > Bush by 20.51%

New York: Kerry by 7% > Kerry by 6.68%

North Carolina: Bush by 7.4% > Bush by 12.44%

Oklahoma: Bush by 31.5% > Bush by 31.1%

Rhode Island: Kerry by 16.5% > Kerry by 20.75%

South Carolina: Bush by 14.5% > Bush by 17.08%

South Dakota: Bush by 14.5% > Bush by 21.47%

Tennessee: Bush by 16% > Bush by 14.53%

Texas: Bush by 22.5% > Bush by 22.8%

Utah: Bush by 45% > Bush by 45.5%

Virginia: Bush by 5.7% > Bush by 8.2%

Washington: Kerry by 4.5% > Kerry by 7.18%

OK, a few notes on this information:

1) The RCP is a pretty remarkable predictor. It predicted a total of 19 of the above races to within .5% of the actual outcome.

2) The only significant misses (more than 2%) were North Carolina, Rhode Island, South Carolina, South Dakota, Virginia, Washington, West Virginia, Florida, Pennsylvania, Missouri, Nevada, West Virginia, Arkansas, Alabama, and Georgia. That totals 16 of the races above. It missed by 5% or more on North Carolina and South Dakota.

3) The averages underestimated Bush’s numbers 28 times. The averages underestimated Kerry’s numbers 13 times.

4) In the states where the averages understimated Bush’s numbers, they did so by an average of 2.04%. Alternatively, they understimated Kerry’s numbers by an average of 1.21%.

Now, the RCP average for the National vote ended up being Bush winning by 1.5%. The actual result ended up being Bush winning by 2.46%. Again, a pretty remarkable predictor. It was off by just .96%

So, why are the polls messed up this year? Here’s why…

1) In 2004, Bush won 23 of the above mentioned 41 states by an average of 15.7%. Kerry won 18 of those states by an average of 8.47%. That all came out to Bush winning the national vote by 2.46%.

2) This year, the RCP national since 10/27 is 4.8%. That’s a swing from Bush’s winning margin of 7.26%.

3) The states polls show this picture right now… Obama winning 27 of the above states by an average of 13.5%. While McCain wins 14 of the above states by an average of 14.8%.

4) Obama’s margins are significant in very large states. In Florida, he is doing 8.5% better than Bush did in 2004, in Ohio it is 7.9%, in Pennsylvania it is 8.5%, in Michigan it is 12.08%, In Missouri, it is 7.4%, In Colorado it is 12.17%, in New Jersey it is 11.32%, in California it is 14.5%, in Illinois, it is 13.16%, in Indiana, it is 22.08%, in Maryland it is 10.02%, in New York it is 11.41%, in North Carolina it is 13.74%, in Texas it is 9.2%, in Virginia it is 15.8%, in Washington it is 9.98%.

Those are the most populous states in the country, and on average Obama is doing 11.7% better in those states than Bush ended up doing in 2004. Alternatively, the only polls showing McCain doing better in one of the above states than Bush did in 2004 is that he is doing 1.6% better in Lousianna and 2.24% better in Arkansas. That means, although I did not mention 23 of the other states listed above, Obama is doing better in the all. Add those 23 states to the 16 large ones I listed, and you can see the problem.

If Obama is currently doing at least 12% better in those states, and McCain is only doing better in two small states, how is the national poll showing that Obama is only doing 7.26% better than Bush ended up doing in 2004?

So now, the question is what are wrong this year… the national polls or the state polls?

The national polls have been closing the last few days and several show the race within 3%. The state polls have not reacted in a similiar fashion yet. There is a long held belief that state polls lag behind national polls. So, maybe that will be the case now.

Alternatively, one may think it is much harder to predict the republican v. democrat turnout state by state than it is to predict it nationally. There are a lot more insignificant pollsters doing state polls whereas we usually only get the most well known doing the national polls. It’s not unreasonable to think there are more mistakes going on in the state polls than in the national polls.

Or maybe you think all the state polls are right and all the national polls are wrong. Either way, looking back and the 2004 numbers and looking at the numbers this year, you have to come to the same conclusion…the polls are just messed up this year.