Diary

DJ Drummond is Great at Dissecting Polls

I have spent a great deal of time analyzing faulty procedures in the national political polls for the presidential race this year. I have noted proofs of invalidity and the significance of party affiliation weighting in the published claims of such polls. But a number of people have observed that Barack Obama is also leading in a number of state polls, and they quite reasonably ask if those polls are valid or not. The chief difficulties in the state polls, are the facts that there are so many polling groups performing state polling (Real Clear Politics has reports from 49 different polling groups on state polls for the presidential race), there are different demographic norms for each state, and whatever is skewing the national polls is likely to be influencing reported state results as well. But while the past does not guarantee the future, I did find some interesting points in the 2004 state polls.

Not every polling group which is publishing polls this year also polled and published in 2004. And many of the polling groups are limited to only a few states, or just one. The Field Poll, for example, is only concerned with California. So I limited my analysis to nine major polling groups which conducted state polls in a large number of states in 2004. I was looking for polls which released polls just prior to the election, and which also released polls during the first two weeks of October of 2004. The results of that examination are presented here. I hope you will find them illuminating.

CNN conducted polls in 6 states just before the 2004 election. Of those 6 polls, in 4 state polls the published support for Bush was off by 3 points or more, as many as 5 points wrong. In 2 states Bush’s support was over-estimated, in 4 states his support was under-estimated. In 4 state polls the published support for Kerry was off by 3 points or more, as many as 6 points wrong. In 3 states Kerry’s support was over-estimated, in 3 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 5 times, off by 5 points or more in 5 states.

CNN conducted polls in 11 states during the first two weeks of October 2004. of those 11 polls, in 4 states support for Bush was off by 3 points or more, as many as 4 points wrong. In 2 states Bush’s support was over-estimated, in 7 states his support was under-estimated, In 6 state polls the published support for Kerry was off by 3 points or more, as many as 6 points wrong. In 2 states Kerry’s support was over-estimated, in 9 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 9 times, off by 5 points or more in 3 states.

http://tinyurl.com/4cuxtq