Diary

The fault in these public opinion polls

I do not post very often (to say the least) but I do read these boards all the time. Much of the discussions that I focus on deal with these polls. I would like to address some of these arguments of how these polls could be wrong. I will try to speak in real language, because unfortunately polling has morphed into a business that is filled with statistical jargon targeted to a select few. Polls have become tools of the elite to not only project, but to dictate public opinion. How do polls dictate public opinion you ask? This could be done to suppress turnout, because after all, no one likes to go to the polls and vote for the guy (or gal) they know will lose. It’s a waste of time quite frankly especially since folks have to take off work, wait in lines, and use the gas to go to the polling place.

Let me real quick give you my credentials on polling. I am well on the way to getting my PhD and my specialty is in public opinion polling. I have submitted papers for political science journal articles on (you guessed it) on the Bradley effect. Of course in academia we do not call it the “Bradley effect” but “social desirability effects”. See what I mean about jargon? I also help run a very bar-bones polling operation for the University that I am getting my PhD. Needless to say, polling is like a sausage factory. In other words, the only folks that do not each sausage are those who work in the sausage factory. After working in polling, I am less trustful of polls than many if not most of you. I see how dirty they are, and all that can go wrong.

After all, all polling is – an imperfect tool to make a estimation (or guess) on what the public thinks or feels. This is why I get so frustrated with the absolute certainty pundits speak on these polls. I do not tell you this to toot my own horn, because I do not look at myself as any better than you or anyone. It just so happens that this subject is my “expertise” just like a car mechanic working on my beaten down, gas guzzling, SUV that I hope to get rid of soon.

And though all these polls show good news for Obama, they are full of holes which communicate to me there is something else going on here. First, let me tell what these holes are and then I will tell you why this might be. The glaring hole of these polls are in the favorability/unfavorability. Well, generally in a perfect polling world, the maximum amount of votes someone can get (percentage wise) is one-hundred percent minus the their unfavorable ratings. In other words, if 42% of the folks have an unfavorable view of McCain, that you know McCain can only get 58% of the votes. Same for Obama. So when we look at the average fav/unfav ratings, we know that McCain could absolute stomp Obama on election day, vice versa, and somewhere in between. Typically in all presidential elections the horse racing numbers converge on the fav/unfav ratings. In other words, the electorate gets to the point where they like their guy and do not like the other guy. In 04, both Kerry and Bush had fav’s/unfav’s of about 49-49. That meant the election distribution would be 51-49 for no matter who one. This has been the same break-down for just about every election from here on out.

This is the first election in my life where the unfav’s for both candidates are both in the low 40’s. It defies logic, and because the horse-racing numbers are not converging on the favs/unfavs – my spidey senses are going through the roof. This tells me that of the people polled, there is a sizeable group of folks that like both candidates? I don’t buy it considering the saturation of negativity about both candidates in the information market (the MSM and their hate-fest towards McCain/Palin, and the blogs and talk radio for Obama).

Now that I have shown where the holes in the polls are let me tell you how this could be. This observation in the polls tell me there is bias and now I would like to discuss how this bias occurs (now, I am not predicting who will win this election at all …. I will let you decide that for yourselves). First, I would like to discuss three potential methodological errors in how these polls are conducted. Any one of them could create bias and if all three are in play, then these polls aren’t worth the paper they are presented on. The three biases are “Bradley effect” – or social desirability effect, something we call “self-selection” bias, and framing/priming effects (which is generally rectified with polling weights – which themselves inject more bias).

The Bradley Effect ….

If McCain wins, this reason will be given most credence unfortunately. In other words, if McCain shocks the world the media, pundits, and many academics will simply say America is racist, stupid, etc. The Bradley effect is not just simply a respondent (the person taking the poll) is trying to hide their bigotry. Anyone doing a CATI poll (caller-assisted-telephone-interview) is subject to social desirability effects. Imagine if you got a polling question about student loans for immigrants (something you might not have an opinion one way or the other about). The person calling you has a beautiful Eastern European accent. In the presence of that accent, you probably would give an answer that you feel the interviewer would want to hear. This doesn’t mean you are a bigot, but it is a simple reality that people change, adjust, or instantly create opinions in the presence of others that you would not have already had. So, if the interviewer identifies themselves as a Vietnam vet, you might be nicer to McCain. If the interviewer is African-American, or a female, you might be more favorable to Obama or Hillary. Many polling people are young and unprofessionalized in many ways. This is not their career but a job. So most interviewers are younger. We can realistically tell a younger voice from an older voice. And if they are younger person that is a female, or a younger minority (something that most folks can sense in a voice) you might alter your opinion in the presence of that person (even if it is on the telephone). So, if you say you are a Registered GOP and a GOP outfit calls you and asks who you vote for, knowing the person on the other line is more than likely a Republican if you intend to vote for Obama you might hide it. But you would say that you like “both candidates”, and leave it at that – because you have nothing to lose because after all you like everyone. So the “Bradley effect” could bias against McCain under the right conditions.

We know, though, this is not likely and if the “Bradley effect” is biasing someone it is Obama. Pollsters assume this “bias” will cancel each other out in the end. But unfortunately, this particular bias is rarely equally distributed amongst ideologies. Most polling outfits are associated with liberals. Most callers are associated with liberals (because they are young and many are ethnic minorities), and considering how popular a figure Obama is you might be more inclined to say you like the guy because you are afraid that if you say you don’t like the guy the other person on the phone will assume it is because of the color of his skin rather than his policy positions. I feel this phenomenon is more prevalent in progressive states than say “Alabama”. This is why I think the polls in California, Minnesota, and other places are way off. In fact, I think California will be very close (not saying McCain will win California, but he will finish closer than most are thinking). Of course, if McCain senses this and goes to California, Obama and his endless supply of cash will hit California very very hard. So McCain’s best strategy is just to stay away from California and perhaps things will break his way. Plus, there are two very socially conservative propositions on the California ballet that might surge social conservatives. In addition, blue-collar Democrats from Ohio, Pennsylvania, etc. might also be susceptible to these effects. Keep in mind that Obama over-polled Hillary in most of these polls by five to seven points during the DNC primary. Now Obama was under polled in rural GOP dominated states as a result of this effect as well … so it can cut both ways. So is there a sizeable group of Democrats and Independents that do not want to look racist (after all, they should vote for Obama because he is supposedly their guy, and Obama is such a popular guy) in these areas? My inclination is yes there is. I do not know how big it is, but this will effect the polling numbers. I tend to believe this effect will be larger in blue states. So, if Obama takes a traditional red-state, McCain could very well off-set it with a traditional blue state and no one would have seen it coming. Again, if you are a election junkie, I just have to say keep a close eye on California, Oregon, Minnesota, and a few others. The MSM has avoided these states for the most part during this election season. Take a look at traditional blue states that Hillary mopped the floor with on Obama, that have more or less been ignored by the MSM, and then you will get a sense of where a few surprises might be.

Self-Selection bias.

Imagine a world in which a poll is done, that a group of people have an incentive to take the poll. If I did a poll asking “Would you like to see more NFL games on TV” (I know they are all televised, but this is just simplistic example), people that would respond to this poll would more than likely be NFL fans. You might ask, how would we know if a poll on the NFL is being conducted when the phone is ringing. Well, all polling firms state their purpose before the poll begins. So the caller would say “I am calling regarding a survey about the NFL”. Most people that don’t give a flip about the NFL would decline to take the survey. Likewise, hardcore NFL fans would take the survey. So, the end numbers would be skewed towards NFL fans, and the numbers would more than likely show that people want to see more NFL games etc. In other words, these numbers would not be a sample of the American public, but rather, the American public that just so happen to be NFL fans.

So, do these same conditions apply currently. I think they do. There are a lot of luke-warm political followers that are sick of the election and more than likely decline to do polling on the election because they are not enthusiastic about either people. So, you have to ask yourself, do these people vote? Yes, a lot of them do vote because if they do not, the ghost of their parents and grandparents will haunt them in their dreams. Who would they likely to vote for? That is the magic question, and one could argue that these people will eventually cancel themselves out. Again, though, I see hopeful signs for McCain. Many McCain supporters are going to vote for him because they respect him, but they aren’t enthusiastic about him. They would likely to turn down doing a poll. Obama supporters would jump at the chance to do a poll. So you might be asking, does enthusiasm work against a candidate? I would say No it does not, but it does misrepresent what the final numbers will look like. Remember, Clinton was projected to win by 14 points but only won by seven. Now, Dick Morris says this is because in the last four days it was because the Dole campaign started hitting a campaign subject harder and it resonated amongst the voters. The problem is Morris is assuming that information actually made it way down to the voters. I do not know about you, but most folks I talk to avoid political election information like the plague this late in the game. They are sick of it.

So are there a sizeable group of people that will vote for McCain, who don’t want to look like racist, who are Democrat or Independent that avoid polling and will hold their nose and vote for McCain? The conditions exist for such a thing to happen. If it will happen, I will tell you on November 5th. Is there also a sizeable population that are conservative who respect but do not like McCain that will vote for him on the basis that Obama makes them very very nervous with his very liberal ideology? Again, I will tell you for sure after election day. But the conditions for such a thing happening exist. Again, some of these people will not vote, but most will vote out of civic duty. Then again, many will vote for Bob Barr, but those who really fear Obama’s policies will opt for McCain because they will not want to waste their vote. Can you imagine a lot of people in Ohio, Pennsylvania, etc. that might fall in this category? By looking at the poll numbers how do we know if there is “self-selection” bias? Simply by looking at the response rates. If about 40% of people polled actually fill out the survey, then this tells us the numbers are pretty good. If, however, only 20% of the people polled actually answer the survey this means that there is about 20% of folks that would have allowed themselves to be polled a few years ago but are avoiding polls today. This is a fifth of the electorate who are hiding their opinions for whatever reason. Perhaps they are afraid of the pressure of not looking like a racist, or are down-right frustrated, but they will vote. And considering turnout will be a lot higher than in previous elections, they will not sit this one out. Ask yourself, does this condition of “self-selection” bias Obama or McCain? My opinion is that there is a stigma against McCain (real or imagined) that someone who is not necessarily enthusiastic about voting for McCain (but will anyways) would lead them just to not answer the phone and allowed themselves to be polled. After all, considering the positive buzz around Obama, why would an Obama supporter not perform the survey? There might be some reason, but in my opinion (and this is just an opinion), luke-warm McCain supporters (especially independent and Democrats) which just likely avoid being polled.

Just a note, most polling firms do not (in their reports) report “completion percentage”. This is the actual percentage of calls made to how many people allow themselves to be polled. There is a very big reason as to why. Because most other polling firms would be quick to shoot their competition down if they presented these numbers.

Question wording

Finally, lets discuss question wording. This is what we call framing/priming effects in polling. Most questions are randomized to off-set these effects. But, of the polls I’ve read many of the questions in them are poorly presented, and an example is that many of these polls ask people their party affiliation. What they do not ask is whether or not someone feels a closer association to one party or the other. So if a poll shows that McCain has an unusual high support among Democrats (or Obama with Republicans) these polling firms will apply weights to these polls to “fix” them. These weights cause a lot of bias. So Democrats that say “yea” for McCain aren’t taken as seriously and are not entirely dropped, but only of portion of them are included in the survey. And because the American public is registered Democrat in higher numbers, these weights historically bias against the Republican. There are a lot of old Southern and Mid-Western Democrats that are more conservative than most Republicans who vote more adamantly for the GOP than for the left (unless the Democrat on the ticket is more conservative than the Republican … which still is possible in the South and Mid-West). In this election, however, McCain is clearly more conservative.

It is not by chance that the GOP have won 7 of the last 10 presidential elections, and of the three “victories” for the Dems in the presidential election they were Southern (LBJ, Carter, and Clinton). In addition, these three candidates appeared to be conservative Democrats as well (even Carter who was a Vet and a born-again Christian- so he was good on social values though he was considered progressive). America is a right of center for the most part when it comes to presidential politics. Yet, there are more registered Democrats. Quite frankly, the cross-over Democrats aren’t taken as seriously today and the view of the electorate in partisan categories by the pundits is extremely superficial.

OK, if you are still with me, this is where we part. I am not saying McCain will win. What I am saying the polls are inaccurate. So if you are a McCain person, don’t pay attention to them. If you are an Obama person, don’t start counting those chickens. And whatever you do, don’t listen to exit polls because they are worse than public-opinion polls and are susceptible to even more bias with how they are conducted (see 04). If you have read this entire post, I hope it helps you get around some of the discrepancies you might be hearing from family and friends and yet not seeing in the media.