November 06, 2012

A Note About Exit Polls & Why They Will Be Wrong

Just a quick reminder from Ace as I look at Drudge's hilarious headline:

In 2004, Kerry was picking out new curtains for the Oval Office because the exit polls showed him destroying George Bush.....

The exit polls had a massive Democratic skew for several reasons. The people conducting the polls were young college students, for example. Older voters avoided them (figuring they would look down on their vote) and they, in turn, avoided older voters (because older voters are square, man!).

So, take that into consideration as the polls leak.

Also, the polling companies claim to have self corrected after the 2004 fiasco. They do so by trying to force the numbers to make them fit the prediction model.

In other words, the exit polls will most likely have the same Democratic bias that the pre-election polls had.

Think of it this way. In 2004 the polls were skewed toward John Kerry because the polling firms hire college students to do the surveys, and those college students selected people for surveying that they could relate to: people who looked like them, Democrats.

I talked about this in 2004 because as a political science professor I was contacted by a polling firm to help them recruit students to do exit surveys. In 1998, as a poli-sci student, I was also asked by one of my professors to do the same thing.

In 2004 the Democratic bias was because the so-called "randomness" (which should, if it was truly random, weed out bias) of the survey was actually skewed by those implementing the survey.

In 2008 the polling firms self-corrected by taking surveys filled out by exit voters, and then weighting them according to the model of the expected vote. In other words, the pollsters intentionally manipulated the real exit data in order to conform to the expected data.

This worked in 2008 because the expectation that Obama would win (the predictive model) conformed to what actually happened.

But in 2012 you'd have to be nuts if you think that, say, a D+5 predictive model would actually conform to the numbers on the ground. And since I'm guessing that these models are very similar to the way the pre-election polls predicted "likely voters", then that's exactly what you'll have.

If the exit polls show, say, an R+1 advantage, then the number crunchers will forcibly weight the sample so that it conforms to within (I'm assuming here because this is how I would do it if I were making the model) two standard deviations of the D+5 model.

And since the only other way to do the exit polling would be to go back to random sampling, then you are back to the same problem we had in 2004: self selection.

In other words, given the bias we've seen in the pre-election polls there is no reason to think that the exit polls won't have the same bias.

Don't panic if you see early exit polls showing an Obama lead. If the research design of those polls is anything like the research design of the pre-election polls, then they are absolutely meaningless.

UPDATE: Exactly as I expected, the Ohio exit poll has a D+7 skew. Which means it will be utterly worthless. So, whether they are using the old "random" method or the new and approved "weighted method", they're probably going to get it wrong.

By Rusty Shackleford, Ph.D. at 07:09 PM | Comments |