If there is one thing that I have found to be an absolute truism as a pollster in this year's election, it's that Barack Obama's supporters are harder to get in touch with than John McCain's and Hillary Clinton's.
During the early stages of the primary we released a lot of one night, one shot polls and consistently underestimated Obama by a whole heck of a lot (even if we still did better than a lot of our colleagues in places like South Carolina.)
So we started doing longer field periods and more callbacks starting with Wisconsin, and our polls were pretty much on the mark for the rest of the primary season (with of course one exception.)
This trend has continued in the general election. We conducted half the interviews for our Florida poll this week last Thursday night. The first night McCain led by 2. The data based on callbacks over the next three days gave Obama a 4 point lead.
It was a similar story in Colorado two weeks ago. The first night Obama led by 5. The rest of the field period he led by 15.
I don't think there was movement in Obama's direction in subsequent days either of those times. It's simply a matter that you have to try more times to get his supporters on the phone.
Which leads me to today's Quinnipiac Ohio poll showing Obama up by 14. That's a big difference from Rasmussen and Mason Dixon's polls earlier in the week that showed McCain winning, and I think part of the discrepancy can be traced back to the length of field time. Rasmussen's was a one night poll. Mason Dixon's was a two day poll. They couldn't have done that many, or any, callbacks to get folks on the line who weren't at home on the first shot. Quinnipiac was in the field for six days, and I'm guessing they managed to get a lot more of those elusive Obama supporters polled when they tried for the third or fourth time.
Another thing to consider when looking at polls is the number of respondents- not just whether it's a large or small number, but whether it's a nice even number or not. If it's a nice even number a lot of the time that means a pollster is just getting a set number of interviews and then stopping. That may have the potential to undercount support for Obama, again because his folks are harder to find. If it's a random number, that's an indication that folks have their sample and they've decided they're going to call everyone in it 'x' number of times and try to get as many respondents as they can get.
We're not going to be putting out Quinnipiac sort of numbers on Ohio this weekend: more like Suffolk numbers. But from my experience anyway the issue of field periods and round numbers may help to explain at least some of the 16 point difference you see between Rasmussen and Quinnipiac.
Thanks for the perspective, Tom. I didn't even think about the fact that pollsters would keep trying to contact the same house rather than just call a different number. So you have a list of 700 people that you absolutely know ahead of time you are going to contact, and keep trying until you contact all of them?
ReplyDeleteTom, thanks a lot for the behind-the-scene look at the polling process :-)
ReplyDeleteSo a Suffolk like number (O+9) out of OH would still be a great one!
Hi Tom, What do you think about the early voting? Make a article about that please.
ReplyDeletehttp://elections.gmu.edu/early_vote_2008.html
766.000 in NC have already cast their ballots. It's huge.
Party
Dem 56.3%
Rep 27.1%
None 16.6%
Age
18-29 11.0%
30-44 18.0%
45-64 41.3%
65+ 29.7%
Race
White 67.4%
Black 28.8%
Other 3.8%
Sex
Men 42.8%
Women 56.3%
Ballot
Absentee 12.0%
One-Stop 88.0%
Hmmmm, that's a pretty low number for voters aged 18-29, actually. Other than that, great numbers.
ReplyDeleteThank you so much for this posting. After looking through both the Q Findings and the Battleground polling, especially Indiana, I'd developed a hypothesis that McCain voters are simply not responding anymore to pollsters somehow, otherwise how do we explain such rapid motion across the board. I had though it may be that they're despondent and simply hanging up, yet they'll still vote. My GOP friends won't even talk about the election anymore, they're so depressed about it, but I would be money they will vote no matter what. This response bias makes me worried in looking at polling results like Indiana +10. It makes you wonder if that sample isn't somehow tainted, and if Indiana is wrong, then that means all of those states are off too, and PA, WI are both closer, as is IA than we may know.
ReplyDeleteThanks for your explanation, it totally counters this hypothesis. Though one still has to wonder how Indiana came to poll +10O, bad weighting?
As always great analysis, Tom.
ReplyDeleteSteven,
ReplyDeleteI looked at the Big 10 polls in question. Weighting didn't seem to be an issue - what was the issue was that their methodology was poor in that they asked all sorts of questions before the presidential race - opinion on Bush, what direction the country is going in, what their most important issue was, state of the economy, etc. A lot of these questions would really skew the results - favored methodology should be to ask the presidential question immediately, as they would in the actual election.
Of course, results like this do suggest that it would be a very effective tactic to have volunteers standing outside polling places on Election day holding up signs asking, "Do you think the country is going in the right track or the wrong track?"
Well if they are harder to find(which I don't believe anyway), what makes you think they will show up to vote?
ReplyDeleteI'm wondering what the probability is that some of those being called were either in a line waiting to early vote (and impossible to reach if they didn't have a cell phone or if the poll only called their land-line #) or involved in a GOTV effort, and thus away from their cells or land-lines.
ReplyDeleteThis effect would be greatest, of course, in an early voting state with long lines.
I would suspect this might knock off a couple % off the Obama figures...and even worse because these are "banked votes".