Background: one of the toughest decisions PPP had to make in conducting its polls last year was whether to weight for party id. Ultimately we decided not to. The main reason was that we really weren't sure, if we did weight for party id, what numbers we should be using. Should we use past exit polls? The most recent Gallup data? The most recent Pew data? What if party id shifted back and forth over the course of the year in relation to various events? Weighting to an out of date or incorrect party id target in a particular state could foul up an otherwise well conducted poll. Ultimately we decided just to weight for gender, race, and age and let party id fall where it fell and given that a Wall Street Journal analysis of swing state pollsters soon after the election showed us as one of the two most accurate in the country I don't regret it.
It was a decision we took some flak for, particularly from Republicans, because the party id breakdowns our polls found tended to be much more Democratic than what the exit polls showed even for the very Democratic year of 2006. I have a feeling Democrats in the next few years may use the Gallup data to criticize our polls for being too Republican in their party breakdowns.
Our polls in Indiania, Missouri, Florida, North Carolina, Ohio, Virginia, Pennsylvania, New Mexico, Oregon, Colorado, and West Virginia last year all came within two points of the correct margin so those seem like a good launching point to look at how the party id we found differs from Gallup and from the exit polls:
| State | PPP Party ID | | Exit Poll Party ID |
| | R+1 | D+9 | R+5 |
| | D+3 | D+11 | D+6 |
| | D+3 | D+9 | D+3 |
| | D+11 | D+11 | D+11 |
| | D+6 | D+18 | D+6 |
| | D+5 | D+9 | D+6 |
| | D+9 | D+16 | D+7 |
| | D+17 | D+14 | D+16 |
| | D+14 | D+17 | D+11 |
| | D+3 | D+11 | R+1 |
| | D+10 | D+19 | D+14 |
Our polls in these states were pretty much right. The exit polls get to adjust their numbers to fit the final outcome so they're certainly right. And Gallup's research was very rigorous. Nevertheless there are major disparities in the party id breakdown the three organizations find in every state except North Carolina, Virginia, and New Mexico. Those differences are particularly large in states like Indiana, where there was a 14 point difference between the findings of Gallup and the exit polls, Ohio, where there was a 12 point difference between Gallup and what both PPP and the exit polls found, and Colorado, where again there was a 12 point difference between Gallup and the exit polls.
Nate Silver has correctly pointed to a few of the reasons the Gallup numbers are so Democratic. While the exit polls were surveying actual voters and we were surveying likely voters, they were just talking to adults in general. Also, they prodded independents to say they leaned toward a particular party, which in last year's political climate meant more respondents identifying as Democrats. Still, those small differences in methodology don't explain some of the huge differences the organizations here found for party id in particular states.
I don't think any of the companies here is necessarily right or wrong- the truth is that when it comes to party id, particularly at the state level, there is no 'correct' black and white answer. And even if there was, it might be different six months from now.
No comments:
Post a Comment