I've had several people ask me what I think about the Strategic Vision controversy.
I think that all companies doing public polling should make their topline demographics and crosstabs easily publicly available. It should be right there on your website. I remember when I was in high school all of my math textbooks had the answers to all of the odd numbered questions in the back. When you did your homework you could certainly look back there and get the answers right but you didn't get any credit if you didn't show your work.
I could leave PPP, start Tom Jensen Polling, put out a bunch of topline numbers the day before an election that just copied the Pollster, RCP, or Nate Silver predictions and be one of the most accurate pollsters in the country. That would be pretty darn easy and anyone could do it. And that's why public pollsters should hold themselves to a higher standard and also be held to a higher standard by the media. I don't think the press should report on any polls that they can't see the party ID breakdown for, etc.
(To be clear I'm not saying that Strategic Vision or anyone else necessarily does or has done what I'm describing above, but you could certainly get away with it if nobody ever asked you to show your work.)
I also hope that Nate Silver will do the sophisticated analysis he did to raise questions about Strategic Vision's polls for every national pollster that has released more than 50 public surveys in the last two years. If it turns out that one of us is cheating then suspicions will be raised that any of us could be cheating and a more thorough look at what's going on out there might be necessary to keep public faith in our field.
Well Tom,
ReplyDeleteStrategic Vision pretty much nailed the Iowa caucuses as it was only one of two pollsters to conclude that Edwards would finish ahead of Clinton. It also found the same 32% for Obama that Ann Selzer found (before she bombed in the Iowa general election polling by finding a final result of 54-37).
So if Strategic Vision wanted to copy everyone else, it probably would have taken the safer route in Iowa and had Clinton ahead of Edwards at least.
I didn't say Strategic Vision did that, I just said that's something pollsters could do if nobody expected them to provide any crosstabs or demographics of their sample.
ReplyDeleteI don't think it's very meaningful to get Clinton/Edwards or Edwards/Clinton. It was basically a tie (31.2%) and Clinton actually earned one more "projected" delegate.
ReplyDeleteGreat analysis as usual, Tom!
ReplyDelete"Copying everyone else" isn't what some of us suspect Strategic Vision of doing. Rather, if the results are faked, it would presumably be in the interest of helping Republicans. Other polls might have suggested Hillary would beat Edwards, but a Republican would prefer Edwards to beat Hillary (still the strongest general election candidate at that time). A projected second place showing for Edwards (and third for Clinton), if it became a self-fulfilling prophecy, would have suited Republicans to a "T." Strategic Vision's results are often out of line with other polls, and to the benefit of Republicans. I've seen them described as Rasmussen on steroids.
ReplyDeleteAnyone know why the addresses of their offices turned out to be UPS stores?
ReplyDeleteYou don't even have to imagine anything as scandalous as trying to influence an election to see the room for unethical behavior. A polling organization could quite simply cut corners for greed. They could do some, high profile polls at one standard, but fudge the low profile numbers to rip off clients. They could doctor data to hide a lack of rigor. They could even take kickbacks for good numbers. All are things they could plausibly get away with for quite some time.
ReplyDeleteI agree I'd like to see analyses of all major pollsters a la Nate Silver on Strategic Vision. I suggest starting with Rasmussen.
ReplyDelete