Margin of error and our polls

Laura Leslie, who does an outstanding job covering state government for WUNC, wrote an interesting post last night on her blog about our polls and the margin of error.

One of her key points was this:

Every matchup except one (Moore-Smith) fell within the 4.5 (times 2 remember) margin of error, which means it’s a statistical draw. McCrory and Graham got higher numbers than Moore in their match-ups against him, although with the MOE, it’s a tie all 'round.

That the races are a statistical draw is sort of, but not entirely, true.

Washington Monthly wrote an interesting piece on the issue of statistical ties during the 2004 election- I don't necessarily agree with the tone but the substance is correct:

There are two problems with this: first, 95% is not some kind of magic cutoff point, and second, the idea that the MOE represents 95% certainty is wrong anyway. A poll's MOE does represent a 95% confidence interval for each individual's percentage, but it doesn't represent a 95% confidence for the difference between the two, and that's what we're really interested in.

In fact, what we're really interested in is the probability that the difference is greater than zero — in other words, that one candidate is genuinely ahead of the other. But this probability isn't a cutoff, it's a continuum: the bigger the lead, the more likely that someone is ahead and that the result isn't just a polling fluke. So instead of lazily reporting any result within the MOE as a "tie," which is statistically wrong anyway, it would be more informative to just go ahead and tell us how probable it is that a candidate is really ahead.

So what's the bottom line? Washington Monthly prepared a table showing what the percentage chance there is that a candidate is winning based on their lead in the poll and the margin of error. Rounding up our MOE of 4.5% from this poll to 5, here are the odds:

Bev Perdue's two point lead over Pat McCrory means there's a 66% chance she's leading him. Her four point lead over Bill Graham means there's a 79% chance she's leading him. Her six point lead on Bob Orr means there's an 89% chance she's leading him. And it's higher for Fred Smith.

On the polls involving Richard Moore, his two point deficit against Bill Graham means there's a 66% chance Graham is in the lead. Same odds with Pat McCrory, who he also trails by a couple points. His margins over Fred Smith and Bob Orr are large enough to get to close to a 100% chance that he's leading them.

Anyway check out the article and the chart, it's an interesting read.

Wrapping up, to quote Kevin Drum again, “from a pure statistical standpoint, a lead is a lead and it's always better to be ahead than behind.” Even when two candidates are within the margin of error, it doesn’t mean that no one is leading.