Pollsters lament confusion over 'margin of error'
If there's one thing reporters do that drives professional pollsters crazy, it's our often ham-fisted use of a poll's "margin of error."
"Nobody really uses it right. It's the most overused and underused critique," said Mark Mellman of national polling firm The Mellman Group.
He said journalists writing about poll results often say races are tied if a poll's margin of error is 5 percent and one candidate has a 5-point lead.
"That's not really true," he said. "The margin of error is a likelihood.
"The single most likely thing is they're five points ahead. But more than likely, they're ahead. The likelihood they're tied ... that's pretty low."
That's because margins of error work in both directions.
Here's a hypothetical example:
John Smith is running for Possum Hollow dog catcher. A recent poll shows 50 percent of voters plan to vote for Smith over his opponent, Bob Jones.
The same poll, which has a 5 percent margin of error, found Jones has 45 percent support among voters. Does that mean the men are tied?
No, it does not, because margins of error affect underdogs, too.
With a 5 percent margin of error, Smith might have as much as 55 percent support among voters, or as little as 45 percent.
Jones, meanwhile, could have as much as 50 percent support, or as little as 40 percent.
That means the gap between the men could be as wide as 15 points, or it could be a dead heat. But more than likely, Smith and Jones are five points apart.
Mellman said journalists bungle "margin of error" in other ways, too.
He said one poll might show a candidate has a 56 percent approval rating, but another poll conducted a month later might show that approval rating has risen to 59 percent.
If those polls have 3 percent margins of error, the approval rating might not have changed at all.
"It's not necessarily anything," he said.