Political Animal

Blog

October 16, 2012 9:43 AM Poll Spin Madness

By Ed Kilgore

With the polls coming at us more rapidly every day, each greeted with a blare of trumpets from one camp or the other, it’s very helpful to keep a few interpretive rules in mind, particularly one the spinning starts. One is to stay focused on poll averages rather than individual results, and avoid the temptation of excluding numbers (or ignoring pollsters) you don’t like. Another is to look at the trends as reflected in a given firm’s results over time, which will probably be more reliable than the absolute numbers. Still another is to be alert for sketchy or unconventional methodologies.

But the rule that’s probably easiest to forget is to pay attention to sample sizes. Here’s a pertinent warning from just last night by Nate Silver about the “battleground state” subsamples we are beginning to see regularly a day or so after national polls come out:

Monday’s Washington Post poll had Mr. Obama performing better in what it termed swing states than in the country as a whole; the Gallup poll showed just the opposite.
This data is largely useless. A typical national poll might interview 1,000 people, of which perhaps 250 or 300 will live in swing states, depending on exactly how it defines them.
The margin of error on a 250- or 300-person subsample is enormous: about plus or minus six percentage points. (The swing state sample from the Gallup poll was somewhat larger, but still small as compared to the 3,000 or so voters that it interviews for each instance of its national tracking poll.)
In contrast, in the state polls, there are often tens of thousands of people interviewed in polls of battleground states on a given day. (There were about 2,500 on Monday, for example, despite its having a relatively low volume of state polling.)
There is just no reason at all to care about what 250 or 300 people say when you can look at what 2,500 or 3,000 do instead….

And then there’s the whole thorny issue of how you define “battleground states” to begin with, and how you keep Florida’s big numbers from skewing everything.

In any event, poll findings involving small subsamples can be seductive, giving pollsters second-day coverage of their surveys and leading partisans to see big swings in key demographics that seem to explain everything about national trends. But in many cases, you might as well be just making up a nice fantasy to soothe yourself to sleep.

Ed Kilgore is a contributing writer to the Washington Monthly. He is managing editor for The Democratic Strategist and a senior fellow at the Progressive Policy Institute. Find him on Twitter: @ed_kilgore.

Comments

  • c u n d gulag on October 16, 2012 10:15 AM:

    Here's what I wish - that the polls were only used by the parties and their candidates.
    They should be for internal use only, and not released to the public - and certainly NOT to the MSM!

    When the MSM has access to them, all the people in the media do is talk about these fluctuating numbers to the public, instead of talking to the public about the candidates policies.

    Numbers from polls are like candy - cheap, quick to eat, and easy to digest. But have little or no nutriotional value.

    But, like lobbyists of old, and the new Super PAC's, all of these new polling companies are part of what Stephen Colbert called "The Political Industrial Complex."

    More and more ways to make money for more and more grifters who see "GOLD IN THEM THAR ELECTIONS!"

  • beejeez on October 16, 2012 10:46 AM:

    So I guess we're all thrilled that Obama could win the electoral college and lose the popular vote, right?

  • sjw on October 16, 2012 10:53 AM:

    To stay sane and to keep my hysteria in check, I'm checking in frequently with Nate Silver and Sam Wang: both still have a slim Obama lead. In other words, where we were in August.

    While I understand that Obama's post-debate balloon had started to deflate even before the debate, it remains that the debate was the reason the balloon exploded. That Team Obama could be so blase or deluded in their debate prep is beyond me, and I remain really pissed off at them. They'd better get this right tonight.

  • Karen on October 16, 2012 10:59 AM:

    We need to work in red states. I live in Texas, which once upon a time elected actual progressives. Remember Ann Richards? Ann's biggest error was unfailing to create a party and policy structure that could survive her, and now we are the only big state that the R's can count on. Texas cites all vote D, except Ft Worth. Why doesn't some superPAC work with us to expand on that? We are a majority non white state where the non-white vote makes no difference. That should not be. Flip Texas and suddenly the R's are out of the White House for a ver, very long time. Help us, your lives depend on it.

  • Karen on October 16, 2012 11:05 AM:

    We need to work in red states. I live in Texas, which once upon a time elected actual progressives. Remember Ann Richards? Ann's biggest error was unfailing to create a party and policy structure that could survive her, and now we are the only big state that the R's can count on. Texas cites all vote D, except Ft Worth. Why doesn't some superPAC work with us to expand on that? We are a majority non white state where the non-white vote makes no difference. That should not be. Flip Texas and suddenly the R's are out of the White House for a ver, very long time. Help us, your lives depend on it.

  • Lucia on October 16, 2012 11:47 AM:

    I take the message of this post to be that lefties shouldn't freak out too much that Romney has now taken a lead in swing states according to Gallup. That's as may be, but on the flip side Nate Cohn warns us not to count too much on Ohio if Obama's lead there is only a point or two. Polls can easily be wrong by that much. And right now, if Obama loses Ohio we all lose.

    I too wish the media would talk less about polling. It's generally agreed that polling news itself affects the polls, because everyone loves a winner and the loser's supporters figure it doesn't matter what they do. That's why Romney supporters kept screaming about skewed polls.