Political Animal


April 17, 2012 8:45 AM High Polling Season Begins

By Ed Kilgore

If you look at aggregator sites that show what gabbers are gabbing about today, there will naturally be lots of “Tax Day” stuff, some of it prefab fluff and partisan spin, some of it actually useful (viz. this set of graphs from WaPo comparing top marginal tax rates since the 1930s with econonomic growth indicators, which pretty much makes a mockery of supply-side economics).

But the other buzzy topic this morning involves general election polls, with many conservatives bellowing in excitement at the first official Gallup Tracking Poll for the general, showing Mitt Romney with a two-point lead, and many liberals responding by pointing at a new CNN/ORC poll showing Obama up by nine points.

So it’s the perfect time for Nate Silver of FiveThirtyEight to issue one of his excellent reminders about how to read general election polls, and the common mistakes to avoid.

Nate offers twelve separate points of advice, but I’d emhasize and amplify five: (1) particularly this far out from the election, polling averages and internal trend lines are the most reliable indicators; (2) it’s helpful to understand “house effects,” the methodogical habits of individual pollsters that tend to produce different results—but that doesn’t mean just ignoring or throwing out data from pollsters whose “house effects” tend to produce top lines you don’t like (e.g., Rasmussen!); again, polling averages will sort all that out; (3) crosstabs providing demographic breakdowns are fascinating and sometimes useful, but be aware they typically involve very small samples and very high margins-of-error; (4) the closer we get to election day, the more it becomes important to distinguish between polls testing “likely voters” and those with less selective samples (e.g., registered voters, all adults, etc.), and the more assumptions about likelihood to vote will be worth arguing about. I’ll directly quote Nate on a fifth point, because I want to return to it later today:

Don’t over-learn the lessons of history. A final and more general point is that there have been only 16 presidential elections since World War II. That simply isn’t a lot of data, and overly specific conclusions from them, like “no recent president has been re-elected with an unemployment rate over 8.0 percent” or “no recent incumbent has lost when he did not face a primary challenge,” are often not very meaningful in practice and will generally not carry much predictive weight.

It’s tempting, of course, to ignore all polls as just so much noise and spin, or even as actively evil influences on U.S. politics. I’m in the camp of those who think the problem with excessive reliance on polls is usually one of exaggerating limited or bad information, and that the answer is more and better information, along with the perspective no public opinion survey can supply.

Ed Kilgore is a contributing writer to the Washington Monthly. He is managing editor for The Democratic Strategist and a senior fellow at the Progressive Policy Institute. Find him on Twitter: @ed_kilgore.


  • Hedda Peraz on April 17, 2012 9:18 AM:

    Never forget, that polls are a revenue stream.
    -And have been, ever since the oracle at Delphi interpreted chicken entrails. For money.

  • fredamae on April 17, 2012 9:25 AM:

    Polls are Only as reliable as the person who Paid for them is honest!

  • kd bart on April 17, 2012 9:35 AM:

    I tend to ignore daily tracking polls. Especially ones that average the last 3 days. One day of polling can badly distort this type of poll. I can recall Gallup's Daily Tracking poll from 2000 having a swing of 12-15 points in a matter of days without much of anything happening in the news. Just a case of one night's strong polling for one candidate replacing a night of strong polling for the other candidate in the aggregate.

  • AndThenThere'sThat on April 17, 2012 9:40 AM:

    Off thread but I had to share this laugh out loud headline of the day.

    Ann Romney defends husband Mitt Romney for strapping dog to roof of car: Pooch ‘loved’ it


  • stormskies on April 17, 2012 9:43 AM:

    The Gallup Poll is the most corrupt of them all. Simply reflect on the fact that Obama leads buffoon Romney by at least 20 points with women, at least 80 points with the Hispanics and Latinos, 90 points with African-Americans, and is tied with while males. Given those actual facts how is it possible for buffoon Romney to be ahead by any margin at all ? It's not. Total fucking corruption equaling propaganda is what this is by Gallup.

  • SadOldVet on April 17, 2012 9:50 AM:

    Forecasting is difficult. Especially about the future.

    Which is why we should take all of the polls with a great deal of scepticism and get out and work to make sure that Romney is not elected.

  • Ron Byers on April 17, 2012 9:54 AM:

    SadOldVet and I agree.

    By the way great spring training post Ed. This should help tune us up for the regular season.

  • T2 on April 17, 2012 10:03 AM:

    Polls are like PolitiFact....they start out "non-partisan" but eventually the money angle always comes in, and from then on it's "Who wants to pay the most" for results. The only poll that matters is the one in November, and the Supreme Court can apparently change that if it wants to.

  • Still Fighting the Same Ole Battles on April 17, 2012 10:06 AM:

    Polls are disingenuous, doing nothing more than dumping another layer of mud into the black silt-filled river of polluted politics.

    The hardest thing I've had to learn in my years is how little truth life contains. The average Joe has to make decisions on information that's been thoroughly branded®, sensationalized by pretty faces and ugly hearts, "analyzed" by "experts", coated in gilt religion or faux patriotism, and then filtered by special interests bent on pushing the masses this way and that for their own gain.

    I miss truth. Or rather I miss truth being allowed to stand without constant refutation by liars which is where we are today.

  • hornblower on April 17, 2012 10:33 AM:

    I got polled yesterday on my cell. I don't know what my demographic was but I got the sense that they were trying to push me in a certain direction but that may be just my own suspicion. Anyway, there were only three questions, how to improve the economy, which party could do it better and did I favor the President. I pressed numbers to indicate my choices.
    I guess I must have represented the old white man demo. So if you see a change in that group you will know where it came from.

  • James Conner on April 17, 2012 1:27 PM:

    The margin of error should be kept in perspective, as by convention it is set at the 95 percent confidence level, a standard for rejecting the null hypothesis in the social sciences.

    But as Kevin Drum and others have noted, what we really want to know is the probability that Smith is ahead of Jones. And it turns out that even when Smith's lead is within the margin of error, there can be a fairly high probability that Smith really is leading Jones.

    The American Research Group's website has online calculators for calculating the ballot lead probability and the margin of error.