Political Animal


September 11, 2012 11:45 AM Beware Small Samples

By Ed Kilgore

Near the top of the aggregator I most use is a post from Commentary’s Jonathan Tobin claiming on the basis of a new national poll from TIPP that Obama’s support among Jews is collapsing. Seems TIPP’s crosstabs show Obama only taking 59% of the Jewish vote. That’s enough to send Tobin off to the races:

While that is still a majority it is a dramatic decline from the 78 percent of the Jewish vote he got four years ago.
Obama has a 46-44 percent lead over Romney in the TIPP poll. That means Obama is suffering from a decline in support throughout the electorate from his 2008 victory when he won 53 percent of the vote. But the president’s loss of approximately 25 percent of Jewish voters this year is not matched by a similar decline in any other demographic group. Indeed even in the unlikely event that Obama was to win almost all of the undecided voters in the survey, that would barely match Michael Dukakis’ 64 percent of Jewish votes in 1988. Far more likely is a result that would leave the president with the lowest total of Jewish votes since 1980 when Jimmy Carter received 45 percent in a three-way race with Ronald Reagan and John Anderson. While some losses in Jewish support could be put down to disillusionment with his economic policies that is shared across the board, the only conceivable explanation for this far greater than average loss of Jewish votes is the administration’s difficult relationship with Israel.

And on and on he goes with other explanations of this demographic disaster for Obama, up to and including the platform kerfuffle over language about Jerusalem as the capital of Israel. Trouble is, slicing and dicing small demographic groups in a national poll (with a total sample of 808 registered voters) is a very nearly meaningless exercise unless the pollster has over-sampled the group in question (and there’s no sign of that here). I’m guessing the margin of error for TIPP’s assessment of Jews significantly exceeds the drop in Obama support levels that Tobin is so excited about.

You’ll see a lot of this when political writers get really obsessed with polls as we approach November 6. Don’t buy it.

Ed Kilgore is a contributing writer to the Washington Monthly. He is managing editor for The Democratic Strategist and a senior fellow at the Progressive Policy Institute. Find him on Twitter: @ed_kilgore.


  • Equal Opportunity Cynic on September 11, 2012 12:06 PM:

    Basic stats should really be required material in high schools -- not that it would matter, since I don't think anyone remembers algebra or trig either.

  • David Patin on September 11, 2012 12:14 PM:

    Obama got 78% of the Jewish vote in 2008. From my informal poll of zero I suspect he will get slightly more than 80%, probably 82 - 85% range in 2012.

    Jews are still liberals. And Romney is a disaster.

  • Lifelong Dem on September 11, 2012 12:16 PM:

    You’ll see a lot of this when political writers get really obsessed with polls as we approach November 6.

    Yeah, because at this point they're only mildly obsessed with polls.

  • Censustaker on September 11, 2012 12:19 PM:

    Exactly -- even being generous, Jews represent at most 3% of US adults (best current estimate from Pew Forum on Religion and Public Life -- 1.7% of US population). The survey has 808 respondents so probably 15-25 respondents are Jewish (most likely toward the smaller end). The "margin of error" on 59% is probably close to +/- 20% which places it well within the margin of error of the 2008 results (which have their own maregin of error). The difference between 59% and 78% is probably 3-5 survey respondents!

  • bleh on September 11, 2012 12:34 PM:

    A handy rule of thumb is that, in a "sufficiently large" sample, which means at least 20 and usually 30 or more, of a population with a reasonably balanced distribution between two possibilities (say, Republican and Democrat), the "margin of error" (ie, the 95% confidence interval of the estimate of the mean) is approximately plus-or-minus one over the square root of the number in the sample. For a sample of 1,000 voters, 1 / sqrt(1000) = 0.0316 = 3.16%, ie, the "margin of error" of a poll of 1,000 is around plus-or-minus 3%.

    In this case, it would depend on the number of JEWISH voters sampled. Overall, about 2% of US citizens identify as Jews. If this sample were representative, then of the 808 total voters in the sample, about 16 of them would be Jewish. This would be too small to use the rule of thumb -- indeed, nobody would consider it a proper sample at all -- but applying it anyway yields a "margin of error" of plus-or-minus 25%.

    So yeah, he's full of it.

  • c u n d gulag on September 11, 2012 12:35 PM:

    Parsley is for food, parsing is for pundits.

  • Bobby Goren on September 11, 2012 12:37 PM:

    Methodological issues aside, all GOP spin aside, there's probably only one state where the race is tight enough for any loss in support to be influential in the race and that's Florida where the Jewish population represents about 3.5% of the total. The other major populations centers are either safe blue or red (mostly blue).

    Granted, Florida's important. But I'm still not convinced this finding is accurate anyway.

  • Walker on September 11, 2012 1:00 PM:

    The mathematical illiteracy of journalists is one of the great unspoken threats to democracy.

  • The Fool on September 11, 2012 2:04 PM:

    Yeah it's complete bullshit -- even worse than most of the above are saying.

    In the 2008 exit poll, Jews were 2% of the electorate. 2% of 808 = 16 respondents. That is a ridiculously low sample size. Most polling organizations won't go below 50 and even then the margin of error is +/= 14%. With a sample size of 16 absolutely no reputable pollster would even publish results at all. None. The MOE on n=16 is +/- 24.5%.

    And then when you compare results between two polls, the MOE on the difference is roughly 1.5 the MOE on a single poll so the MOE on the Jewish subgroup between polls would be about +/- 37 points.

    Completely worthless.

  • paul on September 11, 2012 2:52 PM:

    17 responders. 10/17 = 0.588.

    Pretty much any time you can tell the number of responders from the percentage, you're looking at a bogus result.

  • grandpa john on September 11, 2012 4:04 PM:

    One of thee first pieces of knowledge I gained about polling from Nate Silver was many years ago, even before he moved to the NYT. That piece of information was any one poll taken by itself is useless. Any body trying to establish a trend using the data from one poll is a fool. I think we can see from this mishmash here that Nate was correct .