Editore"s Note
Tilting at Windmills

Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for Free News & Updates

February 6, 2008
By: Kevin Drum

EXIT POLLS....Michael Sean Winters grouses about the media's continuing reliance on exit polls despite the fact that they seem to be consistently inaccurate these days:

The problem last night was that the exit polls were way wrong. At about 6:35 p.m., the Huffington Post leaked the exit polls, predicting that Obama would win New Jersey, Arizona and Massachusetts. In fact, he won none of those states. But, the expectations were set.

....Bill Schneider gave thoughtful analyses from the same exit polls, telling America how women had voted, how Latinos had voted, what issues mattered most. He neglected to say that the polls had failed to get the winning candidate correct. On ABC, Charlie Gibson noted that the exit polls indicated that late-deciding voters had broken towards Clinton by a significant margin, but did not share the bad news about those same polls misjudging entire states.

There's really something to this. Four years ago I had exit poll fever too, but this year I don't. Why? Because they've been wrong so often. Why would I get all excited about data that's little better than random noise?

Once all the slicing and dicing is done, exit polls provide useful demographic information. But if I were a professional reporter or talking head, I'd actively try to avoid early exit poll data these days. Why run the risk of twisting my expectations in the wrong direction and then having to clear my head later on? You wouldn't wait breathlessly for the afternoon astrological predictions, so why do the same for the afternoon exit polls?

Kevin Drum 12:24 PM Permalink | Trackbacks | Comments (36)

Bookmark and Share

Exit polls are just as flawed as all other polls by virtue that they are completely reliant on the respondent telling the truth about whom the voted for; something the reposndent may not always do.

Posted by: mfw13 on February 6, 2008 at 12:25 PM | PERMALINK

Because the ethic among today's journalists is to opine rather than report, and polls are easy "facts" on which to hang endless commentary and speculation.

That they are of shaky reliability is immaterial; they're out there, they're quantitative, and they're free.

It's info-tainment. It's not journalism.

Posted by: bleh on February 6, 2008 at 12:27 PM | PERMALINK

People gotta know RIGHT AWAY. Reporters will go to them. Eventually, they'll say things like, "...remember dum dums, these are exit polls, so they're not that reliable, but you want 'em so we give 'em..."

Posted by: rusrus on February 6, 2008 at 12:28 PM | PERMALINK

What bleh said.

And, the talking heads are compelled to talk, period. Accuracy does not matter, literally. I mean, it's not like any of them get dressed down by higher management for never being right.

Posted by: zak822 on February 6, 2008 at 12:34 PM | PERMALINK

Is it my imagination, or are exit polls actually getting less reliable as the years go on?

Posted by: Hyde on February 6, 2008 at 12:35 PM | PERMALINK

Probably just widespread vote rigging.

Posted by: Boronx on February 6, 2008 at 12:35 PM | PERMALINK

Actually, the exit polls are quite useful provided they're properly used. With the exception of the bad AP Missouri call [a marginal error], the pros generally got their projections right--but only after weighting the exit polling data with actual returns. Your example, the Huffington Post leak, is a prime example of how *not* to use exit polling data. I love exit polling data, but really only for after-the-fact analysis, once they've been properly weighted for actual turnout. Using them, like the Post did, as insider horserace scoop is bad practice.

Posted by: David in Nashville on February 6, 2008 at 12:36 PM | PERMALINK

Exit polls showed Kerry winning big in the afternoon in 2004. Later, we found out that they were not correct. Did people not vote because Kerry was ahead? Hard to say.

Exit polls should be treated the same way that regular votes are treated - not released until the polls are closed, only released after careful examination. There are HUGE biases in response.

Posted by: POed Lib on February 6, 2008 at 12:39 PM | PERMALINK

What bleh said, and what David and POed said.

Technical question behind this. Did Huffington leak 'unweighted' data like TPM did? If so, is that part of the situation? Caveats disappear at some point in the communications process...

Posted by: davidduck on February 6, 2008 at 12:43 PM | PERMALINK
Is it my imagination, or are exit polls actually getting less reliable as the years go on?

There are three obvious possibilities:
1) Exit polls, which have been reliable worldwide as checks for election fraud, are suddenly becoming less reliable in the US for some reason, or
2) The counting of actual votes in the US is getting less reliable, or
3) Media coverage of elections in the US increasingly focuses on exit polling, magnifying the apparent differences between exit polling and actual election results.

My guess is that there is a combination of factors at work, and that #1 is the least significant of the three.

Posted by: cmdicely on February 6, 2008 at 12:49 PM | PERMALINK

How many incidents are there of election polls getting it wrong in *favor* of the establishment candidtate?

Posted by: Boronx on February 6, 2008 at 12:50 PM | PERMALINK

Because Obama's momentum had reached the end of its useful life as a dramatic headline, it was essential that a Clinton resurgence dominate today's news. The fact that Obama won more delegates and more states was secondary to the need to attract/stimulate readers and viewers. Every few days the media slides into another new outfit from Fredericks of Hollywood.

Being the first to proclaim the most dramatic bullshit is far more critical to viewership and readership than getting at the real truth of anything.

Posted by: chance on February 6, 2008 at 12:51 PM | PERMALINK

"You wouldn't wait breathlessly for the afternoon astrological predictions, so why do the same for the afternoon exit polls?"

I wouldn't, and I assume you wouldn't. But there are vast numbers of people who watch CNBC and similar constantly and, let's face it, most of what is on that channel isn't much different from astrology (Keith Olbermann aside).

Posted by: Maynard Handley on February 6, 2008 at 12:51 PM | PERMALINK

2) The counting of actual votes in the US is getting less reliable Bingo.

I seem to remember that exit polls were pretty reliable until 2004. Suddenly, they're a joke. Hmm... could someone have a vested interest in exit polls coming to be perceived as less than reliable?

Posted by: thersites on February 6, 2008 at 1:03 PM | PERMALINK

There's another factor: exit polls only count those voters who physically show up at the polls. Absentee ballots and early vote-by-mail people are not counted. In California this was a huge effect.

Exit polls are useful for telling which subgroups went which way.

Posted by: Joe Buck on February 6, 2008 at 1:07 PM | PERMALINK

David in Nashville has it correct. For what the exit polls are used must await some actual vote returns for calibration and adjustment. There is historic data on what vote turnout is for a set of precincts, but the actual turnout in any given election is going to be different from that historic data. In other words, you don't know what the exit poll data even means until it has been calibrated with, at a minimum, voter turnout data. The way most people interpret the raw data, or even the data adjusted to reflect historic voter patterns is just stupid.

Posted by: Yancey Ward on February 6, 2008 at 1:16 PM | PERMALINK

In every other country on the planet, if the results don't match the exit polls, they storm the parliament.

In America, we blame the pollsters.

Posted by: scarshapedstar on February 6, 2008 at 1:31 PM | PERMALINK

I am not hearing anyone talking about how this shows a resurgence for Hillary. Just the opposite. I have heard a lot of talk about how she was supposed to have been able to do this so easily (a media story line)that this was bad for her,how the next states favor him, how he has the momentum. No mention of how she was considered done after Iowa or that Obama was expected to do so well in Cal. I have heard some talk about how the Kennedy support didn't seem to matter.

As for exit polls, one problem is trying to get an accurate sample that represents the people who came out to vote. Since turnout is much larger than usual this is harder to do.

Posted by: BernieO on February 6, 2008 at 1:34 PM | PERMALINK

Oh c'mon only idiot networks would start analyzing from the original leaked polls. They were fun to see how the initial early returns were shaping up and trying to game out how'd they'd change later. They were a fun timewaster. Anyone who thought anything more of them is deluded.

Posted by: MNPundit on February 6, 2008 at 1:36 PM | PERMALINK


Congrats on your state's vote last night! I'm in love with Minnesota, having beeing the Boundary Waters a few times. Great state.

Posted by: nepeta on February 6, 2008 at 2:07 PM | PERMALINK

beeing = been to

Posted by: nepeta on February 6, 2008 at 2:08 PM | PERMALINK

Perhaps folks are wising up and intentionally lying to pollsters.

Posted by: Chris Brown on February 6, 2008 at 2:15 PM | PERMALINK

All the exit polls were updated on the course of the night and they seemed pretty accurate to me.

The exit polls upon which people predicted an Obama win missed the latest wave of respondents which gave a boost to Clinton.

Posted by: Nick Kaufman on February 6, 2008 at 2:19 PM | PERMALINK

I refuse to talk to them. I remind them that my ballot is supposed to be secret, and besides that, they unduly influence the electorate on the west coast. Sometimes I apply a little Jewish Mother shame, if I think it will be effective.

Posted by: Blue Girl, Red State (aka G.C.) on February 6, 2008 at 2:23 PM | PERMALINK

some of the old fallacies have been repeated here.

1. the exit polls have always been inaccurate in the U.S.


2. exit polls have been even further off in the UK and Germany.

3. although German exit polls are usually considered the gold standard and are accurate most of the time. (election authorities work with the pollsters to give them access...unlik in the U.S.

so stop repeating that crap about how they've always been accurate before.

they've always been inaccurate.

Posted by: Nathan on February 6, 2008 at 2:33 PM | PERMALINK

If it is slow news day the newspapers print an edition with less pages, but the TV news still has an hour to fill. Cable news still has 24 hours to fill.

They gotta fill the time with something.

Posted by: Tripp on February 6, 2008 at 2:58 PM | PERMALINK

Talk about "random noise," Zogby's California poll for the democratic race was off by about what, 20%. That's not a poll, that's a dartboard. Dear Mr. Zogby, allow me to point you to Monster.com. A second career (or in your case a first one) can be good for the soul.

Posted by: steveboy on February 6, 2008 at 3:09 PM | PERMALINK

I've also noticed that quite a few election eve polls were way off. Yesterday's DrudgeReport had a link that supposedly had Obama leading in California by 13 points. Also, I saw in several media outlets polls and pundits saying that Romney had a growing breakout lead in California as late as Monday!

The media, especially MSNBC and Tim Russert, kept hinting at a very tight race in Massachusetts at the 7:00 pm EST and ultimately it was a Clinton blowout of Obama 56% to 41%. We did have a far bigger turnout of about 43% of the electorate way above the Secretary of State prediction of a 30% turnout. My town voted for Clinton over Obama by almost 2 to 1 with 85% of the voters turning out.

By and large the media coverage of the election has been superficial and largely wrong. Just a couple of days ago they were proclaiming a McCain coronation.

Posted by: John on February 6, 2008 at 3:43 PM | PERMALINK

Blue Girl: Sometimes I apply a little Jewish Mother shame

Oy. You're not wrong, and I might very well do the same if anyone tried to exit-poll me. (Don't know how well I could do the Jewish Mother thing, though.;))

But the fact remains, people do respond. And the article Nathan linked to reads: The problem with this reasoning is that exit polls [have been] similarly "wrong" before, though perhaps not to the same degree or consistency. This was written in 2004.

So no, I'm not repeating crap. I'm pointing out that somehow, exit polls are suddenly more inaccurate that they used to be. And asking why.

Posted by: thersites on February 6, 2008 at 3:52 PM | PERMALINK

Well then, I have no idea...:)

Posted by: Blue Girl, Red State (aka G.C.) on February 6, 2008 at 3:56 PM | PERMALINK

Maybe entrance polls would be more accurate.

Posted by: AJ on February 6, 2008 at 4:02 PM | PERMALINK

Don't overlook the spin part of these exit polls: the Clinton campaign intentionally released phony exit data showing close races, knowing that they had larger margins locked in. This added to the false expectation game that permeated the media, and the jive they perpetuated that Clinton had a "comeback" of "bigger than expected" margin!

Posted by: Tom on February 6, 2008 at 4:08 PM | PERMALINK


the specific issues with 2004 were here:


the 2004 inaccuracies were not vastly out of range compared to the inaccuracies of previous years (and nothing has been as far off as the 1992 UK exit polls).
furthermore, it's not like there's some long 100 year history of exit polls. they've only been around for a few elections. one of those elections is going to be a little bit more off than the others. that happened to be 2004. 2008 would have to be even further off before you could even begin to talk about a pattern (and you'd still have an awfully small sample size).

Posted by: Nathan on February 6, 2008 at 4:10 PM | PERMALINK

Oh, stop bludgeoning me with factual reports!
I go with my gut instincts, is all. If the Leader of the Free World can do it, why can't I?

Posted by: thersites on February 6, 2008 at 4:26 PM | PERMALINK

The truth is that it's difficult to tell whether it's the polls or the actual vote tallies that are inaccurate.

I was a poll watcher on Tuesday in Brooklyn for 9 electoral districts at one poll site. Approximately 12.5 % of voters were not listed the voter rolls and were forced to fill paper affidavit ballots. These ballots must be inspected and verified before they are counted, and they were certainly not part of any vote counts announced thus far.

That means that a pollster could get a 100% accurate count of all votes cast, and be off as much 12.5% of the vote tally that you're rushing to judgment on and condemning the validity of the poll.

As a side note, the voters forced to submit affidavit ballots were mostly younger and male in a district that is predominantly black. Or maybe that's not a side note; maybe that's why we need exit polls.

Posted by: Poll Watcher on February 6, 2008 at 9:55 PM | PERMALINK

Tom: "Don't overlook the spin part of these exit polls: the Clinton campaign intentionally released phony exit data showing close races, knowing that they had larger margins locked in."

I suppose that the next thing you'll tell us is that Hillary's also suppressing evidence that Bill was involved in the disappearance of Natalie Holloway.

Posted by: Donald from Hawaii on February 6, 2008 at 10:23 PM | PERMALINK



Read Jonathan Rowe remembrance and articles
Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for Free News & Updates

Advertise in WM

buy from Amazon and
support the Monthly