Editore"s Note
Tilting at Windmills

Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for Free News & Updates

July 18, 2008
By: Kevin Drum

GENERATION CELL....Are telephone surveys that don't include cell phones accurate? Apparently so, even as the number of cell phone users continues to grow. Here's the latest from Pew, which just conducted a survey of both cell and landline users:

Cell-only respondents are significantly more likely than either the landline respondents or the cell-mostly respondents to support Barack Obama and Democratic candidates for Congress this fall. They also are substantially less likely to be registered to vote and — among registered voters — somewhat less likely to say they are absolutely certain they will vote.

Here's what this boils down to. In the landline-only sample, Obama was ahead by five points, 46-41, but in the combined sample he was ahead by eight points, 48-40. Among people who are certain of their vote, Obama leads by eight points in the landline sample and ten points in the combined sample.

This suggests that although we're starting to get to the point where pollsters risk some genuine distortion if they don't include cell users in their surveys, we're not there yet. The differences in the Pew poll are still small and might just be a sampling fluke. Within a few years, though, this is likely to change, and in the meantime maybe it would be worth someone's time to do a mega-survey (say, 10,000 respondents) to get a better handle on this.

UPDATE: One more thing, though: it's worth noting that among cell-phone-only users (i.e., people who don't have landlines at all and use cell phones exclusively), Obama beat McCain by a whopping 29 points, 61-32. The reason it didn't affect the overall numbers much is because — so far — they're still a pretty small proportion of the population. Also, there's this interesting demographic breakdown of cell-only and cell-mostly users:

Compared with all respondents reached on a landline, both groups are significantly younger, more likely to be male, and less likely to be white. But the cell-only and cell-mostly also are different from one another on many characteristics. Compared with the cell-only, the cell-mostly group is more affluent, better educated, and more likely to be married, to have children, and to own a home.

Kevin Drum 12:04 PM Permalink | Trackbacks | Comments (32)

Bookmark and Share
 
Comments

I'm too lazy to do the numbers but I guess that cell phone only vs. landline is a good proxy for age.

Posted by: Josh on July 18, 2008 at 12:18 PM | PERMALINK

There's been no evidence that, controlling for demographics, cell phone users lean one way or another. Cell phone users are more likely to be young, urban, etc. But for a given white 28 year old male who lives in Chicago, being cell-phone only or not doesn't correlate with voting patterns.

What this means is that demographic re-balancing of polls will become more important as time moves on.

Posted by: Shock Mouse on July 18, 2008 at 12:20 PM | PERMALINK

The market research company I used to work for used area codes, the chosen geographical area's prefix number(s) and then randomly selected the last four digits (random digit dialing) to find respondents for polls. This method of sample selection should include cell phones, or cell phone prefix numbers could be added to the selection process.

(Anyone remember Watts lines?)

Posted by: Brojo on July 18, 2008 at 12:31 PM | PERMALINK

This seems like a remarkably inefficient method for determining voting patterns. I'll bet Pew could do a lot better if they could monitor key words from central telephone hubs.

Posted by: asdf on July 18, 2008 at 12:33 PM | PERMALINK

Josh is right, and I hope we democrats don't count too heavily on "young people" to push our candidates over the top this year. In every single election since I started voting in 1980, the "young people rock the vote" meme has been there (even before the word "meme" was a word and "rock the vote" was an MTV trademark), and each year, it's been like Lucy pulling the football away from Charlie Brown right before he kicks it. I don't get how they can be so allegedly pumped up about the election each cycle yet fail to show up the first Tuesday in November. Is November the month when the most potent pot harvest hits the market or what?

Posted by: Tom in Houston on July 18, 2008 at 12:40 PM | PERMALINK

Compared with each other, "cell only" are probably more likely to be immigrants from Asia, Latin America, etc. While "cell-mostly" are probably richer guys who could live easily as "cell only" but since they've got the money, have a land-line that they don't really use.

Posted by: flubber on July 18, 2008 at 12:45 PM | PERMALINK

Josh is right, and I hope we democrats don't count too heavily on "young people" to push our candidates over the top this year. In every single election since I started voting in 1980, the "young people rock the vote" meme has been there (even before the word "meme" was a word and "rock the vote" was an MTV trademark), and each year, it's been like Lucy pulling the football away from Charlie Brown right before he kicks it. Posted by: Tom in Houston

The economy may roust a few more to the polls, but the young ceased being a major political force not long after the least likely to participate (and least well-informed of them, 18-20 years old) got the vote. Vietnam and the Civil Rights movement were winding down. The protest marches of that era were mostly comprised of the young. Prior to the current illegal and unnecessary war, the protests were fairly "grey."

Surprisingly, though gaining the vote in 1971, their ire was felt once, perhaps, with the election of Carter as reaction to the Nixon years and all that entailed. The "youth vote" has been pretty much a non-starter ever since.

"If they'd only let us text in our votes!"


Posted by: Jeff II on July 18, 2008 at 12:56 PM | PERMALINK

How about those of us who avoid phone calls altogether, especially if the caller ID is "Unknown Name"?

Posted by: PapaJijo on July 18, 2008 at 12:59 PM | PERMALINK

Many Republicans I know use cell phones only, and they are not young.

Posted by: Brojo on July 18, 2008 at 1:14 PM | PERMALINK

Brojo is right about Randon Digit Dialing methods. But the bigger issue for market research is the difficulty of getting any phone response. Online research is more heavily utilized, and can even have a "random" aspect to it through pop up or ads. And more generally, everyone uses quotas, over-sampling or weighting.

Posted by: Ronnie P on July 18, 2008 at 1:18 PM | PERMALINK

I think you're right, and I was actually wondering about the effect of technology and polling. While you were on a panel, I listened to Joe Trippi and he mentioned the JFK-Nixon debate, where Nixon won on the radio, and JFK won on TV. He said it was evidence that TV audiences could be tricked by a lack of sincerity. But I wonder, could it just be that the audiences were demographically different and have different leanings, just as in this case?

Posted by: Michael Spitzer-Rubenstein on July 18, 2008 at 1:18 PM | PERMALINK

Brojo,
Using local prefixes wouldn't work because cell phones use their own prefixes. Not to mention, I wonder what percentage of cell users have the area code of their local area rather than their home town area.

Posted by: Mo on July 18, 2008 at 1:19 PM | PERMALINK

I maintain a landline as a backup to my primary cellular number. I never answer the landline it only rings when pollsters call.

Posted by: Grumpy on July 18, 2008 at 1:23 PM | PERMALINK

One more thing, though: it's worth noting that among cell-phone-only users (i.e., people who don't have landlines at all and use cell phones exclusively), Obama beat McCain by a whopping 29 points, 61-32. The reason it didn't affect the overall numbers much is because — so far — they're still a pretty small proportion of the population.

I guess they're balanced out by the "no Internet" demographic.

Posted by: Swan on July 18, 2008 at 1:28 PM | PERMALINK

I have a landline, but the phone connected to it has gone uncharged for months, never use it.

Posted by: SteinL on July 18, 2008 at 1:47 PM | PERMALINK

But I wonder, could it just be that the audiences were demographically different and have different leanings, just as in this case?

I would be surprised if they didn't account for demographics in their sampling. Pollsters aren't dumb: they set quotas, weight data, etc. The bugaboo of market research is when the white, female, 20-something, middle income Republicans in your sample are for some reason unlike the white, female, 20-something, middle income Republicans not in your sample.

Posted by: Ronnie Pudding on July 18, 2008 at 1:51 PM | PERMALINK

I second PapaJijo. No name, no pickup for me.

Posted by: Matt on July 18, 2008 at 2:08 PM | PERMALINK

Come November, one of three things will happen:

1. Obama does better than polling indicates. Amid lots of breast-beating, pollsters admit they have to bite the bullet and start polling cell-only people.
2. The polls accurately predict the election. Pollsters use this result to vindicate their contention that the cell-only vote need not be counted.
3. McCain does better than the polls predict. The Bradley effect lives! Feel guilty, America.

Note that the polls might make an accurate prediction because Obama's cell-only advantage is cancelled by the Bradley effect. Two polling error cancel out, with no one the wiser.

Posted by: tomtom on July 18, 2008 at 2:38 PM | PERMALINK

Brojo,

The methodology you cite is the "old methodology". Companies today use autodialers. Numbers are purchased from suppliers, and there are separate lists for landline and cell phones. I work for a major marketing research company, and households that are cell-only are severely under-represented in almost any research study. You can't use autodialers to dial a cell-phone number - its against the law. Since cell-only households are more than 15% of the US population (source CDC studies), this could be a major miss by Gallup, et. al.

Posted by: Geeedavey on July 18, 2008 at 3:43 PM | PERMALINK

Within a few years, though, this is likely to change, and in the meantime maybe it would be worth someone's time to do a mega-survey (say, 10,000 respondents) to get a better handle on this.

It'll happen. The government statistical agencies (I work for one) are paying close attention to this. So far, our conclusion has been the same as Pew's: not including cell-only households doesn't yet make a big difference in the outcome of a telephone survey. But we all know the day isn't too far off.

Posted by: low-tech cyclist on July 18, 2008 at 3:53 PM | PERMALINK

Here's the numbers Geeedavey mentioned:

Preliminary results from the July-December 2007 National Health Interview Survey (NHIS) indicate that nearly one out of every six American homes (15.8%) had only wireless telephones during the second half of 2007. In addition, more than one out of every eight American homes (13.1%) received all or almost all calls on wireless telephones despite having a landline telephone in the home.

The NHIS is an in-person survey (the questionnaire is just too damned long to do over the phone); they take a sample of households, and Census Bureau interviewers go out and knock on the doors and ask the questions. I think that in 6 months, they get about 15-20,000 households responding (it probably gives a sample size somewhere at the link, but I'm lazy right now), so the sampling error is quite small.

At any rate, that's a ton of mostly and completely cell households. As those numbers grow, my guess is that the cell phone population will increasingly look more like the population as a whole, simply because as P-->100%, the characteristics of the subpopulation of proportion P will converge to the population.

But the problem will be on the flip side, since as households that are either primarily or exclusively landline become an increasingly small group, they will become increasingly different from the population as a whole - increasingly old in particular, with all the other differences that go with that.

Posted by: low-tech cyclist on July 18, 2008 at 4:09 PM | PERMALINK

How do they get the cell numbers?!? These aren't available in any directory.

Posted by: Cathexis on July 18, 2008 at 4:18 PM | PERMALINK

My extended family are cell phone only users and we are extremely likely to vote and will certainly not vote for corrupt, evil, low intelligent, retards; aka republicans.

Posted by: Captain Dan on July 18, 2008 at 4:37 PM | PERMALINK

I'm a cell-only user. Land lines are a waste of money! I'm 28/m/white/middle-class, and voting for Obama. Oh, and I've never been contacted by a pollster - ever!

Posted by: Michael on July 18, 2008 at 5:06 PM | PERMALINK

I was surprised at first that the cell-only demo is so non-white, but it must correlate with low-income, since one phone of course is cheaper than two, and a cell is more useful than a landline.

How do they get the cell numbers?!? These aren't available in any directory.

It's just some random dialer program / machine or something like that. They can find out w/in a certain range which numbers are cell numbers (maybe in a given area code, certain prefixes- the first 3 digits after an area code- are mostly reserved for cells, and they dial randomly within that range).

Otherwise there might be a way for a machine to tell whether it's a cell-line or land-line that picked up without resort to the prefixes, but I bet the first version I described is more likely, and the second version, if it exists, is more something you use in law enforcement to find criminals.

Posted by: Swan on July 18, 2008 at 6:52 PM | PERMALINK

In fact the difference between cell phone and landline polls is statistically significant, that is, it is quite unlikely to be a sampling fluke (probability well under 5% being under one tenth of one percent according to my calculations which rely on the Pew results rounded to the nearest percent so they aren't exact). I believe when Pew wrote "significantly" they mean "statistically significantly" in other words the chance of a difference that large due to random sampling is less than 5%. I run the numbers here

http://rjwaldmann.blogspot.com/2008/07/pollster.html

Posted by: Robert Waldmann on July 18, 2008 at 7:20 PM | PERMALINK

Another way of phrasing your update (and a far more concise way):
Cell only users are younger than cell mostly users.

It is not that cell only users prefer to be single/not have children...etc.;. They just haven't got there yet.

The better educated bit could actually be a different dynamic...with poor credit it is actually easier to get a cell phone than a land line...which also makes sense if you are likely to not live in the same location for long (land-line portability not withstanding). I don't know enough about the market proportions to say it definitively: but I think this is just saying younger people skew more towards obama...and this is not news. Younger voters have tended to be more liberal for a long time.
The news is/will be when more of these younger voters turnout to exercise their preference

Posted by: Sam Jackson on July 18, 2008 at 7:32 PM | PERMALINK

regarding the comments on autodiallers; this is probably more a US only problem for now.
elsewhere in the world, cell phones are already in use by surveyors and telemarketers.
Which got to be quite an annoyance when I was visiting India for only a couple of weeks and get a prepaid sim card to keep costs down. Most of the calls I got were for mortgages and the text messages were for ring tones.
Until we get the caller pays model for cell phones this is likely to persist. But in order to move to a caller pays model, we will also need to have a fairly distinguishable set of numbers for cell phones.
Hopefully the resistance on the part of industry to that will go away...because I am sure there are a lot of users who want people to know that they are calling a cell phone; also the whole long-distance charge barrier has long since come down.
Sorry about the duplication of ideas in the earlier post...meant no disrespect to people who had commented earlier. You think your thoughts are different until you read them on the screen and realize they are not that different.

Posted by: Sam Jackson on July 18, 2008 at 7:44 PM | PERMALINK

http://mhollick.typepad.com/homodox_a_blog/2008/07/no-cells-in-this-poll.html

Posted by: Martin Hollick on July 18, 2008 at 8:54 PM | PERMALINK

http://gogle.com google

Posted by: lkg25yfaie0s on October 14, 2008 at 7:47 AM | PERMALINK

http://gogle.com google

Posted by: lkg25yfaie0s on October 14, 2008 at 7:47 AM | PERMALINK

http://gogle.com google

Posted by: lkg25yfaie0s on October 14, 2008 at 7:47 AM | PERMALINK




 

 

Read Jonathan Rowe remembrance and articles
Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for Free News & Updates

Advertise in WM



buy from Amazon and
support the Monthly