Editore"s Note
Tilting at Windmills

Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for Free News & Updates

August 11, 2008
By: David Moore

A few days ago I wrote a post about The Opinion Makers: An Insider Reveals the Truth Behind the Polls, a new book by David Moore, a former Vice President of the Gallup Organization and Managing Editor of the Gallup Poll. We talk about polls a lot in the blogosphere, so I thought it would be interesting to invite David to guest blog about his book this week. You can read more about the book here, and of course the book itself is available via Amazon and other online booksellers. David's first post is below, and he'll be around all week writing about poll issues and answering your questions.


DRILLING FOR OIL....One reason we can't trust most media polls is that they don't differentiate between "those who express deeply held views and those who have hardly, if at all, thought about an issue." That quotation comes from the book's announcement, and it caused Kevin to write:

This is disturbing. Either Moore managed to find a publisher for a book thesis about as obvious as "college students like to drink," or else Moore's thesis actually isn't as bog obvious as I think it is. I'm not sure which is worse. Or there's a third option: his thesis really is as obvious as I think it is, but everyone keeps pretending not to know it anyway.

Like Kevin, I believe that in their heart of hearts, most political observers know that polls measure superficial opinions. Still, when the poll results are presented, it's hard to dismiss what appear to be "scientific" measures of the public will.

Last week, for example, Paul Krugman cited with dismay a CNN poll showing 69 percent of Americans in favor of expanded offshore oil drilling. An article in The New Yorker cited similar poll results, as did pundits on several television news shows.

CNN was not alone. CBS, Fox, and the Los Angeles Times all showed similar or greater support for offshore oil drilling. It appears that this general consensus about public opinion has even persuaded Barack Obama to modify his opposition to offshore oil drilling.

But does the public really support more offshore oil drilling? Clearly the public wants the government to do something about the energy problem, and when presented with an isolated proposal — with no mention made of either possible environmental trade-offs or of the long time it might take for expanded oil drilling to actually produce more oil — the proposal sounds good. But that doesn't mean people are not willing to consider trade-offs or other approaches.

Most of the polls frame the issue as though it were a problem of "energy independence" or of dealing with the "rising cost of gasoline." But the energy problem is much more complicated.

A couple of polls addressed the energy issue a bit differently, and they found a more ambivalent public. Pew Research, for example, asked which of two approaches should receive higher priority: "expanding exploration, mining and drilling and the construction of new power plants, OR, more energy conservation and regulation on energy use and prices?" Instead of overwhelming support for more oil drilling, the public was evenly divided between that approach and conservation (47 percent to 45 percent respectively).

An NBC/Wall Street Journal poll offered five different approaches to dealing with the energy problems. Almost half (46 percent) opted for energy conservation and more emphasis on wind and solar, while 40 percent chose offshore oil drilling and drilling in protected areas in Alaska, while 10 percent preferred nuclear power.

And the CNN poll actually measured intensity of opinion, by asking if people "strongly" or "mildly" favored, or "strongly" or "mildly" opposed increased offshore oil drilling. The results found 46 percent "strongly" in favor, with 18 percent "strongly" opposed. More than a third, 35 percent, held only "mild" opinions. (In all discussions of the CNN results, however, there was no mention of the "mild" and "strong" opinions. The two groups were combined according to favor and oppose, which is typical of the way poll results are treated.)

Despite conventional wisdom these days that the public overwhelmingly supports expanded offshore oil drilling, some careful poll results reveal a more complicated public opinion. If all the pollsters were willing to explore the uncertainty of public opinion, they would find a large segment of the public that genuinely doesn't know the best strategy for the energy problems this country faces. Instead, the polls give the erroneous impression that the vast majority of Americans have a clear opinion — with zero percent undecided, according to CNN, and no more than 10 percent according to other polls. And that opinion favors expanded offshore oil drilling.

Do you believe them? I don't.

David Moore 1:17 AM Permalink | Trackbacks | Comments (28)

Bookmark and Share
 
Comments

When someone starts talking to you about "energy independence", keep one hand on your wallet and start looking for the exit.

Energy independence might have been a plausible goal forty years ago, but we didn't go that way. Now it's like perpetual motion -- a sure sign that the speaker is either delusional, ignorant, or using emotional manipulation to sell something.

Posted by: joel hanes on August 11, 2008 at 1:33 AM | PERMALINK

Another polling technique not seen in MSM surveys too often is to ask the question, provide information, then ask the question again. An informed opinion should count more than one that's not.

Moore also makes a good point about the misleading tendency to combine strongly and mildly held opinions to give an aggregate "for" or "against."


Posted by: DevilDog on August 11, 2008 at 1:49 AM | PERMALINK

There's no mistrust helter-skelter it, this d webcam version from [url=http://cams.informe.com/forum/]cams.com[/url] ,is one horny slut!

Posted by: Lolfunloara on August 11, 2008 at 2:18 AM | PERMALINK

Ahh, David, but doesn't this get back to a steretyped view of the majority of the American populace -- one that I think has a large degree of truth -- that Americans WANT black-and-white answers?

Posted by: SocraticGadfly on August 11, 2008 at 2:25 AM | PERMALINK

Let's see the poll:

Do any of the three branches of government, the media, or large corporations work in the interests of the American people?

Then, I'd bet, we can start having some interesting conversations.


Posted by: jim p on August 11, 2008 at 2:28 AM | PERMALINK

Do you believe them?

Depends on who you mean by "them". From your excerpts, the poll results suggest more nuance and range of opinions than simply "favors expanded offshore oil drilling".

It seems this is less a problem that "...the polls give the erroneous impression...", and more that the reporting of poll results give an erroneous impression.

Posted by: has407 on August 11, 2008 at 3:57 AM | PERMALINK

Certainly this expands the error bars in polls and tells you how an advertising campaign might move numbers. However, I'm afraid people with weakly held opinions based on shallow analysis do quite a bit of voting. There's a reason pollsters have fairly inclusive turnout models prior to the conventions.

On issues like this perceived action gets a lot of votes for purely psychological reasons. If the nation was jumping off a cliff we'd be voting to flap like a bird and float like a feather. In fact, maybe we are.


Posted by: B on August 11, 2008 at 7:10 AM | PERMALINK

Rule of 4, change the equation, watch for what moves votes. This is a SCIENCE -- why not learn how it works?

Posted by: anonymous on August 11, 2008 at 8:02 AM | PERMALINK

I noticed the stories about the polls that said American's prefer drilling to conservation didn't really reflect the poll results or questions. Maybe reporters should have their work checked by somebody. Maybe the job would have a nice title. I don't know, "editor" comes to mind. That way the reporter who dummies up his conclusions wouldn't be embarrassed by somebody who actually reads the poll.

On second thought who needs an editor. Most people don't read beyond the headlines anyway, and facts have such a messy liberal bias.

Posted by: Ron Byers on August 11, 2008 at 8:43 AM | PERMALINK

Who funds these polls is an important question to ask. A fairer way to frame a clean energy polling question is to give people options, like:

Please rate in order of preference the technologies and policies the US government should support in order to achieve energy independence and reduce the negative environmental impact of burning carbon-based fuels:

1. More mass transit
2. Higher CAFE standards & let the market sort out the technology to achieve it
3. Hydrogen-fueled cars
4. Compressed air-fueled cars
5. Ethanol/Methanol-fueled cars
6. Battery-powered cars
7. Hybrid-fueled cars (part gasoline)
8. Nuclear power
9. Wind power
10. Solar power
11. Hydro power (dams)
12. Tidal-generated power
13. More cutting-edge R&D
14. Combinations of the above (solar on cars, etc.)

Or, if the question has to deal with drilling offer something like:

"The oil anticipated from more drilling in Alaska and off the shores of California and Gulf Coast States will be enough to meet XX percent of US energy needs in 10 years, when it is anticipated new wells would become productive. Investment in new wells would create jobs and help reduce dependence on foreign oil but they could also damage the environment. Furthermore, the oil from these wells would expand America's carbon footprint and experts say there is only enough to last XX years. In view of these potential benefits and risks is it wiser to invest in more domestic oil production or invest in alternate forms of energy?"

A) Oil exploration is wiser B) Both options are equally wise and should be pursued C) Alternate energy investment is wiser

Posted by: pj in jesusland on August 11, 2008 at 8:47 AM | PERMALINK

I used to conduct opinion polls in college as an interviewer. One thing the people who analyze polls don't appreciate is that a high percentage of poll-takers -- maybe 20% -- are highly disengaged from the survey-taking. A lot of people are answering questions by rote because they're too polite to hang up yet really aren't willing or able to give the poll their full attention. I got many Moms trying to take a survey while they cooked dinner, tamed their kids, and kept an eye on the TV. I even got people in the middle of sex.

People who don't care don't even try to answer polls accurately. I always keep that in mind when I see a crazy poll result that ascribes an absurd opinion to a hefty percentage of the American population.

Posted by: Wagster on August 11, 2008 at 8:48 AM | PERMALINK

You want NUANCE?

Do you NOT pay attention?

The polling organizations are as much instruments of the Corporate State as any of the rest of the So-Called Unbiased Media (SCUM). As with eht "press," their purpose is to promote the "race," and ultimately to keep it close enough to steal.

And you want these folks to ask nuanced questions about a gauranteed-farking-WEDGE issue?

Really?

Bwahahahahahahahahaha-gasp-bwahahahahahaha

Posted by: woody, tokin librul on August 11, 2008 at 8:50 AM | PERMALINK

I also have trouble with the polls. As a psychologist and statistician, I had tremendous problems with the 2004 exit polls which supposedly did not implicate gay marriage in the result. I believe that it was important in two ways which cannot be measured by polls:

1) It got people out who would not have voted

2) It was the marginal edge which pushed them to vote for Bush

That's what polls cannot get at: nuance. Polls can get at big causes but cannot do nuance. And the polls cannot measure more than 1 thing. Let's say that you support McCain for his background and his position on abortion. Most polls will ask "What motivates you to support McCain?" They don't give you two, three or four choices. That was the other problem with the 2004 exit polls; they only allowed one answer.

Posted by: POed Lib on August 11, 2008 at 8:56 AM | PERMALINK

Obtw: I got a poll call night before last.

After a couple of pretty much standard yes-no qualifier questions, the pollster unreeled a list of 10 or 12 issues--health care, war, environment, economy, big, bloated issues like that--was the "Most Important." Which was when I hung up, but not before telling the caller that that was SUCH a bullshit question that it made me wince. "Who the Fnck wrtiest this shit?" I screamed as I slammed the phone back in the cradle.

Posted by: woody, tokin librul on August 11, 2008 at 8:56 AM | PERMALINK

Woody's comment is exactly correct. For many many people, there are many issues which motivate them. What is MORE important is a multiple choice question about "Which one or more issues do you not give a shit about?"

Posted by: POed Lib on August 11, 2008 at 9:08 AM | PERMALINK

On Saturday, I got a call from Gallup, which was doing a survey on the beverage habits of 21- to 34-year-olds. They asked to speak with someone in the household in that range, and I said there wasn't anyone, and he said, "How old are you?" And I said, "48." And he said, "Close enough -- will you answer some questions?" Obviously this guy had some kind of quota to meet, and didn't care how he met it. So much for the sanctity and accuracy of polling...

P.S.: I did answer the questions, because he sounded pretty desperate, and it was fun to mess with some corporate behemoth's marketing efforts. Most of the questions involved how much booze I drink, and what might make me drink more. Charming.

Posted by: eparker on August 11, 2008 at 9:15 AM | PERMALINK

The reason that people support oil drilling is that they want to do SOMETHING, ANYTHING, to reduce gas prices. It's always better to do something rather than nothing is the thought. If we can do oil drilling, let's do it, is the thought. And further, to drill a hole is very quick. To develop an electronic car or fusion is 20 years away.

Posted by: POed Lib on August 11, 2008 at 9:18 AM | PERMALINK

This is not about drilling.

More leases = more reserves = higher company value.

Posted by: pebird on August 11, 2008 at 10:49 AM | PERMALINK

Most importantly
it points to the need for leadership on the energy issue by certain party nominees who seem to be reverting to the OLD politiics

Posted by: david mace on August 11, 2008 at 10:52 AM | PERMALINK

Many people have trouble believing the current administration and Republicans in general. For some, the Republicans are considered the “enemy” of “We the People”.

Yet we still rely on the corporate MSM to provide honest news of daily events and are eager to swallow their swill.

The MSM is also the enemy of “We the People” and one should not let their guard down just because the MSM is wearing sheep’s clothing.

I suspect most polls are skewed to reflect the “right” results of the corporate MSM agenda.

Posted by: Meg on August 11, 2008 at 11:01 AM | PERMALINK

Another distorting factor is the relative precision of election polls, which spills an undeserved favorable aura onto other polls. I am the world's leading expert on who I am going to vote for. Even then, I may change my mind before the election, or lie to the pollster, or toss off a semi-random choice on a race I haven't thought about. But I am the expert, the one who knows if anyone does. Many other poll issues are orders of magnitude fuzzier -- even in I am engaged and trying to answer well, which as others have stated is not always the case.

Posted by: Ken D. on August 11, 2008 at 11:26 AM | PERMALINK

If polls are trying to measure likely election results, then it doesn't really matter if a vote is because of deep conviction, or barely more than a whim, it still holds the same weight when the votes are counted. So the real issue with the lessor committed, is how likely they are to vote, and how likely they are to change their vote if they do.

We also have issues where multiple responses are needed. Energy is the perfect example here, all the experts agree about there being no silver bullet, and only a combination of several approaches, often called wedges has any chance of working. So for instance, I personally support expanded drilling, but its priority is low because I recognize it as a very small wedge (sliver). But a simple yes/no question in inadequate. Also in this particular case my position is robust (not likely to change), but not strong, in the sense that I might give it a fairly low weighting (because it is a sliver not a wedge), in selecting whom to vote for.

Posted by: bigTom on August 11, 2008 at 12:00 PM | PERMALINK

Hi,

This is the relevant poll question for U.S.

When it comes to using the remaining crude oil reserves in the world:

a. Should we pump the last remaining barrel of American oil as quickly as possible?

or

b. Should we drain Saudia Arabia first?

Follow-up question--If a., what is your plan after that?

Have a nice day,
Antti

Posted by: Antti Nannimus on August 11, 2008 at 12:27 PM | PERMALINK

Antti: Just use less.

Posted by: Mike Meyer on August 11, 2008 at 12:49 PM | PERMALINK

The new Democratic energy policy is: "Don't ask, don't drill." Catchy, isn't it?

Posted by: DBL on August 11, 2008 at 1:38 PM | PERMALINK

Instead of "DRILL HERE, DRILL NOW, PAY LESS", I've been thinking of getting a bumper-sticker that says "DRIVE LESS, DRIVE SLOWER, SAVE MONEY". The only problem is that almost no one would see it, as I rarely drive my car - less than once a week! No wonder the Republican message gets out better.

Posted by: William Fraser on August 11, 2008 at 3:30 PM | PERMALINK

The ultimate commentary on frameing effects in polling.

http://www.youtube.com/watch?v=2yhN1IDLQjo

Posted by: Marc in Asia on August 11, 2008 at 11:22 PM | PERMALINK

Isn't it a bitter irony that Bush, Cheney, Rove, Congressional Republicans, et al, say they don't believe in polls when 70+% of Americans say we should get the hell out of Iraq, yet continually throw slanted compromised propagandized polls at us claiming over 60% of Americans favor drilling in ANWR & the OCS?

This is truly a "have your cake and eat it, too" ploy for them, but total BS for the informed among the population.

Just like Stalin's famous (infamous) statement, "It's not who casts the votes, but who counts the votes that counts (paraphrased)", it seems the motivations of those calling for a poll are the prime consideration.

Unfortunately, although most Americans seem to have rudimentary reading skills & rather advanced skill at being mesmerized by propaganda emmitted from a TV screen, they sadly lack abilities in critical thinking, logic, and correlation of data from multiple sources other than what they have erroneously been taught to consider 'safe & true'.

The information super-highway (internet) is only useful when the vehicles (minds) on it have capable competent drivers.

How embarrassing for McCain, after girlish giggling laughter about Obama's 'proper tire pressure & tune ups' was proven misplaced by the DOE & EPA, to have to eat crow & admit Barack was correct. Immediately, according to those government agancies, proper tire pressure & tune ups saves Americans an average of 12 cents per gallon, while ANWR & OCS drilling will only lower price at the pump about 6 cents...and only by 2020 at that. Who is perpetrating a hoax on the American people, the 'drill, drill, drill' oil bribed Republicans, or conservation & alternate power minded Democrats in Congress?

Posted by: PaulMagillSmith on August 12, 2008 at 8:36 AM | PERMALINK




 

 

Read Jonathan Rowe remembrance and articles
Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for Free News & Updates

Advertise in WM



buy from Amazon and
support the Monthly