Editore"s Note
Tilting at Windmills

Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for Free News & Updates

February 17, 2008
By: Kevin Drum

RESEARCH ON THE WEB....The world is full of advice on how to do research. There's general advice that applies to research of any kind. There's particular advice for particular fields. There's advice that applies to archival research and advice that applies to field research. There's advice for novices and advice for graduate students.

But how about bloggers doing research on the web? Here are five pieces of advice for them. Enjoy.

  1. If you use Google (and who doesn't?) don't use the default page. Use the Advanced Search page instead:


    Sure, the Advanced Search page is sort of a crutch for people who haven't memorized Google's set of Boolean operators. But that's most of us, right? And since any advanced search you use is better than any advanced search you don't, you're better off with the crutch than with nothing. So bookmark the Advanced Search page and use it.

    While you're at it, you should also free yourself from the tyranny of getting only ten results per page. The best hits aren't always in the top ten, and you're more likely to see them if you just have to scroll down a single page rather than going back and forth between different result pages. So go to http://www.google.com/preferences and set your default to 50 results per page.

  2. Whenever you read something by someone you don't know, Google 'em. Find out what axe they have to grind. Are they liberal or conservative? Do they work for a think tank? Do they have a history of being obsessed by weird stuff? What expertise do they have? The web allows you to root out this stuff in less than a minute or two for most people. Take advantage of it.

  3. If you're writing about a specific topic that you're not that familiar with, take a minute and find an article that provides a quick outline of the general subject area. Even a modest 60-second familiarity with the lay of the land can save you a lot of grief and keep you from making an idiot of yourself.

  4. Speaking of which, use Wikipedia. No, it's not 100% reliable. And given the nature of the internet community, it's better on some topics than others. You're more likely to get a useful description of the binomial theorem than you are of the objective correlative in Heart of Darkness.

    But all reference works have limitations, and virtually all popular references should be taken as starting points, not final authorities. And that's how you should use Wikipedia: as a starting point. The scope of Wikipedia is vast; it's extremely useful for recent events; it frequently does a decent job of summarizing a topic; and most articles come with a lot of highly useful links. Sure, you have to be careful with Wikipedia, but you should always be careful anyway.

  5. And while we're on the subject, always click the link. The web makes checking sources so easy that there's no excuse for failing to at least skim the primary links in an article. Click, click, click!

Kevin Drum 1:44 PM Permalink | Trackbacks | Comments (49)

Bookmark and Share

Good suggestions. I also think you could be a little more specific about the strengths and weaknesses of Wikipedia.

WP is intentionally a tertiary source, and information is only as reliable as its sourcing to primary and secondary sources. If you read a "fact" on Wikipedia, it either has a solid source that backs it up, or its very presence on there is dubious and it's subject to deletion at any time.

But your general tone is quite accurate. "Sure, you have to be careful with Wikipedia, but you should always be careful anyway." Agreed 100% -- just trying to suggest one specific way to be careful. Hopefully to most of us checking the sourcing is obvious anyway.

Posted by: Equal Opportunity Cynic on February 17, 2008 at 2:08 PM | PERMALINK

Duh, just realized you covered that in #5. Good post.

Posted by: Equal Opportunity Cynic on February 17, 2008 at 2:09 PM | PERMALINK

My, but we're meta today.

Posted by: Lucy on February 17, 2008 at 2:17 PM | PERMALINK

Thanks for the good advice.

Because you asked, I don't use Google. I use Yahoo and a number of specialized sites, such as Findlaw if I'm looking into legal cases.

Posted by: Chris Brown on February 17, 2008 at 2:23 PM | PERMALINK

Thanks for the suggestions. I have a blog that deals with motion picture history, and this will help immeasurably with my research.

Posted by: Vincent on February 17, 2008 at 2:26 PM | PERMALINK

When you reveal Wikipedia as your source, people often groan and dismiss it. My reply is that if Hitler told you that it was raining out and no ulterior motive was apparent, would you believe him?

Good tips on Google.

Posted by: Michael7843853 G-O/F in 08! on February 17, 2008 at 2:32 PM | PERMALINK

As someone who spends a significant portion of each day researching on the internet, let me suggest 2 simple things I would show all google users to make their searches better. And these two are really easy to remember.

1 - use quotes around a specific phrase. You find those words in that exact order. Very helpful in narrowing down a search.

"boat engine repair" gives you a much shorter hit list than boat engine repair

2 - use the hyphen (aka dash, aka minus symbol) to remove a topic from your search. You make a search, you see a bunch of links that you can tell are not relevant, mixed in with what you want. add a -irrelevanttopic at the end of your search and search again. keep subtracting topics until your hit list looks promising. For example in the search for boat engine repair there are also places doing auto repair. I don't want that, I want someone who specializes in boats so I put a -auto at the end of my search like this:

"boat engine repair" -auto

Posted by: merelycurious on February 17, 2008 at 2:32 PM | PERMALINK

Great suggestions. As for Wikipedia's accuracy the journal Nature tested the accuracy of its articles vs. Britannica's and found that while they found 162 errors in Wikipedia they also found 123 in Brittanica. The conclusion was that Wikipedia was an acceptable source for science information. That does not prove that it is as good for other topics, but it is reassuring. And amusing.

Posted by: BernieO on February 17, 2008 at 2:48 PM | PERMALINK

A good and much-needed post, Kevin.

It is remarkable how many reporters and bloggers don't bother to do the most elementary research (which in this internet age can be done in seconds) before writing things that are false.

Posted by: captcrisis on February 17, 2008 at 2:49 PM | PERMALINK

Students trying to use Google to do internet research have a problem when their search turns up a long list of sites, some of which are meritorious and some of which are utterly crackpot. Lacking the proper background and the correct level of skepticism, they can't always make the proper distinctions between the valuable and the nonsensical. It's useful therefore to provide students with the names of a few sites. I also explain that university sites are more likely to be decent.

Posted by: Bob G on February 17, 2008 at 2:51 PM | PERMALINK

Another good way to check out reliability of a wiki is to hit the 'discussion' link at the top. There you will find a debate among 'experts' on sometimes esoteric, but often more general, points.

OT: My husband told me yesterday that Google has 'server farms' located all over the country and that together they use more energy than all the TV sets in America. He saw this info in a news article. Perhaps we could 'google' it? (gr)

Posted by: nepeta on February 17, 2008 at 2:56 PM | PERMALINK

WRT #2. If you're thinking critically, it shouldn't matter too much what an author's biases are. The only time you should care is when they are asking you to take something on the credit of their good word, and you really shouldn't be doing that, anyway, not when there's so much information out there available at the click of a few buttons.

WRT #4 Wikipedia is a useful starting point, if you don't know anything about a topic. One of the good things about it is that typically you get a variety of opinions coming from many different people who actually care enough about the subject to take their time to add to an entry. That creates problems of its own, of course, but I've generally found the biases even out, whereas if you're reading something in a standard encyclopedia written by a paid academic hack, there is just no filter at all. The best advice I can give about Wikipedia is to read the history and discussion pages of an entry if you are really interested in it, and then do your own outside research if you have doubts. Sometimes there's much better stuff in the discussion of an entry than there is in the entry itself.

Posted by: mg on February 17, 2008 at 2:57 PM | PERMALINK

Michael, Michael, Michael...

First off, Hitler is dead. So anyone who says it's raining outside is not Hitler. And if you'd lie about being Hitler, I'd suppose you'd also lie about the weather.

(people today are dumber than when I was king!)

Posted by: Hitler on February 17, 2008 at 2:59 PM | PERMALINK

Bob G: That's why we need a course on critical thinking at the high school level.

Posted by: SocraticGadfly on February 17, 2008 at 3:06 PM | PERMALINK

Nice advice. I'd have written #3, though, as follows:

If you're writing about a specific topic that you're not that familiar with, take a minute and think about whether anyone will be interested in reading a post written by someone who knows next to nothing about the topic.

Posted by: RSA on February 17, 2008 at 3:07 PM | PERMALINK

"He saw this info in a news article."

Actually, no. The Google 'server farm' article is in Harper's March issue.

Posted by: nepeta on February 17, 2008 at 3:18 PM | PERMALINK

My daughter is doing a book report on Gary Paulsen. She needed to know when and where he was born. Literally moments ago, we decided he was born of Aztlan in the kingdom of Narnia, he is 300 years old, was enrolled in Slitherin House at Hogwarts, and owes lots of money to the Starbucks that he lives in. We know it's true, because we put it into the Wikipedia (and then reverted it.)

I am hopeful she will learn more than most journalists about the wikipedia.

Posted by: jerry on February 17, 2008 at 3:39 PM | PERMALINK

Another good way to research is just to listen to what many of our reality based bloggers are saying.

It turns out that Paul Krugman is shrill, and that Hillary Clinton killed Vince Foster.

Posted by: jerry on February 17, 2008 at 3:40 PM | PERMALINK

All sound bits (pardon the intentional pun) of advice, Kevin. The Internet, like so many things, is both a blessing and a curse.

Posted by: The Conservative Deflator on February 17, 2008 at 3:45 PM | PERMALINK

Your comment about "clicking the link" couldn't be more true and there is a clear example of why that is important floating around on the web these days. It is a variant of the "Obama is a Muslim terrorist and went to school in a madrassa" e-mail blitz letter.

In the midst of its litany of claims, the variant suddenly announces that if you are wondering whether any of this is true, you should go to www.snopes.com and when you do, you will see that they have checked the facts and found them correct.

(For those not familiar, snopes is a website dedicated to deflating the kinds of nonsense "urban legend" e-mail blitzes that periodically pop up on the net such as "you have to delete this file immediately or your system will crash," or the, "a dying little boy in England wants you to send him a postcard.")

Snopes generally does a pretty good job of poking holes and that is what the senders of the variant are counting on.

If you actually go to snopes to search out the truth about these claims, they won't be on the front page when you get there. You have to do a quick search and that again is what the senders know is a benefit for them.

You have three things you can do in response to the e-mail:

1- You can say to yourself, "Well, if snopes says it is true, then it must be." And the senders win.

2-You can go to snopes...not find the story you want and immediately think..."Well, if snopes says it is true, then it must be. I'm not going to waste my time trying to look it up again." And the senders win.

3-You can actually search for a listing on snopes and lo and behold, you will discover that everything in the message you just got is nonsense. But hey...if two-thirds of those you send the mailing to believe it....and tell others....and/or forward it to their friends...then mission accomplished.

And the senders are not doing this out of ignorance....it is willful. A friend of mine got one of these and immediately replied to everyone, including the sender, that in fact the snopes citation called the claims lies.

The sender immediately replied with a note to her, dripping in venom and stating essentially that since Obama was almost the same name as Osama, then by straight logic, Obama was thereby complicit in the 9/11 tragedy. With a rationale like that, and the internet to lie and obfuscate, that's why it is important to "click the link."

Posted by: dweb on February 17, 2008 at 3:46 PM | PERMALINK


First, a little snark: The problem with teaching critical thinking in high school is that it would undercut the institutions that govern and pay for public schools -- the taxpayers, the state governments, and the teachers themselves -- because it would eventually get around to questioning the authoritarianism that tells physically adult people that they are legally, morally, and vocationally second class citizens. What institution other than the criminal court system or a jury summons can compel an American citizen to appear at a certain place and remain there against his will? We don't even have a draft anymore. I suspect that this is one reason we don't see more teaching of critical thinking in the public system -- it comes across as rebellious, and this brings out the authoritarians to protest.

That having been said, it is useful to teach science, math, history, language -- all those things -- because they provide a factual basis for lots of critical thinking. Biology is particularly useful if taught correctly because evolution is the central theory that underlies all modern biology, both structural and molecular.

I was trying to get at the concept of mature judgment: You can evaluate a web piece according to formal logic, but if the underlying assumptions are wrong, then the conclusions are also likely to be wrong. Creationist arguments have a certain logic to them because they postulate so many wildly ridiculous things, use terms like thermodynamics inappropriately, and then mix them together in a pseudological stew. It's hard to refute them properly without pointing out that there is an enormous history of geological exploration that supports the evolutionary model. In order to make this argument (and equally important, to make an informed judgment that the argument is correct), you have to be aware of the existence of these geological observations and be able to draw a conclusion that the considerable bulk of geological evidence is credible because the considerable number of geologists make it credible.

In this sense, critical thinking often comes down to making a choice as to who is credible and who is not. I just got through reading Snopes about the story that Bill Clinton has had dozens of people assassinated; if you were to read one of these crackpot claims and try to evaluate it directly, you would only have your own life experience to go on, unless you were to do the investigation yourself. Alternatively, we can read Snopes, which cites a number of studies by sources that they obviously see as credible, and then decide that the final Snopes story comes across as credible itself.

So yes, teach the beginnings of critical thinking in high school, at least to the point which allows students to recognize resorts to ad hominem argument. Just don't expect high school teachers to go out on the limb and question the system itself, don't expect them to raise doubts about religious dogma, and don't expect them to take sides in political controversies.

Posted by: Bob G on February 17, 2008 at 3:50 PM | PERMALINK

Another useful tip: You can limit a Google search to a single site by including site:abcd.com

This can be used, for example, to find all the comments to this blog by a particular commenter. For example, to find nepeta's posts you would google the following:

"by: nepeta" site:washingtonmonthly.com

(note the double quotes -- also, there is a space after the colon in "by:" but not after "site:".

The amazing thing is that it is usually much better and faster to use Google to search a specific site that to use whatever local search may be provided.

Posted by: JS on February 17, 2008 at 3:57 PM | PERMALINK

I also use Yahoo search rather than Google. Yahoo search has a feature I've found very helpful called Search Adviser (which can be turned off or on). If you're searching for a phrase and you know one or two of the words in it but aren't entirely sure of the others, if you type in the ones you know, Search Adviser will give you a list of possibilities to complete the phrase.

Typically (but not always), you'll recognize one of the possibilities as what you're looking for, and you click on that to bring up a much more focused hit list than if you'd just used the words you were sure of.

Half the trick of searching for pages on a particular topic is to put in as many words as you can think of that would have to appear in it. With Yahoo, you simply put a plus sign + in front of each word. I've found some pretty obscure pages by putting in as many as seven or eight single words or phrases (phrases in quotation marks). (And as Kevin suggests, you can narrow the search results by putting in words that would not appear on the page with a hyphen, or minus sign, in front of them.)

Posted by: Swift Loris on February 17, 2008 at 4:01 PM | PERMALINK

Also good to remember: FOX News -- fair and balanced.

Posted by: jerry on February 17, 2008 at 4:15 PM | PERMALINK

Also good to remember: FOX News -- fair and balanced.

Posted by: jerry on February 17, 2008 at 4:21 PM | PERMALINK

Also good to remember: FOX News -- fair and balanced.

Posted by: jerry on February 17, 2008 at 4:55 PM | PERMALINK

I love Wikipedia because it IS a great place to begin research. I've warned my teenager not to use it as a reference in her writing projects, but advised her it is a good place to begin her efforts.

Posted by: pol on February 17, 2008 at 4:59 PM | PERMALINK

Why can't I (or can I?) use case-sensitive search in Google etc? Really, I would often like to look for the difference between maybe "see" and an organization named "SEE" etc. Also, Google doesn't search directly for all arbitrarycharacter strings (I mean, just as typed in to be matched up exactly), so it is hard to find "Neil'" when I want to look for some posts of mine at blogs where I use/d that handle. Can I change that with tricks, are there SEs that make it easier, thanks.

Posted by: Neil B. on February 17, 2008 at 5:49 PM | PERMALINK

1. *Nothing* in WP can be trusted. The articles constantly change even if the underlying facts don't. And, the more pernicious issue is that unless you're familiar with a topic you don't know what's missing. Some of the things that are missing are due to people removing things they don't want others to know. Considering that WP turns up at the start of most searches, it's a very pernicious influence. More on that here: wikipediabias.com/about

2. I run a blog with 1000 posts on one topic, tagged with over 400 tags many of them very obscure. Many of the posts are summaries/excerpts from MSM articles, but a few involve original reporting. The site was almost immediately linked by the Yahoo directory; if anyone's tried to get in there without paying you'll realize how difficult that is. I added it to the closely-related WP entry, and, in bytes, it has 100 times as much content as that WP entry.

It stayed there for several months until someone started a crusade to remove it. I went to arbitration on it and a higher-up said it wasn't eligible.

To summarize: it's got 100 times as much raw data as the WP entry, covering the subject in very great detail and a great resource for those doing research. Yet, WP doesn't want to link to it. Details here: tinyurl.com/2d6dso

3. WP is very stingy on links, so if something's there there's a good chance that someone higher-up wants it there for some agenda-based reason or there was some sort of fight over it before it was allowed to remain.

4. WP puts nofollow tags on almost all their outbound links, meaning that those linked receive no search engine benefits, even if some parts of the WP entry are based on the sites that are linked.

5. However, some interesting sites don't have nofollow tags on their links, meaning that WP funnels all the search engine juice they get from the useful idiots who link to them into a very small set of interesting sites.

Posted by: The annoying LonewackoDotCom on February 17, 2008 at 5:55 PM | PERMALINK

Thanks Kevin. I get irritated when I forget how to use the date restrictions with Google.

Posted by: jim.rockett@mchsi.com on February 17, 2008 at 6:13 PM | PERMALINK

Also, it's interesting that so many bloggers link to WP, when WP strongly discourages linking to blogs, and when the very word "blog" is a pejorative to many there.

I won't even link to them with a nofollow tag; if I have to I just write out the URL (i.e. non-HTML format).

I used to make contributions, but I stopped for the reasons above. I've made one change there since 4/07, and it was reverted a few days later: tinyurl.com/2jmj9k

I very strongly suggest dropping all your links to WP.

Posted by: The annoying LonewackoDotCom on February 17, 2008 at 7:34 PM | PERMALINK

Mr Hitler,

I didnt feel the need to inform you, me , or anyone else that you were dead and that I knew it.
I thought the readers here were more astute than an NFL audience.

Posted by: Michael7843853 G-O/F in 08! on February 17, 2008 at 7:41 PM | PERMALINK

And use Google Scholar, as well. It's a fine research tool.

Posted by: mere mortal on February 17, 2008 at 8:43 PM | PERMALINK

I hear that books are a pretty good way to learn things, too.

Posted by: nobody on February 17, 2008 at 10:43 PM | PERMALINK

What institution other than the criminal court system or a jury summons can compel an American citizen to appear at a certain place and remain there against his will? We don't even have a draft anymore. I suspect that this is one reason we don't see more teaching of critical thinking in the public system.

Posted by: alice on February 17, 2008 at 11:07 PM | PERMALINK

If you use firefox, get to know zotero

Posted by: jason on February 17, 2008 at 11:24 PM | PERMALINK

Speaking of Wikipedia, last time I checked (a while ago, these things make me too riled up) Heather Poe, Mary Cheney's fellow right-wing lesbian issues-traitor kept deleting her biographical information from Wikipedia, either from a separate Wikipedia page (she IS a public figure), or from Mary Cheney's page.

And the rapist Alex Kelly also (apparently) keeps deleting his Wikipedia page, despite the fact that his case was a huge national story.

Tell me I'm wrong about all of this.

Anyway, these assholes' stories will someday get told, whether they like it or not. They put themselves out there, so they're going to get analyzed and discussed.

Posted by: Anon on February 18, 2008 at 12:19 AM | PERMALINK

Bob G: Snark appreciated. Of course, a good critical thinking class would go beyond learning logic and fallacious reasoning to challenge presuppositions, and the truth value of actual warrants/statements.

But, you're right.

I actually had a junior high English teacher have a section of class devoted to how English was folded, spindled and mutilated by the misleading/false/incomplete claims of advertisers. Invaluable.

Posted by: SocraticGadfly on February 18, 2008 at 12:28 AM | PERMALINK

Anon, you're wrong. There's plenty of info about Mary Cheney's sexuality on her web page.

Ditto on Alex Kelly. There's a full-blown Wiki page about him.

Please stop contributing to misstatements about Wikipedia; it ain't perfect, but it ain't a shop of horrors either.

Posted by: SocraticGadfly on February 18, 2008 at 12:38 AM | PERMALINK

Wikipedia is fine, as someone once said, if you quickly need to know the capital of Indiana, or the birthdate of a famous person. And even then you should check another source. But you should do that even with a print encyclopedia. It's not terribly different from print reference books, which are not always coordinated or crosschecked.

You're more likely to get a useful description of the binomial theorem than you are of the objective correlative in Heart of Darkness

Ha! There is no objective correlative in Heart of Darkness.

Posted by: Tim Morris on February 18, 2008 at 9:00 AM | PERMALINK

JS: tried your technique to find all the comments to this blog by a particular commenter. It works, but the awful thing is that the text displayed by Google that accompanies each link is often incorrect. In other words, Google puts somebody else's words, on the same comment thread, by your name as though you said them. And those words, which are not your words, may be obscene, stupid, whatever.

Posted by: little ole jim on February 18, 2008 at 9:49 AM | PERMALINK

Drum 2. Whenever you read something by someone you don't know, Google 'em.

2.5: Follow the money...looking at someone's campaign contributions is a good way to get an idea where they are coming from, fer instance.

Oh; use the minus sign when using teh Google to restrict the display of data, along with quotes around a phrase:

grape_crush "Washington Monthly" -"Norman Rodgers"

Posted by: grape_crush on February 18, 2008 at 1:04 PM | PERMALINK

"Brevity is the soul of wit". I think Oscar Wilde said that but I was too lazy to google it.

Posted by: leslie on February 18, 2008 at 2:31 PM | PERMALINK

little ole jim, are you describing what happens on the Google page that lists all the hits? That's true of all Google searches. The text snippets on that page can never be assumed to be related to your search -- each one comes from the vicinity of one of your search terms. In the case we are discussing, the text on the listing often comes from what follows the commenter's name -- so it is is the text of the next comment.

As always with Google, you should click on the link and then search the target page for your search terms (using the browser's search) to find what you are looking for.

Posted by: JS on February 18, 2008 at 2:53 PM | PERMALINK

Helpful tip borrowed from Crooked Timber, where a similar discussion on the proper use of Google is proceeding:

1. Type "find Chuck Norris"

2. Press "I Feel Lucky"

3. Look and learn.

Posted by: Henry on February 18, 2008 at 5:07 PM | PERMALINK

JS: yes, that's what I'm talking about. It was startling to see my name followed by an obscene quote. Unfortunately, not everybody will understand that is a Google idiosyncrasy.

Posted by: little ole jim on February 18, 2008 at 5:24 PM | PERMALINK

jim, Google will usually display the sentence within which your search term occurs. In this case, however, the commenter's name in "by: name" is not part of a sentence. Evidently what Google does in this case is to display the next sentence, which belongs to the next comment (rather than the previous one, which belongs to you). This should work well on blogs that display the commenter's name at the beginning of the comment -- which actually works better when you read the comments as well, don't you think? (You don't have to scroll down to the comment's end to see who wrote it -- though if you have to scroll it's probably scotian...).

I think it would be a reasonable request to ask Google to provide an option here (display previous vs. next sentence) and add it to user preferences. Write to them and suggest it. Explain why. I think they will read it, and you might change the course of history. (But still, others who use this search may not necessarily use this preference even if it exists).

In the future, I think we can expect smarter searches that make use of standardized formatting guidelines that all websites will use. (For example, all blog comment sites could include hidden html that indicates how they are formatted). But we are not there yet.

Posted by: JS on February 18, 2008 at 6:09 PM | PERMALINK

And while we're on the subject, always click the link. The web makes checking sources so easy that there's no excuse for failing to at least skim the primary links in an article. Click, click, click!

Hey, how did all this spyware get installed on my PC?

Posted by: Alan Bostick on February 19, 2008 at 2:05 AM | PERMALINK

For in-depth researchers, Google has some pretty sophisticated search tools - you can find information about them at the following sites:

You can also search by categories using the Google Directory:

There's also Nancy Blachman's Google Guide:

Happy hunting!

Posted by: airolg on February 23, 2008 at 12:14 AM | PERMALINK



Read Jonathan Rowe remembrance and articles
Email Newsletter icon, E-mail Newsletter icon, Email List icon, E-mail List icon Sign up for Free News & Updates

Advertise in WM

buy from Amazon and
support the Monthly