"...my instinct is that the study is off by a factor of about two..."

Hey, if that kind of reasoning is good enough for Stephen Colbert and Kevin Drum, then its good enough for anyone.

I think that Tim Lambert over at Deltoid has the best series of posts about Lancet II.

http://scienceblogs.com/deltoid/

LOL just pull numbers out of your hat, Kevin. Why not? Bush gets to. So does Rush, and his bff Dick.

How many hundreds of thousands of these people does it take to stain the American Republic indelibly?

The true figure grows by a calculus unknowable and is buried quietly before sunset every day--be it from thirst, disease, hunger, or hand grenades.

These things were done in our name. The statistical argument seems an evil outrage. But that's just me.

Kevin, when you apply a factor of 2, you should at least take a few minutes to jot down your reasons for that factor. If there was something so obvious, why didn't the folks who authored the study take that into account? What do you know that those meticulous statisticians and researchers in the field (Iraq) didn't know? Did their political beliefs factor into their high estimate?

Iraq Body Count has an excellent "rebuttal" of the study. Since that group is definitely anti-war and has obvious experience tracking dead people in Iraq, it is worth reading.

http://www.iraqbodycount.org/press/pr14.php

Anybody have a link to the study itself yet?

I'm interested to read it.

And 200-300,000 shouldn't consign Bushco to hell any less than 600,000+? Or at least to war crimes tribunals?

At least that 600,000 or 200,000 (whatever) didn't have to suffer in one of Saddam's rape rooms.

Democracy is on the move!

I'll take that as not worth much then. Unless you have a reasoned argument against the methodology employed, I think I'll stick with the professional epidemiologists, who, you know, do this shit for a living. take that for what it's worth.

Oh, and seeing how this was published in a peer reviewed journal, I'll take those other epidemiologists who did the reviews as having a more informed opinion as well.

Oh, and I gather you haven't read the study. But don't let that stop you from having an opinion. It doesn't stop the "there is no global warming crowd", which your comment seems to mirror in all the wrong ways.

I guess you can tell im pissed, but I do research for a living, so that kind of casual dismissal makes my blood boil.

I think I adequately disclaimered my reason for thinking the study numbers were about 2x too high. It's based on reading some pretty good, specific criticisms from both liberals and conservatives, but it's obviously not based on any serious methodology. You can take it or leave based on how highly you think of my judgment. I may go into greater detail on this later, but it's probably not worth it.

Needless to say, this has nothing to do with whether the war was justified. It's strictly a technical question — or should be, anyway. It's a big number no matter how you crunch it.

Now the President has the authority to declare anybody an enemy combatant and lock him away, and the person has no recourse.

Even if you are an American citizen.

Just great.

One question, with no malicious intention, more in good humor: Were you holding your thumb up with one eye closed when you made that estimate? Throwing darts?

And more importantly, can you provide a similar estimate for the price of oil in, say, two months?

So what?

This is NOTHING compared to how many are going to die when the neocons finally are forced to deal with Iran and NK. By the way, I suggest y'all stock up on iodine tablets.

Ah, Kevin.

So in your professional opinion, you "feel" that 300,ooo were killed in this war. Just because.

The best, credible, substantiated estimates out there, supported by the President, is around 60,000. That's over a three year period. Every life is precious, don't get me wrong, but 20,000 dead per year in a country of 30M falls well within the range of background noise. You think nobody was killed in the American Revolution?

"It's based on reading some pretty good, specific criticisms"

Well now I'm confused, because I thought it was based on "intuition" and "instinct." But in any event, exactly which "pretty good, specific criticisms" lead you to the "about two" error factor?

92% of the victims in the study had death certificates from the Iraqi Ministry of Health according to the families polled.

Great!

Now lets just ask the Ministry how many they handed out, add 10%, and there is your number.

The study included a simple means to prove if it is legitimate or say... completely full of crap.

I think I adequately disclaimered my reason for thinking the study numbers were about 2x too high. It's based on reading some pretty good, specific criticisms from both liberals and conservatives, but it's obviously not based on any serious methodology. You can take it or leave based on how highly you think of my judgment. I may go into greater detail on this later, but it's probably not worth it.

Dear lord. You should have just kept your mouth shut, Kevin, if this is your rationale. You need to back up what you're saying with stronger stuff than this.

The blogosphere may be kicking the survey around, but I've seen practically no substantive criticism of the study from experts within the field of epidemiology, nothing at least that suggests we should reduce the findings by a factor of 2.

This reminds me of the debate over the Rape of Nanking. Historians and propagandists have debated for decades over the death toll, and who to include as victims, and estimates range from 100,000 to 300,000. But mere numbers will never change the basic facts of the atrocity. Is the death of 100,000 any less horrific than the slaughter of 300,000, merely by being a smaller number?

Kevin wrote: "It's based on reading some pretty good, specific criticisms from both liberals and conservatives, but it's obviously not based on any serious methodology."

Well, rather than read criticisms from both liberals and conservatives, why don't you read criticisms from epidemiologists, biostatisticians, and demographers?

NAyone who wants to access the actual study need only go to Lancet.com. They opened it to full access, not just subscribers.

Kevin, I'm going to disagree with you for the following reasons:

Last month alone, Baghdad suffered more than 2,660 killed. Further, the August figures for that city were recently revised from 550 to 1,535 -- a threefold increase. If anyhing, this indicates that the number of killed in Baghdad alone may have been grievously undercounted.

To be fair, you should also have noted that the authors of the Johns Hopkins University study in question never claimed that their figure of 654,965 offered as "excess Iraqi deaths as a consequence of the war" was etched in granite; the authors also offered a figure as low as 392,979.

Finally, academic studies published in prestigious medical or professional journals are rigorously vetted by peers before they ever see print. Publications like The Lancet are hardly to be considered in the same class as The National Review or The Nation.

Given that, I can readily accept the notion that the numbers offered in the Johns Hopkins study aren't really that far off.

Dunno about that Kevin. I listened to an interview with one of the surveyers that seemed pretty convincing to me.

http://www.democracynow.org/article.pl?sid=06/10/12/145222

jefff: "Anybody have a link to the study itself yet? I'm interested to read it."

The Lancet Medical Journal / London, UK (October 11, 2006)
Johns Hopkins University: Mortality After the 2003 Invasion of Iraq - A Cross-Sectional Cluster Sample Study
Authors: Gilbert Burnham, Riyadh Lafta, Shannon Doocy, Les Roberts
Background: An excess mortality of nearly 100 000 deaths was reported in Iraq for the period March 2003 – September 2004, attributed to the invasion of Iraq. Our aim was to update this estimate.
Methods: Between May and July, 2006, we did a national cross-sectional cluster sample survey of mortality in Iraq. 50 clusters were randomly selected from 16 Governorates, with every cluster consisting of 40 households. Information on deaths in these households was gathered.
Findings: Three misattributed clusters were excluded from the final analysis; data from 1,849 households that contained 12,801 individuals in 47 clusters were gathered. 1,474 births and 629 deaths were reported during the observation period. Pre-invasion mortality rates were 5.5 per 1000 people per year (95% CI 4.3-7.1) , compared with 13.3 per 1,000 per year (10.9-16.1) in the 40 months post-invasion. We estimate that as of July 2006, there have been 654,965 (392,979 – 942,636) excess Iraqi deaths as a consequence of the war, which corresponds to 2.5% of the population in the study area. Of post-invasion deaths, 601,027 (426,369 – 793,663) were due to violence, the most common cause being gunfire.

The best, credible, substantiated estimates out there, supported by the President,

You do realize that the final clause is mutually exclusive to the rest of the statement.

OT but related......For those who saw on TV last week the explosions at the Camp Falcon ammo dump were pretty impressive.We could see huge blasts every few seconds and were told this went on for about 8 hours.We were told at the time that there were no killed or wounded.There has been NOTHING about this since that night.Watching it, one would think that some people had to be killed or injured. I've since read that there were about 70 Iraqi military killed and many wounded. Many Americans were reported killed and wounded. The truth? We'll never know.

I'm more than a little dubious of meta analysis by blog, but the core point is that even if the study was a factor of two too high, it still kicks the last leg out from Bush's justification for the war.

There were no WMDs, let alone nukes. There was no Al Qaeda connection. Bush's last refrain has been Iraqi's are better off without Saddam. That argument has been difficult to refute.

How is 650,000 -> 250,000 a factor of 2?

egbert: 20,000 dead per year in a country of 30M falls well within the range of background noise. You think nobody was killed in the American Revolution?

Jeez, I gotta say, I can't figure out whether this is serious.

At the risk of seeming clueless, I'll assume it is and offer the obvious rejoinder. The US has a population of 300M. Even if you accept the Administration's numbers (nobody does), the comparable number here would be 200,000 dead per year.

We've gone into a 5-year paroxysm of war, geostrategic suicide, torture, and destruction of civil liberties unprecedented in at least 50 years and arguably 150 -- over THREE thousand deaths. Can you imagine what we would have done over SIX HUNDRED THOUSAND?

Background noise? Have you been taking transfusions from Paul Wolfowitz or something?

One serious issue that comes out of this is the fact that no one is making the count easy. In fact there are some serious efforts to keep data related to the subject classified.

Not that I blame them. It wouldn't look good domestically for the Bush administration and through their rose colored glasses they probably figure the Iraqi public is not already aware.

A link to the IraqBodyCount article here.

Some comments on the value of peer review here:

The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability -- not the validity -- of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong. A recent editorial in Nature was right to conclude that an over-reliance on peer-reviewed publication "has disadvantages that should be countered by adequate provision of time and resources for independent assessment and, in the midst of controversies, publicly funded agencies providing comprehensive, reliable and prompt complementary information".

Note the author.

"It doesn't matter if the new number is a complete pantload. Any number of deaths is too high."

Repeat as necessary.

system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong.

As a grad student who has work peer reviewed and reviews the work of others, I call bullshit. I don't cheat, asshat, and resent the implication.

B - Did you get the props my daughter paid you for the math humor?

Global Citizen:

Note the author.

I saw who wrote the editorial. I can still take issue - no cows are sacred.

G.C.

You don't see a problem with the Lancet editor who put out the casualty study saying peer review is useless?

ein;
blind faith in peer-review has its problems, of course, but peer review is the best method, we, as humans have to arrive at consensus. Until someone invents a perfectly impartial artificial-intelligence.

The fact is that peer review is the process by which engineers validate the designs for bridges you drive your family over in your car, or rockets to put satellites into orbit, or probes onto other planets, etc. It is not a perfect guarantee against bias or groupthink - but what's your alternative? Ask the good folks at the Heritage foundation? Criticizing the findings of a scientific study because it was "peer reviewed" is about the most retarded and ill informed crap I've heard all night.

This is ridiculous reasoning. If I follow your example, then I should conclude that the age of the universe is something like 6 billion years old. True, astronomers put the number closer to 14 billion years. They've used accepted methods of data sampling and analysis to get that number and and the confidence region around it. They also put those claims through the rigors of peer review. On the other hand, there are these creationists who put the number closer to 10,000 years and they've got webpages with claims that seem credible to someone who doesn't know anything about physics or astronomy or statistics.

Clearly, the sober, reasonable stance is to go with a number that splits the difference. Obviously, to choose one conclusion over the other would make me seem like an extremist and we can't have that.

I just read the entire editorial, twice. Horton seems to be talking about the issue of genetically engineered foods specifically, not the scientific method and peer review in general.

I lived in Europe during the FrankenFoods protests, it was a hot-button issue. And that debate is what is addressed in the editorial. He is writing about one emotionally tinged field of study.

So I think I answered your question too, veritas.

Criticizing the findings of a scientific study because it was "peer reviewed" is about the most retarded and ill informed crap I've heard all night.

If you look through all the links, the scientific study is being criticized across the board because its conclusions don't match anything going on in the real world. The "peer review" bit is almost beside the point, and I don't know why anyone brought it up.

Add to that the obvious timing, same as last time, and you don't have a science paper, you have political hacks.

He is writing about one emotionally tinged field of study.

And the current study doesn't fall into that category? Can you think of any reason at all his concerns would not apply to ANY politically-loaded science issue?

It suddenly occurred to me that I should really get a life.

I was addressing the editorial specifically. The quote was offered as evidence that all peer-review is meaningless. I disagree. I read the editorial which is just opinion, by the way, and it was not a broad indictment, it was addressing a specific issue.

As to the study, I have not rendered an opinion. I have a copy of it, but I have not had time to pore over it. I printed out an extra copy and gave it to the head of the Statistics department at the University. I have not made any claims about the study myself, and won't until I have time to study it better and have a conversation with the department head when I have class with him on Thursday.

I am not a statistician, I am a biochemist by training. My statistics education is limited, I am taking a grad level inferential stats course now; and teaching basic stats in a community college. But I am not a statistician. (I am a scientist who has had a lot of calc.)

As to my visceral raction in defense of peer review, and I can only speak for myself here, but I do not cheat, and I hope those who review my work do not either.

Global Citizen:

Forget that peer review stuff, and whether or not the statistics meet mathematical muster.

Do the results make sense in the real world? A lot of people, if you go by the links, don't think so.

In the real world, I can see it. The deaths are not all attributed to violence. It is an across the board increase in the overall death rate. Prior to the invasion, the death rate was 5.5 per 1000. Now they are reporting a 13.3 death rate per 1000. That is a factor of 2.418 increase.

Consider that chronic conditions are going untreated, accute onsets are not attended to, diseases springing from inadequate sanitation and water supply, coupled with the violence, and yes, a doubling of the death rate is not out of line.

Then throw in the "brain drain" that has occured as healthcare professionals, doctors, nurses and lab-rats like me, have fled the country.

When I consider the facets, I can see it. I still have not rendered any judgement, but I am not dismissing it either.

Do the results make sense in the real world? A lot of people, if you go by the links, don't think so.
Posted by: bart

opinions are, of course, like assholes. why the fuck should I care what ignorant republican and sensible-dem apologists think about data that makes them uncomfortable?

or more politely ... what exactly should make me disregard the peer-reviewed findings of people trained in that field for the uninformed opinions of the (relatively) illiterate?

Global Citizen: no cows are sacred.

Well said.

What do you think about this critique? I don't mean their motives, I mean their analysis of the internal validity of their results.

http://www.iraqbodycount.org/press/pr14.php

One of the interesting implications of the Lancet study is that the pre-war death rate in Iraq was unusually low, quite contradicting the pre-war claim that the sanctions were causing extreme sickness and starvation.

The Meagan McCardle web pages were worth perusing, but most links were not detailed quantitative analyses.

I spoke with Gilbert Burnham, but he hasn't committed yet to presenting his study at the Joint Statistical Meetings. Probably someone on the team will present the study.

Matthew

This is me editorializing: I think the invisible deaths - those not attributed directly to violence are being underreported. Perhaps the deaths directly attributable to violence are being overstated, but my hunch is as valid as Kevins, and I think the hidden deaths tell a horrific tale that is not getting it's die.

One of the interesting implications of the Lancet study is that the pre-war death rate in Iraq was unusually low, quite contradicting the pre-war claim that the sanctions were causing extreme sickness and starvation.

That's another oddity. Almost every other source, including the U.N. and other NGOs, described terrible effects from the sanctions. If the study is accurate, a lot of other people weren't.

Listening to the BBC world service just now:
6 more US troopers dead (already highest monthly for 2 years), and nobody knows how many civilians. The journalist was saying "at least 100 killed in Baghdad, but possibly many more" although he gave no reason for a much higher number or for a disparity, plus the rest of the country.

Even if there was some distortion in data collection and you take the very lowest likely numbers, it is horrific.

The US government has no leg to stand on in criticizing the data as they have made a point from the start of being totally disinterested in recording civilian deaths.

Which tells you as much as you need to know about the morals, intent, and operation of this criminal administration.

"the Iraqi death toll since the invasion at 600,000+"

pretty sure the numbers bantered about from the study are not the Iraqi death toll since invasion... the number is the ADDITIONAL deaths since the invasion that otherwise would not have ocurred absent the bushliar-criminal regime's most excellent Iraqi advenure.

Setting aside Kevin’s “sweet wild ass guess” (SWAG) comment about half of the 655,000 being a better number, I looked at the Lancet’s methodology and I think it appears to be quite sound, in terms of stochastic forecasting applied to an epidemiological survey. Of course, these are all statistical forecasts and the exact number will never be known. However, it is no less sound than, say, the projected number of birth defects expected to be caused from environmental carcinogens.

The study also does not forecast long-term deaths and cancers from exposure to depleted uranium munitions, which is certain to be high. If I were you, Kevin, leave the statistical forecasting to people that know what they are talking about.

Let’s just leave it at this – The U.S. government, and George W. Bush in particular, has the blood of thousands of innocent people on his hands!

Global, my email is a fake one until I pay for the domain name. Your daughter's message must have bounced back.

. . . Just kidding. I got the message. I get about 5 minutes a day to glance at Washingtonmonthly so I'm never really able to respond before a thread takes a death spiral with two or three trolls firmly attached. I wish I had more free time like Al and Jay.

Bart: I have read a lot of analysis and anyone who disputed the figures usually falls into one of two categories:

1) The number it too high so it can't be true.

2) The authors are engaged in fraud.

I you have a substantial criticism of it then please post a link to it. However the methodology is well established and sound.

By the way, your comment on the pre war mortality rate is not correct. It was not all that low at all. I would encourage you to take your arguments over to

http://scienceblogs.com/deltoid/

They have covered a great deal of ground over there including the arguments about the pre-war death rate and the very high response rate.

Hey, I make those wacky calculations every day- y'know, maybe guess that if you saw three state patrolmen traveling the opposite way past you on the freeway, how many should you expect to be going the same direction as you are.

The key is, don't tell anyone.

As for taking the average of partisan opinions, well, if that don't beat all!

So a conservative democrat (Drum) thinks the Lancet calculations are out by a factor of 2? That means (since conservatives have gotten everything else wrong) the Lancet numbers must be right on the mark.

Kevin,

A better way to make your comment (if you improve my sentence structure):

There is a lot of uncertainty in the estimate made in the lancet study. However, they do make a good case for the number being in the hundreds of thousands. In critical reviews that I have read there are arguments (which I lean towards) that the 650,000 number might be inflated by a factor of two. I have found no good arguments that the number is inflated by a factor of five or ten.

-------
back to sleep

I think I adequately disclaimered my reason for thinking the study numbers were about 2x too high. It's based on reading some pretty good, specific criticisms from both liberals and conservatives

Gunning for Broder's column when he retires, aren't you, Kevin?

"That is, the total number of deaths is probably in the range of 200-300,000. Take that for what it's worth."

At the risk of offending you, Kevin, that's worth pretty much nothing. It is wildly inappropriate to respond to a scientific, statistically supported, peer-reviewed study with seat-of-the-pants wild guessing. The difference between Burnham et. al.'s methodology and yours is that, um, theirs exists.

Is this what pundits do in their spare time? "Oh, I know you're an expert in this field, and I cannot quite follow exactly all the details of your work, so I'm going to wildly claim that you're off by a factor of two." If you can't be bothered to do the due diligence to verify the number, do us all a favor and shut up. All you're doing is feeding the frenzy of asshats who think their gut feelings are more important than careful methodology.

As for consulting "liberals and conservatives", I have to agree with the other commenters that it would have been far more reasonable to consult with "people who know what they're talking about and other people who know what they're talking about". You act like striking a balance between "liberals" and "conservatives" should be a goal for some reason. This is a false epistemology. We would be better off if you tried to strike a balance between "statisticians" and "taxi drivers", because then we'd know that there was a chance you'd actually consulted with somebody who had more to offer than seat-of-the-pants bloviating.

I'm curious - if a medical professional says a patient has a 50% chance of surviving for two years, do you feel empowered to step in and say "I think that's off by a factor of 2." If not, why the differential in respect given to people of varying professions?

I hate to do this, but doesn't "off by a factor of two" mean something very different from "200% what it should be", which is what you're really saying?

I have to agree with those who are saying that you can't just split the difference between two partisan sides when one side is the current conservative movement. These are people who think you can debate the morality of torture, fergodssake. They think George Bush was a hero in the National Guard because 60 Minutes forgot to dot one I.

The study is based on death certificates, which is still a damned sight better than, "Higher numbers make Bush look bad, so it can't be that high."

These people are dying because Bush started a war for no good reason, and that's the only thing we need to make Bush "look bad". The rest of us are just trying to figure out how many people he's killed, and death certificates seem like a real good start.

I have 19 posts on the new Lancet study and I've tried to link to everything relevant. I haven't linked to McArdle. If you want one post to summarize the debate I recommend Rebecca Goldin's of STATS.org comments. I have no idea whether she is liberal or conservative.

Next time try to avoid going to Megan McArdale and go soem palce where they actually have a clue.

But then again it does follow that you would accuse the Lancet authors of fraud while going to a woman who would feel free to bash you head in with a 2x4.

The response by Iraq Body Count is methodological and scientific. You really should update your post to include it, especially since the Megan McCardle links says "there aren't any methodological" rebuttals.

http://www.iraqbodycount.org/press/pr14.php

Obviously, IBC is no fan of this war, and their Concluding Remarks also might be excerpted.

Oh, sweet Jesus, Kevin -- tell me you did not just 1) cite your "intuition" to dispute a peer-reviewed study and 2) cite Jane freakin' Galt -- twice, yet! -- as a source, rather than bloggers like Tim Lambert, Dan Drezner and Linsday Berenstein who have been shooting the warbloogers' criticisms full of holes.

By the by, one development I just love is how many Bush apologists are now embracing IBQ -- which notes up front that its count is certainly a lowball figure -- now that a higher (and theu even more ruinous to their rationalizations than IBC, which is plenty ruinous on its own to all but cretins like "egbert").

This is really depressing. Your instinct is meaningless, and I have a pretty good idea that McArdle didn't even read the study. If she did read it, she clearly doesn't understand the statistical concepts involved. When she tried to discuss the study over at Crooked Timber, for example, her lack of fundamental understanding was immediately obvious.

Please do not contribute to the corruption of public debate about science with such nonsense. As another commenter suggested, ignore the biased, uninformed commentary by nonscientists and ask someone who actually has the intellectual and academic foundation for commenting on the study.

Actually the primary criticism was over at dailykos by an epidemologist who is actually in Iraq.

in a nutshell: the Lancet study oversampled the most violent clusters.

further, the prewar death rate is inexact and unreliable.
(yes, the CIA World Factbook lists a similar figure. however, the UN gives a significantly higher prewar crude death rate)

Hopkins has been known to make less than perfect Public Health estimates and proclamations before. The Lancet exists to provide a forum for methods, ideas and research pieces that need to be published, but are not always of the highest quality.

We all live in a world of countered statistics. This latest declaration of 650K civilain deaths is in contrast to the 30K claimed by the Bush administration. Somewhere we all have to come up with numbers that seem reasonable to us. Too many people don't even question the numbers that are generated by "their" side.

The disparity, roughly 22x, is not all that great when you look at some of the numbers that have been swallowed whole over the past 25 years. My personal favorite was the 1980's claims of 50K murdered children in the US per year. That number was accepted for years by supposed authorities despite that fewer than 40K total were murdered, and only about 500 children per year. That was a 100x disparity between report and truth. And that wasn't even between contending factions. Neither the Bush Administration, nor Hopkins is off by anywhere near that kind of an error fact.

funny, no one here remembers the UN study which found far fewer civilian deaths (granted, this was before the worst of the violence...which is why Kevin's 200,000-300,000 figure is tenable)

the UNDP study also used cluster sampling.

the difference? it utilized 30 times as many clusters as the Lancet study. it also took demographic data to further ensure accuracy.

far greater numbers of clusters were sampled for the Congo et al....as well.

in a nutshell: the Lancet study oversampled the most violent clusters.

As usual, here comes Nathan to pose the question, am I clueless or intrinsically dishonest?

The Lancet study excluded the most violent clusters, Fallujah being a notable example.

One thing's obvious: Like the Schaivo affair, the Lancet study -- in which the basic facts are, essentially, unassailed -- is going to be subject to the same bunch of whack-a-mole bogus talking points conveyed by useful idiots or the intrinsically dishonest.

It's a little disappointing to see Kevin participating in the process and not on the side of the angels.

One thing's for sure, though -- the bottom line is, for the warfloggers, the study kicks the legs out from under their last, feeble refuge of justification, so they adopt the same tactic Kevin does -- it feels wrong, therefore, it must be.

I expect that kind of feeble logic from the likes of Nathan. For Kevin to demonstrate it is simply sad.

Based on "intuition" and "instincts" and "broad reading" you opine that that the study is off by a factor of two.

I will take it for what it is worth.

It is worthless. You need to look at the data closely, be well versed in the methodology, and know something about conditions to make such a judgement.

You might as well be a Fox reporter.

The IBC analysis pretty well destroys the actual accuracy of the Lancet study.

People focusing on the statistics and the peer-review issues are missing the point. People on the ground in Iraq, and who think the entire Iraq invasion is a travesty, find the numbers from the Lancet completely unrealistic.

All you're doing is feeding the frenzy of asshats who think their gut feelings are more important than careful methodology.

I think you meant to say 'joining'.

...my instinct is that the study is off by a factor of about two. That is, the total number of deaths is probably in the range of 200-300,000.

Golly, I feel SO MUCH better now. Whew, ONLY 200-300K senseless deaths.

Kevin,
You have claimed in the past to have a better-than-average understanding of formal statistics. If so, you know from experience that "intuition" fails 93.756% of the time when confronted with statistically complex results.

OK, I just made that number up. But at least in my personal experience 2/3 of the problems in the 5 undergraduate and graduate level statistics and probability classes I took did not have solutions anything like what "intuition" might have suggested.

The one thing the Radicals and their useful idiots have not done, of course, is point to a statistically-valid, peer-reviewed study done by an agency not controlled by George W. Bush that comes to a substantially different conclusion from the Johns Hopkins study. Because they don't have one.

Cranky

I note today that the organization that has conducted 120 polls in Iraq over the last two years reports that typical Iraqi survey response rates run greater than 97%, as compared to 12% in the Atlantic world. Either Iraqis love filling out surveys or they are absolutely desperate to get the truth out to someone, somewhere. Regardless of "intuition".

> People on the ground in Iraq, and who
> think the entire Iraq invasion is a
> travesty, find the numbers from the Lancet
> completely unrealistic.

And what percentage of westerners on the ground do you estimate are NOT having their reports filtered by the US military? Those soldiers brought in as props for the last Bush Q&A on Iraq had quite different questions when they didn't realize they were on-camera from when they knew they were being watched by their superior officers.

By the way, for anyone hoping that a reality-based approach to government or anything else will emerge from a Democratic majority in November 06, this is what you can expect for the next two years: the Radicals will just turn their noise machine up to 12 or 13 and then demand "smell-test compromise" halfway between their utterly Radical position and, well, a slighly less utterly Radial position. On everything.

Cranky

I wish someone had "peer-reviewed" Bush's Iraq folly pre-invasion. I have a suspicion it would have been rejected.

Cranky says:

"The one thing the Radicals and their useful idiots have not done, of course, is point to a statistically-valid, peer-reviewed study done by an agency not controlled by George W. Bush that comes to a substantially different conclusion from the Johns Hopkins study. Because they don't have one."

um, bullshit. as I said above, see the UNDP study.

now shut up.

...because its conclusions don't match anything going on in the real world.

Shorter Bart: "This study doesn't match my pre-fabricated notions."

Gregory:

you obviously don't know the difference between Lancet I and II.

so, why don't you actually go read the studies and then come back.

moron.

oh, and Gregory:

yes they did oversample the violent areas.

my source is here:

http://www.dailykos.com/story/2006/10/11/135644/20

Cranky wrote:

The one thing the Radicals and their useful idiots have not done, of course, is point to a statistically-valid, peer-reviewed study done by an agency not controlled by George W. Bush that comes to a substantially different conclusion from the Johns Hopkins study. Because they don't have one.

Nathan rejoineded: as I said above, see the UNDP study.

But wait! In the citation above, Nathan issued the caveat: the UN study which found far fewer civilian deaths (granted, this was before the worst of the violence.

So, by Nathan's own admission, the UNDP study -- assuming, of course, that we can take Nathan's word for it, which is always a suspect proposition -- is a "substantially different conclusion" of apples to oranges.

Advantage: Cranky. Good Ford, Nathan, I weep for your clients.

Nathan: Cluelessness or inherent dishonesty? Like the Bush Administration he shills for, there's no need to choose!

I'm unclear what the argument is about. Either >60K senseless deaths have occurred from this Bush-induced international disaster or >200K senseless deaths have occurred from this international travesty. Either way it's a crime against humanity.

Gregory:

compare the UNDP study to the figures for 2003 and 2004 in the Lancet study. that would be oranges to oranges.

Idiot.

Nathan on October 18, 2006 at 12:01 PM,

thanks for the link.

Tim Lambert on October 18, 2006 at 9:15 AM

thanks for the link.

To all the rest: after you read the Lancet report (instead of the news accounts), and after you read the criticisms (instead of the institutiional affiliations), then you might discuss the pros and cons. If the Lancet results are correct, then all of the irreconcilable oddities pointed out in the IBC critique are also correct.

A little bit on science. Once a peer reviewed study is published anyone has a chance to call into question the study. Normally the person will first contact the authors with their question to see if it can be explained adequately. If there is no adequate explanation, the critic can then write a letter to the journal asking for an explanation. If the criticism appears to have merit, the journal will publish the letter. The authors will usually include a reply defending their study after the criticism. If the authors cannot defend, then the authors may need to issue corrections or in extreme cases the journal will repudiate the article.

The other approach is to try to replicate the study to see if you get the same results.

That is how science works and until the process is allowed to run its course, I (as a scientist) will assume that the research is tentatively acceptable.

I wish to add my thanks to Tim Lambert for the link provided.

From the link to Stats.org that Tim Lambert provided:

While the Lancet numbers are shocking, the study’s methodology is not. The scientific community is in agreement over the statistical methods used to collect the data and the validity of the conclusions drawn by the researchers conducting the study. When the prequel to this study appeared two years ago by the same authors (at that time, 100,000 excess deaths were reported), the Chronicle of Higher Education published a long article explaining the support within the scientific community for the methods used.
President Bush, however, says he does “not consider it a credible report” and the media refer to the study as “controversial.” And even as the Associated Press reported mixed reviews, all the scientists quoted in its piece on the “controversy” were solidly behind the methods used. Indeed, the Washington Post points out that this and the earlier study are the “only ones to estimate mortality in Iraq using scientific methods.”

I wonder if we will get an entirely new set of Radiacal trollnames/personas and attack memes on the liberal blogs after the November elections? The old "how's that goin' for ya?" schtick is going to sound a bit hollow I suspect.

Cranky

Possibly some of the problem comes from the pre-war death rate they're using. 5.5 per thousand per year, right? But how accurate is that number? The US death rate for 2003 was 8.41! Yes yes, we have crime to deal with, but are the authors really asserting that the death rate in pre-war Iraq, with all of its shortcomings in medicine, child nutrition, and violent government is really 40% lower than the US? (For smell testing purposes, deaths due to firearms and violence are less than 1% of the US total, so they're not seriously weighting the scale.)

On the other hand, Iraq may have a relatively low death rate just because of its demographics - i.e. the median age there is damn near 10 years lower than the US. Compare to Saudi Arabia, with a published death rate under 3 and a median age under 22... Also Japan and the UK, which have higher death rates than the US and also have higher median population age (though not by much in either case).

So realistically the death rate in Iraq was pretty high to begin with, given their huge preponderance of kids in the population (40% of Iraqis are 14 or younger!) Still don't know if the pre-war figure is accurate, however... and if the death rate is so much higher according to Lancet, why is the published death rate still around 5.5?

Jordan: 92% of the victims in the study had death certificates from the Iraqi Ministry of Health according to the families polled.
...
Great!
...
Now lets just ask the Ministry how many they handed out, add 10%, and there is your number.

This is a good point, harped on by the IBC analysis of the Lancet study. IF the Lancet study is accurate, then the method proposed by Jordan will be accurate.

I am unable to find the link to Nathan's cited UNDP study. Probably I am skimming too fast. If one of you found it, could you post it here, please?

Yelling in the fog: By the way, your comment on the pre war mortality rate is not correct. It was not all that low at all. I would encourage you to take your arguments over to

If the Lancet study is correct, then the pre-war death rate was pretty low.

Global Citizen, did you also give your teacher the quantitative critiques of the Lancet study, such as the IBC critique? The IBC claims that the number of death certificates claimed by the Lancet paper to have been recovered exceeds the number of death certificates actually issued by the Iraqi government, by a large margin. It's not likely that anybody here in the states can resolve who is correct on that point.

Matthew

that Lancet study that pegged the Iraqi death toll since since the invasion at 600,000+,

my instinct is that the study is off by a factor of about two. That is, the total number of deaths is probably in the range of 200-300,000. Take that for what it's worth.

Sorry, Kevin, if your reaction to an epidemiological study that uses established methodology (of a quality that would have you nodding your head in sheep-like acceptance if it were about cases of the flu in Southern Alabama, instead of about your deeply embarrassing invasion), a study that estimates a 5% chance that the figure is less than 400,000 due to random variation, and a 5% chance that it is greater than 800,000 for the same reason, as "pegged"...

...then your instinct is worth nothing. I don't mean to be harsh on you just for the hell of it, but if you lack even that much of an understanding of statistics, you just aren't fit to be discussing the subject.

the UNDP study is here:

http://www.iq.undp.org/ILCS/PDF/Analytical%20Report%20-%20English.pdf

see page 54.

for the record, the UNDP study (which also used cluster sampling)....had a much greater and more extensive sample size.

further, one can simply compare their count to the count in Lancet II up to the conclusion of the UNDP study. that would be an oranges to oranges comparison, Gregory to the contrary.

derek: um, try reading the actual substantive criticisms of the study.

the problem isn't methodological.

until you understand that you're attacking a strawman.

I have not seen him to pass on the critiques. I suppose I could email him the links today so he will have them for class tomorrow.

Kevin, that statement "my instinct is that the study is off by a factor of about two" is about the dumbest thing you've written.

Seriously.

My instict is that it is off by a factor of 7-10.

My intuition is obviously no better than anyone else's here, but based on a broad reading of all the discussion on both sides, my instinct is that the study is off by a factor of about two. That is, the total number of deaths is probably in the range of 200-300,000. Take that for what it's worth.

Christ, what a stupid, moronic thing to say. No one has posted any sort of scientifically valid refutation of the study, and yet your "instinct" is that it's off by a factor of two.

Well, my instinct is that's its off by a factor of 100 -- deaths are closer to 60,000,000. What's that, you say, that's larger than the entire population of Iraq? Yes, true, but how can you place facts and numbers against my instinct?

Every life is precious, don't get me wrong, but 20,000 dead per year in a country of 30M falls well within the range of background noise.

And yet 2,9000 dead at the World Trade Center in a country of 300 million is such a national trauma that we must shred the Constitution to prevent it ever happening again....

My intuition is obviously no better than anyone else's here, but based on a broad reading of all the discussion on both sides, my instinct is that the study is off by a factor of about two.

Everyone has already jumped on Kevin for this, but I'll add 2 comments:

1. If honest, this means that he thinks the U.S. forces kill rate has averaged about 78 per day, day in and day out for the duration of the occupation, based on his previous calculation of 155 per day. I think he ignored his own computations when he wrote the quoted line.

2. In one sense, we all have the same problem with pre-war excess deaths caused by the U.N. Sanctions. Published estimates for the excess deaths caused by the sanctions ranged from 8,000 per month to 36,000 per month, for either the 12 year duration or just the last few years. But nobody knows because the Baathist regime manipulated the figures and controlled access to the country. For all of us, our assessments of the pre-war costs of the sanctions entail some such gut feeling in response to the inadequate published records.

Now Stefan, you only live in New York. That day was terrible for us in the heartland. Why we had to watch those towers fall over and over again in an endless loop on Faux News. You just can't imagine the trauma we suffered...

"Christ, what a stupid, moronic thing to say. No one has posted any sort of scientifically valid refutation of the study, and yet your "instinct" is that it's off by a factor of two."

oh bullshit.

obviously you haven't been reading.
no one is asserting a methodological critique per se. rather the data collection is being criticized.

I have given numerous such examples above.
including a link to a far more exhaustive UN study (using the same methodology) which found a much smaller mortality rate.

Now Stefan, you only live in New York. That day was terrible for us in the heartland. Why we had to watch those towers fall over and over again in an endless loop on Faux News. You just can't imagine the trauma we suffered...

For the past month or so they've been blasting into the bedrock here at the WTC pit outside my window, in preparation for putting up the "Liberty" Tower, so every few hours I'm interrupted by a long loud whistle and the "crump" of a deep explosion. It's doing wonders for my nerves.....

Possibly some of the problem comes from the pre-war death rate they're using. 5.5 per thousand per year, right? But how accurate is that number? The US death rate for 2003 was 8.41! Yes yes, we have crime to deal with, but are the authors really asserting that the death rate in pre-war Iraq, with all of its shortcomings in medicine, child nutrition, and violent government is really 40% lower than the US? (For smell testing purposes, deaths due to firearms and violence are less than 1% of the US total, so they're not seriously weighting the scale.)

Let's look at the US govt's own numbers for mortality rates of other countries in the region (courtesy CIA Factbook): Syria is 5.0, Iran is 5.6, Turkey is 6.0, Egypt is 5.2, etc. Iran's prewar rate of 5.5 which was used as the basepoint for the study seems, therefore, to be squarely in the middle of what one would expect.

Why would the region have a lower death rate than the US? Largely because it has a very young population, with a larger percentage than in the US of people 25 and younger, and of course the younger the population the lower the death rate.

Stefan: No one has posted any sort of scientifically valid refutation of the study,

Links to scientific criticisms of the study have been posted here. You might in the end judge the Lancet study to be robust against those critiques, but to claim taht there are no scientifically valid refutations is just not correct.

Stefan:

the UN gave a higher pre-war mortality rate for Iraq.

but anyway, that's not the substantive criticism.

the substantive ones can be found in the dailykos diary I linked to above (written by someone who knows far more about this topic than the rest of us) and in comparison to the UNDP study which I linked to above.

your failure to even grapple with those (as well as the disappearance of Gregory) is telling.

papago:

he's right in the narrow sense that no one is critiquing the methodology of the study per se.

rather, the substantive critiques deal with the study's data collection format...as well as its failure to norm for demographic data

Nathan, the phrase I quoted from Stefan was "scientifically valid". The critique by the IBC is certainly "scientifically valid"; it lists many of the logical/mathematical consequences of the Lancet study results, and points out how extraordinary all those implications taken together are, including the implication that more death certificates can be found than the Iraqi government actually has issued.

Good Ford, Nathan, I haven't disappeared.

Nathan, might I suggest that you take your argument over to Deltoid.

rather, the substantive critiques deal with the study's data collection format...as well as its failure to norm for demographic data

Nathan, given your vast knowledge of statistics how would you improve on the study's data collection format, as well as its failure to norm for demographic data? Be specific, please, and please show your work.

your failure to even grapple with those (as well as the disappearance of Gregory) is telling.

Nathan, I can understand that an obviously piss-poor debater such as you can take plenty of time from his struggling legal practice to post your intellectual dishonesty here, but the fact is I was working.

As for your so-called point above, I can only chuckle again that you admit the UN study discounts the past several violent years but then cling to it as an example of a lower death count.

And I'll point out that, despite the warflogger's very, very recent rush to embrace Iraq Body Count as a less embarrassingly shameful death toll, even it is unacceptable.

You're just lucky, Nathan, that Kevin pulled an even bigger boner than you usually do, to spare you your usual helping of embarrassment at your shameful combination of idiocy and dishonesty.

If you had any clients, I'd weep for them.

D'Oh, er... you use more cluster points and fewer interviews at each one? And also take demographic data, so that you can compare your sample to Iraqi norm, and thus if someone asks you "you sure you didn't talk to an oddball sample?" you can say "yeah, here's the data"?

Sounds easy, costs more, naturally. Though if news networks can run a poll with three times that many clusters, and the UN uses many more than that, we can tell that the difficulty isn't insuperable.

I'm not myself convinced that the lack of demo data had anything to do with the willingness of the providers to provide it - if these people are digging out death certificates, is it gonna kill them to say how old they are?

But yeah, the extraordinary number of death certificates points to something weird. Either there's been that many death certificates actually issued (in which case, why bother using statistical sampling... just find out how many certs were issued!), or the sample is way atypical in how many of the issued death certificates ended up in their possession. And since getting a death certificate is HIGHLY correlated with dying...

Finally, I think the key point is that "extraordinary claims require extraordinary evidence". If this study had found there were 50k dead instead of 600k, nobody would be too worried about shortcomings in the methodology. But the claim is -way way way- larger than anyone else, so if they want people to take it seriously, the methodology had better be damned well bulletproof.

By the way, I do urge everyone to read the Kos post, and judge for yourselves if it's as declarative as Nathan makes it out to be.

"extraordinary claims require extraordinary evidence"

Unless your itching to invade Iraq.

For the past month or so they've been blasting into the bedrock here at the WTC pit outside my window...

Stefan, as a New Yorker in the middle of this, what is your feeling towards the non-construction of any tower, memorial (other than the faux temporary water pool while Bush visited) etc in the past 5 years at ground zero? Also, did you ever see Keith Olbermann's scathing comments regarding the 9/11 anniversary and Bush? Just curious.

Gregory:

did the UN study find much lower mortality figures for 2003 and 2004 than Lancet II did? or did it not?

put up or shut up.

Stefan, as a New Yorker in the middle of this, what is your feeling towards the non-construction of any tower, memorial (other than the faux temporary water pool while Bush visited) etc in the past 5 years at ground zero? Also, did you ever see Keith Olbermann's scathing comments regarding the 9/11 anniversary and Bush? Just curious.

You know, I'm highly conflicted. As I've mentioned before the WTC pit is the view out my window, and I walk directly by the site twice every day. And yet despite my constant visual exposure to it I still don't know what to do with the site.

I used to think we should just rebuild the damn towers with one story added, but then of course I came to realize that no one would really want to work in them. Now I'm sort of the view that a nice little park would be the best use of the site -- a sort of miniscule Central Park downtown, with fields, trees, a little pond, etc., that could be tied into the greenbelt running from Battery Park up the Hudson River. A park like that that would provide a living breathable space for those of us who live and work down here might make the best memorial of all.

As to Olbermann's comments, though, I'm not at all conflicted -- they were dead on. I cheered.

ckelly: "extraordinary claims require extraordinary evidence"
...
Unless your itching to invade Iraq.

Well, yeh. That required 12 years of an air-ground shooting war, deaths from sanctions, the civil war in 1991, 17 UNSC resolutions pointing out that Iraq was in violation of the cease-fire terms from the declared war of 1991, a 1998 Congressional resolution aiming at the regime change in Iraq and then the 2002 Congressional resolution authorizing force to achieve the regime change -- to go along with the nearly unanimous international consensus that Iraq still had the non-existant WMDs. This review is a reminder that there is still even less compelling reason to do anything about Darfur.

The associative jump to Darfur is beyond the scope of this thread, I admit. maybe after 12+ years of relentless bi-partisan demands to do something about Darfur the US might act. It took relentless demands by mostly Democratic advocates before the US acted in the Balkans as well. I don't mean to make a partisan point here, only that the US acted in response to "extraordinary" conditions before the invasion of Iraq, not a one time cluster sample.

Riverbend
is back after two and a half months, and she addresses the lancet study.