Facebook
Twitter
Ten Miles Square
Blog
I spent the last two days on jury duty in the District of Columbia. (Whatever the broader shortcomings of DC municipal government, their process for hauling you into the jury pool every two years works with uncanny efficiency; watch this space for jury-related blog posts in early May 2014.) It was a DUI case, and, sidebar, before we talk about the need for statistics education, let me say this: If it at some point in your life you decide to spend a long Tuesday evening partying at the home of a friend / business associate known only as “Cesar,” and after the conclusion of said partying you elect to get home by driving 60 miles per hour through Rock Creek Park at 3:00 AM, and you get pulled over for speeding, and you fail the various standardized roadside sobriety tests, and get arrested for DUI, and rather than take a plea deal decide to avail yourself of your constitutional right to a trial by a jury of your peers, and when the time comes for you to testify under oath your attorney–your attorney–says to you, “Can you tell the jury how many beers you consumed that night, while you were partying with Cesar,” it may not be in your best interests to answer, “Nine, if you count the two I had after midnight, before getting in my pickup truck.” Just saying.
So earlier in the trial the arresting officer testified about the aforementioned standardized field sobriety tests, or “SFSTs.” Under cross-examination by the defense attorney, he testified that people who fail the first of the three standardized tests, the “Horizontal Gaze Nystagmus,” where they make you track a pen moving back and forth in front of your face, are 77 percent likely to have a blood-alcohol level over the legal limit of .08, and that when the test results are combined with the two other SFSTs, the percentage goes up to 83 percent. (According to the National Highway Traffic Safety Administration, the first test actually “allows proper classification of approximately 88 percent of suspects” and gets up to 91 to 94 percent in combination with the others, so the officer was understating his case.)
During closing arguments, the defense attorney focused on those numbers. 77 percent, he said, was like “C-plus” on a test in school. Not an “A,” not even a “B,” but a “C,” and if we’re going to expect our children do better than that, then by God shouldn’t we expect the same from our police?! It was kind of like this, without the funny.
The problem of course is that the percentage of questions you get right on an Algebra quiz and the statistical likelihood of one thing being correlated to another thing are two very different things. The defense attorney either didn’t know that, or assumed the jury didn’t know that, but either way it points to a terrible statistical illiteracy in the general populace. Which is not surprising, given that statistics isn’t part of the standard curriculum schools require students to complete in order to get a high school or college diploma. Math education is still largely interpreted as a progression through Algebra and Geometry to Calculus. And I’m not against working harder to improve math education. But in terms of things you really need in order to make your way in modern society, statistics is way, way up there, above a lot of things that are currently lodged in the curriculum. Fitness for jury service is one of the few tangible requirements of citizenship–New York relied on it in part when evaluating the adequacy of the state’s funding programs. Everyone should learn statistics. Otherwise slightly wiser (or less honest) drunk drivers will be able to menace the highways with impunity.
[Cross-posted at The Quick & the Ed]
Brian Gulino on May 11, 2012 6:50 PM:
Teaching statistics is one thing.
Teaching people how and when to apply them is quite another.
You get the "this is math" glazed eye look, and "I'll memorize the formulas and hope for the best" strategy, from 60-70% of the students. That's what its like in my college anyway.
RDF on May 11, 2012 7:02 PM:
Couldn't agree more. I'm a scientist, and upon entering industry it was apparent that learning probability and statistics was necessary to further my career. This knowledge has opened many doors -- but I wonder why it was not a basic requirement of my high school education, like logic and economics. Distributions, probability and analysis could easily be covered in one semester.
Sad, but ask most people what they think of statistics, and you'll get a quote from Mark Twain.
Vlad Tepes on May 11, 2012 7:07 PM:
Teach statistics in high school and maybe students will learn it as well as Algebra & Geometry!!! (i.e. not at all)
POed Lib on May 11, 2012 7:13 PM:
AARRRGGHHGHHGHG!!!! What a RIDICULOUS column.
As a statistician (I deal with issues of diagnosticity of markers on a daily basis), I thoroughly disagree that the defense attorney is required to tell the truth. He makes an argument, and the argument is about HOW to INTERPRET the likelihood in question. It is interesting that the likelihood is not 99%, just in the 70s. It is the duty and job of the prosecutor to fully educate the jury.
We have no idea if the defense attorney does or does not understand this likelihood. His job is to make an argument, and arguments are neither true nor false - they are interpretations of the evidence.
Jack on May 11, 2012 7:15 PM:
So Professor, care to explain what exactly the attorney said that was wrong? Because right now your explanation doesn't even merit a C.
anon on May 11, 2012 8:40 PM:
I am not sure this is the best example of statistical illiteracy (a defense attorney has to do the best he can with what he has available, and when the facts are against you, your argument is going to be pretty thin). But I agree that it would be great if we could improve the statistical literacy of the populace.
I am not sure that the vague suggestions made here at curriculum reform are very practical. It would indeed be great if all high schoolers could get educated in practical things like the basics of statistics. But I would argue that one of the main symptoms of our population's statistical illiteracy, is the impression that this would be possible for today's teachers to accomplish if only they weren't too busy teaching things like algebra and trigonometry. This is the implicit assumption of people who lament the fact that statistics is not taught everywhere, without acknowledging the reality-based reason why it is not taught everywhere.
In fact, it is _much_ harder to competently teach statistics than it is to teach algebra. Teaching even the simplest statistical notions correctly is immensely more difficult than teaching (say) how to solve quadratic equations. And we all know what luck our teachers and students have with that.
It is much harder to notice when a teacher or student is making statistical errors, than it is to notice when a teacher or student is making high school algebra errors. So it is easy to have an incoherent understanding of statistics (as a teacher or as a student) and not know it. It is much harder to be bad at algebra and not know it.
It is fantasy to imagine that that teachers and students who struggle with algebra will somehow be able to master the basics of statistics--- which *even at its simplest* combines basic algebra with rather thorny and subjective questions of judgment and interpretation.
I will repeat myself. Statistics is much harder than algebra. A student who struggles with algebra will struggle with statistics. People like to imagine that statistics, being less dry than high school algebra, and less abstract than high school algebra, and more connected to the "real world" than high school algebra, must somehow be easier to teach and learn than high school algebra. This is not the case. Serious discussions of curriculum reform must begin with an understanding that this is not the case. A terrible algebra teacher will be an even worse statistics teacher, and students who struggle with algebra will struggle with statistics. Unless, of course, the curriculum is so watered down that nobody teachers or learns anything of any use. That is what we probably have to look forward to.
John Q on May 11, 2012 8:48 PM:
So if there's an 83% likelihood of the accused's blood level being over .08%, then isn't there a 17% chance that it wasn't? And wouldn't that be enough for "reasonable doubt" in a criminal case?
I'd say it constituted probable cause for an accurate blood measurement to be taken - but enough to convict?
Nah!
huh on May 11, 2012 10:06 PM:
"The problem of course is that the percentage of questions you get right on an Algebra quiz and the statistical likelihood of one thing being correlated to another thing are two very different things."
Actually, they're not; both are probabilities and the attorney's suggestion that a .77 success rate is unsatisfactory is a reasonable interpretation of the data; one that I (PhD, Statistics) happen to agree with.
It sounds like the author of this column actually needs a good lesson in probability theory and statistical inference.
Aalto on May 11, 2012 10:52 PM:
huh:
I feel I have to disagree with you here. Perhaps you can correct me if I'm wrong, but I don't see how the 77% on a test is in any way a probability. It's simply a fraction of correct answers (which may be loosely equivalent to a probability of getting any given question right, but only if the test has uniformly valued and similar questions). Whereas the sobriety test *is* giving a probability of a true positive, with an equivalent 23% chance of a false positive (no indication about true/false negatives here). It may be reasonable to interpret 77% as below "reasonable doubt" (hence combining it with the second test), but I don't understand your argument that they're both probabilities.
Unwisdom on May 12, 2012 12:29 AM:
As a statistician, I agree that an improved emphasis on statistical reasoning would be a significant improvement in the American high school education system. And I would disagree with anon, above, that statistics is intrinsically harder than other high school mathematics subjects. I think that you can engage in reasonably advanced statistical reasoning and thinking without encountering the intrinsic challenges of a geometry, trigonometry or calculus class.
But I have to admit that I am not really sure how this idea relates to the thrust of the article. Was the defense attorney arguing that taken in isolation, a field sobriety test was insufficiently reliable to form the grounds for a conviction? That sounds like a reasonable point to me. Was his point something else? Then what?
On a related point, both the expert witness, the defense lawyer, and you seem to be falling into the same statistical fallacy. The field sobriety test, you say "“allows proper classification of approximately 88 percent of suspects”. But what does this mean?
Considered in the simplest way, there are really two important quantities here; the sensitivity and the specifity of the test. The sensitivity is the probability that someone with a blood alcohol level above 80mg/dL (and could I just add, as my own sidebar, that even if you wish to supplant calculus with statistics, it is still important to state your units!) fails the test. The specifity is the probability that someone who has a blood alcohol level below 80mg/dL passes the test.
The point here is that the actual probability of the test being correct is a weighted average of the sensitivity and the specifity, where the weights are the proportions of people with blood alcohol levels above and below 80mg/dL among the "suspect" population. Therefore, (unless the sensitivity and specifity are equal - and there is no reason to think that they would be) the accuracy rate, 77%, 83%, 91%, whatever, depends in large part on how good the cop is at only testing people who are actually over the limit.
Absent this information, the "accuracy rate" is a largely meaningless statistic!
Walt Slocombe on May 12, 2012 8:18 AM:
You don't have to know anything about how statistics are calculated to understand that there is a difference between exactly 77% of answers being correct and 23% wrong (the test/grade case) and the sobriety test geting a correct answer only 77% of the time. And the lawyer was making a legitimate point about the relatively high wrong results and -- I would maintain -- also legitimate in making a homely analogy to grades -- 77% ain't very good. The problem, if there is one, is logic -- and logic should be easier to teach than Chi-Square.
Statistician on May 13, 2012 5:18 AM:
The problem with teaching statistics early is that the mathematical underpinnings are really, really complex. It's generally not as easy to teach just how to interpret statistical results as one might think, which I've learned from being a TA in stats at a public administration school. Even something as simple as looking up a t-test doesn't make sense if you don't understand the math behind it.