College Guide

Blog

April 03, 2013 11:12 AM Outsiders Should Evaluate Teachers as Well as Students

By Edward Glaeser

The most important determinant of educational quality is teacher quality. Yet, as a recent study of school principals’ permissiveness in teacher evaluations and a cheating scandal in Atlanta show, this performance is difficult to measure.

The best way forward is to move the evaluation of teachers outside the schools entirely, with standardized tests administered by an independent agency. This would be supplemented by classroom assessments based on unobtrusive videotaping, also judged by outsiders, including teachers’ representatives.

Researchers have long noted the power that teachers have over student test scores.

In an influential paper published in 2005, economists Steven Rivkin, Eric Hanushek and John Kain examined administrative data in Texas and found that 15 percent of the differences in students’ math scores were explained by variations in teacher quality. The difference in test-score gains between a teacher who is rated average and one who is better than 85 percent of educators generates the same improvement as dropping class size by 10.

My Harvard colleagues Raj Chetty and John Friedman, together with Jonah Rockoff, link school data with evidence on adult earnings and find that replacing a teacher “in the bottom 5% with an average teacher would increase the present value of students’ lifetime income by more than $250,000.”

Irrelevant Credentials

Teacher quality matters, but standard observable measures of teacher qualifications don’t. Research, including that cited above, typically finds that extra degrees or certificates or years of experience only marginally affect student performance as measured by test scores.

One approach is to follow standard corporate practices, giving principals the power to decide which teachers are good or bad. Good managers should know which workers are more productive, and good principals should be able to assess their educators, taking into account the ephemeral elements that can influence a classroom. Economists Brian Jacob and Lars Lefgren have found that principals are quite good at identifying which teachers produce the highest test-score gains.

But a recent New York Times article reminds us that you can’t always trust principals to use the knowledge they possess. In Florida, Michigan and Tennessee, principals were called on to rate teachers and graded more than 97 percent of them as being effective or better. Now that’s grade inflation.

Like college teachers who freely dispense A’s to unworthy students, principals want to be popular and avoid the hassles inherent in telling people that their work is subpar. In corporations, managers know that their own promotions depend on firing the incompetent. Principals know no such thing, so they take the easy way out.

Just as colleges could fix grade inflation with a simple policy of assigning a grade distribution to each teacher, requiring a fixed number of A’s, B’s, C’s and F’s, principals could be similarly required to fire the bottom tenth of their teachers each year. I suspect that such a draconian policy would help American children, but good luck getting that one through the teachers’ unions.

The standard alternative to relying on principal evaluations is to use student test-score gains. This approach also has limitations. Unions aren’t fond of evaluating teachers this way. In 2008, the New York Legislature went so far as to stop the use of test scores in teacher evaluation. Luckily, the ban hasn’t lasted.

Better Measures

Many criticisms leveled at using student test scores are refutable. We can deal with the problem that some teachers get tougher students both by looking at test-score gains (rather than score levels) and controlling for observable student attributes.

Even the best teacher can get a bad class, yet teachers can be evaluated over long enough periods to smooth out the idiosyncrasies of particular student groups. Teaching to the test may not be ideal. Yet as long as the test is sufficiently broad, it will still measure student learning. Moreover, the Chetty-Friedman-Rockoff work confirms that teachers who raise test scores also raise adult earnings.

Test scores become valueless, however, if they reflect teacher cheating rather than student achievement. Last week, 35 Georgia educators were indicted in a scandal in which seven teachers were accused of raising students’ scores by erasing wrong answers and making them right,” according to the New York Times report.

A study by Steven Levitt and Brian Jacob recounted in “Freakonomics” documented teacher cheating by looking at suspicious patterns of incorrect answers in Chicago’s standardized tests.

Teacher cheating isn’t an excuse to give up on standardized tests. It is a reason to administer them properly. Just imagine if college admissions tests were given by individual teachers rather than by the College Board. Teachers would have a huge incentive to help their favored students; the College Board, therefore, administers tests at well-monitored sites.

If the U.S. is going to use standardized tests to evaluate teachers or schools, it should pay the extra price of using an external agency, such as the College Board.

The idea of outside evaluation also makes sense when it comes to assessing the more ephemeral aspects of teacher quality. One advantage of using outsiders, rather than principals, is that independent experts would be more insulated from the pressure to rate poor teachers as effective. Those outsiders could be held accountable if they routinely gave high marks to teachers who achieved limited test score gains.

Teachers’ Involvement

A second advantage of this evaluation approach is that teachers themselves, including teachers’ unions, can be brought into the process.

Edward Glaeser , an economics professor at Harvard University, is a Bloomberg View columnist.

Comments

  • Masy on April 03, 2013 12:38 PM:

    So, fire the bottom tenth of your teachers? Who do you replace them with? New young teachers? There is a good chance that those new young teachers would be in that bottom tenth the next year. Fire them? How would this work?

    As to videotaping, schools would have to get permission from the parents of every child in every class for that to work. In Florida, for example, you rent even allowed to videotape or take a picture of a student without permission.

    Outside monitors are great in theory. I hope they would be teachers? What do you mean by 'independent experts'? And principals ARE accountable for bad teaching. If their schools don't make gains, they can kiss their jobs goodbye.

    And, ultimately, who pays for all this?

  • RSA on April 03, 2013 10:04 PM:

    Outside evaluation is a fine idea. That said...

    Principals could be similarly required to fire the bottom tenth of their teachers each year. I suspect that such a draconian policy would help American children, but good luck getting that one through the teachersí unions.

    The Bureau of Labor Statistics puts the average turnover rate in all U.S. industries around 3% for 2012 (http://www.bls.gov/news.release/jolts.nr0.htm). Requiring triple that rate for education would help American children? I suspect you're wrong about that.

  • Sad Paul Ryan on April 03, 2013 10:50 PM:

    "The most important determinant of educational quality is teacher quality."

    This is the foundational and first sentence of this essay. AND IT IS WRONG. The paper this idea comes from says that in-school factors are about 20% and of that 20%, teacher quality is about 10%.

    But don't take it from me. Two paragraphs later, we see "In an influential paper published in 2005, economists Steven Rivkin, Eric Hanushek and John Kain examined administrative data in Texas and found that 15 percent of the differences in studentsí math scores were explained by variations in teacher quality." Normally when you want to see drastic changes in something, you don't muck around with a factor that is contributing 1/6th. You work on whatever factor is overwhelming everything else in your analysis. Helpfully, that happens to be parental and student-centered factors. Interestingly enough, you can ameliorate these things, though! What happens if you fund the schools at or above parity and provide wraparound services. Roughly the Harlem Children's Zone. What happens if you pay the parents a living wage and valiantly work to ensure that students' families don't hold them back because of substance abuse, poverty, or domestic violence? Probably something more powerful and reproducible than maxing out your potential gains at 1/6th your capacity for improvement.

    But don't take it from me. I haven't studied econometrics. Or learned that 4/5 > 1/10.

  • Ron Mexico on April 04, 2013 6:59 AM:

    Spoken like a unreconstructed freshwater economist. I will happily support the policy Dr. Glaeser recommends for public primary and secondary ed., however, it should be tested first on a smaller scale. How about we test the policy first at elite institutions of higher education. Teaching goes on there, right?

  • RC on April 04, 2013 7:53 AM:

    Glaeser must be the economist in the "assume a can opener" joke. How would you choose your ten percent? What if it wipes out a whole department? Where do you get the replacements? Who would, in their right mind, enter a profession with low pay, ongoing education requirements, and high turnover? When Harvard starts doing that, and firing bad teachers as well as researchers that don't meet standards, then maybe we can talk.

  • Milt on April 04, 2013 8:54 AM:

    Where to begin. I think the first sentence of this essay indicates the fundamental problem with Glaeser's premise. He fails to define "educational quality" yet the entire essay is about just that term.

    Is a quality education so easily measurable that it can be quantified by mere test scores? Bush II thought so but after a decade replete with failures it is obvious education can not be so defined. It is no easier to identify the factors that contribute to the success or failure in evaluating an economy or a hospital patient or a missile launch.

    Secondly, Glaeser states teachers are the most important part of this "educational quality". Here again, I must disagree. The most important part of any child's education is the parent. They are the ones who instill the drive to learn. They are the ones who reinforce the concepts presented by the teacher. They are the ones who represent the child's best interests to the school administration. Yet their role is usually ignored during the current edition of the ongoing educational debate.

    As is usually the case when non-professionals debate education, teachers are identified as the crux of the problem. Yet in far too many situations today administrations look upon the classroom teacher not as a well-trained professional but as a technician on an assembly line who can easily be replaced without so much as a pause in the operation. Yet when discussing any perceived failures, teachers are seen as decision makers who have complete control over the children's education. Society needs to decide what role the teacher really plays, then and only then can they can be fairly evaluated.

  • Sisyphus on April 04, 2013 9:47 AM:

    This is, easily, the stupidest thing I've ever read on this site. Normally, WaMo is a great place to go for insightful commentary, and well-reasoned articles. This, well, this was like pouring my morning cereal out, and finding a rotting mouse carcass in it. Surprising, upsetting, and enough to make you question every other box with scrutiny and fear.

    You cannot treat education like a business and hope that it will work. The number of variables involved in a child's life is so immense, and so often outside the control of teachers, that to suggest that they can correct for deficiencies in opportunity, nutrition, family, and a general cultural view that's against education. And what does the brilliant economist think we should be teaching? Just the catalog of facts to pass the test? Of should we be teaching logical thinking, inference, deduction and critical thinking skills necessary to see what bullshit these standardized tests really are?

    I used to think it was an MBA that was the graduate degree that most clearly marked stupidity, but it seems that a PHD in economics trumps even that distinction.

  • hornblower on April 04, 2013 10:05 AM:

    I tried to make my kids teacher proof and it worked. Parents need to get more involved and stop blaming the schools. I'm sure that is how Dr. Gleaser got so smart.

  • Jim on April 04, 2013 12:22 PM:

    Glaeser's students award him a 2.5 on a 5-point scale. I can see why he urges outside evaluation.

  • NewIdeas on April 04, 2013 12:48 PM:

    C'mon Ed. You're better than this. But I'll tell you what. I'll take you up on this offer of evaluations and firings if you first agree that all schools should be funded at the state level and to equal levels (that's infrastructure spending and teacher salaries) and that there be state funded mandates that classes cannot exceed 10 students. Once we remove impossible and unequal conditions and create a level playing field for teachers and students then and only then will any evaluation have any meaning.

    While we're at it, let's apply these standards and outside evaluations to CEO's, CFO's and boards of large corporations too. If you do poorly, you get fired, no compensation, just like the teachers :-)

  • Celui on April 07, 2013 7:40 PM:

    Arbitrary dismissals? This is precisely what teacher organizations have railed against for years. A teacher evaluation instruments composed by and agreed-upon by professional teaching staff, administrators and public representatives (school board members) have proven to be effective instruments in teacher evaluation. What does an evaluator look for? How does he assess this? In what ways does he communicate his observations to the classroom teacher? These components have been part of the professional evaluation tools in the school system where I taught for 35 years. "Evaluation" is for the improvement of instruction, just as "testing" is for the evaluation of instruction. In each case, a professional teacher adjusts lessons and techniques to meet the needs of his students. Quality instruction is always the goal. 'Tenure' is the legal term which applies to fair and impartial job evaluation of professional staff by administrators/evaluators. Absent any agreed-upon evaluation instrument, a teacher has no way to know not only what he should be teaching and how, but has no way to ensure that his job performance evaluations are fair and impartial. This nonsense of annual dismissals is just what is wrong with the concept of teaching today: an industrial/business model which, when applied to a profession which is more art than anything else, cannot possibly provide for fair evaluation. A teacher is 'selling' his subject using his art and craft of instruction, but to determine his effectiveness by the number of his 'sales' is just plain nonsense. National, normed tests can provide feedback as to the strengths of a school's instruction, but cannot ever be considered a valid instrument to measure teacher performance.

  • Anonymous on April 14, 2013 2:29 PM:

    Who let the Republican in?
    Really? Assigning a certain number of A's, B's, C's and F's? As though the most important thing in a classroom is how you compare to your peers - education for learning or knowledge's sake isn't worthwhile, it's only if you learn more than the other guy that it really counts. "sorry, you actually learned everything I wanted to teach you but since Jerry over there answered one more question correctly on the test, you fail'".


    Because we've seen how well the 'for profit' schools have performed.