College Guide

Blog

April 20, 2012 3:39 PM Be Careful What You Measure

By Daniel Luzer

One of the more frustrating aspects of writing about higher education is the bizarre lack of measurable data about what colleges actually produce. People, after all, go to school to learn. Doesn’t it make sense for us to have some idea how much smarter people get in college?

Perhaps. And so now there’s a push from some quarters to measure collegiate outcomes. One of the more popular policy ideas is to measure and assess schools based value-added assessments, how much extra students learn in the time they’re in school. The idea seems reasonable, but it probably won’t work.

David Brooks today offers a compelling, though ultimately misguided, push for value added assessments. As he writes he writes

This is an unstable situation. At some point, parents are going to decide that $160,000 is too high a price if all you get is an empty credential and a fancy car-window sticker.
One part of the solution is found in three little words: value-added assessments. Colleges have to test more to find out how they’re doing.
There has to be some way to reward schools that actually do provide learning and punish schools that don’t. There has to be a better way to get data so schools themselves can figure out how they’re doing in comparison with their peers.

There are two problems with this idea. First, value-added assessments of collegiate learning don’t really measure what people what to know about college. Second, there’s no way an assessment, even if it’s wonderful, would do much to make college cheaper.

The biggest value-add in higher education isn’t provided by America’s fanciest colleges. It’s community colleges, which start off with really low scoring students and turn them into reasonably well educated people, that win here. Princeton, or the University of Missouri, start off with reasonably well educated people and turn them into then reasonably well educated people with a slightly more sophisticated understanding of the way the world works and a few more crucial skills that might be useful for getting a job. Those are really important things; that’s why people go to college.

But the added value, the learning gains, of Raritan Valley Community College are going to be much higher than the learning gains at Princeton. Is Raritan Valley a better school?

Now, it would be perhaps be useful to know which colleges do a really great job with low performing students and which don’t, but once we start to look at any somewhat selective schools, all of this information is going to be kind of negligible, and more a function of student effort and the types of classes than the institutions themselves. Does that mean they’re not valuable? No, this doesn’t mean such schools are worthless; it’s just that value-added assessments aren’t very good at measuring what they do well.

Furthermore, the value added component here doesn’t address the real problem Brooks presents: college is too expensive. As I’ve pointed out before, the reason people go to college is because it’s become essential to obtaining a professional job. As trends are moving now in American labor, that credential will continue to be essential, no matter how high tuition goes.

The college bubble is often compared to the real estate bubble, so let’s use a real estate analogy. If I want to buy a house as an investment it would perhaps be best to buy a reasonably cheap house in an up-and-coming neighborhood. And so it would make sense to have information about demographic trends in various neighborhoods and real estate trends in the city where I live, not to mention the structural integrity of the house and what improvements I might have to make

This information is certainly valuable, but if I earn $25,000 a year and have no savings, it’s all irrelevant, because I can’t afford to buy a house. The policy we want to move toward in higher education is one where more people, to extend this analogy, own their own houses. The way to do this (think of the country in the aftermath of World War II) is to build cheap housing and heavily subsidize that housing so that more people can afford it. Providing more information about things people can’t afford doesn’t make it cheaper.

I commend Books for making the point about learning, and for suggesting that colleges should make some effort to demonstrate their effectiveness, but this just won’t have the effect he would like it to.

It might not matter though. While measuring effectiveness might seem like a good way to monitor and improve higher education, that might not really be the point.

Many states are now working on efforts to monitor and sanction colleges based on their “effectiveness”; most of these efforts to get college to prove their worth is driven not by a desire to improve education, but just to cut costs. Or to find a reason for states to use to justify taking money away.

Daniel Luzer is the news editor at Governing Magazine and former web editor of the Washington Monthly. Find him on Twitter: @Daniel_Luzer

Comments

  • boatboy_srq on April 22, 2012 12:23 PM:

    While I agree that Brooks' reasoning won't produce a system which provides meaningful metrics, there's a very good reason to go along with the value-added criterion. Most of the best value-add schools would be, as you say, community colleges (and right behind them the state universities). These are the institutions that public policy is hurting the most right now: budgets are being shrunk, tuition hiked into the stratosphere, faculty and staff laid off, syllabi either rewritten for philosophical reasons (think Intelligent Design) or shredded by funding cuts, and admissions policies drastically rewritten to exclude many of the most in need of higher ed. It would be instructive to see the pols' (and the public's )reactions to the knowledge that the schools ranked best in the value-add results are the ones they're intent on wrecking.

  • Sgt. Gym Bunny on April 23, 2012 12:31 PM:

    Princeton, or the University of Missouri, start off with reasonably well educated people and turn them into then reasonably well educated people with a slightly more sophisticated understanding of the way the world works and a few more crucial skills that might be useful for getting a job. Those are really important things; thatís why people go to college.

    I would argue that it's not really the "learning" that goes on at college that makes certain graduates more employable in the high-paying job market (therefore, justifying the high price tag). Maybe my inner cynicism or just hearing many a conversation about--ahem--grade inflation at certain Ivy League institutions, or perhaps dealing with attorneys who were as dumb as a sack of hammers, but I don't think that how much students "learn" is a going to do much for the broader discussion on college affordability.

    Considering the growing cost of higher ed, the learning factor is still low on the totem poll of desirable qualities that college typical affords. True one could learn a lot at Colleg A, but if College B provides more networking to industry VIP or its prestigious name just looks good on a resume even if you majored in turf science, then what/how much you learn is a moot point. Cheerleading for "learning" is just one spoke on the wheel of the Great Misunderstanding of (Academic) Merit: that people are financially successful because they had the most learning.

    But, I do agree that the learning metric is a good assessment tool for highly subsidized community and state colleges. When you take cost out of the analysis--rather like we do when we compare public schools--it is useful. But for the purpose of evaluating the financial cost/benefits of college, in general, "learning" will, unfortunately get the back-burner treatment. Most employers simply don't look as hard for minimum GPAs on resumes as they look for extracurricular/leadership/internship/actual work experince, etc.

  • Anonymous on January 14, 2013 11:29 AM:

    "The biggest value-add in higher education isn’t provided by America’s fanciest colleges. It’s community colleges, which start off with really low scoring students and turn them into reasonably well educated people, that win here. Princeton, or the University of Missouri, start off with reasonably well educated people and turn them into then reasonably well educated people with a slightly more sophisticated understanding of the way the world works and a few more crucial skills that might be useful for getting a job. Those are really important things; that’s why people go to college. " very much agree on this is quite true