College Guide

Blog

August 20, 2009 12:47 PM International Studies

How America’s mania for college rankings went global.

By Ben Wildavsky

Colleges have never much liked the idea of being ranked. When U.S. News & World Report published its first full-fledged guide to colleges in 1987 a delegation of college presidents and senior administrators met with the magazine’s editors and asked that the ranking enterprise be stopped. Purely numerical indicators were an inappropriate way to measure the varied institutions and missions of American higher education, they argued. During my two-year tenure as editor of U.S. News’s annual undergrad and graduate rankings issues, part of my job entailed dealing with similarly unhappy administrators, who regularly visited our office to complain about the treatment they received in our rankings. Some made it pretty clear that they thought outsiders—especially journalists—shouldn’t be ranking colleges at all. Others took exception to one or more aspects of the U.S. News methodology.

Such complaints were hardly disinterested—we rarely heard them from universities that fared well in the rankings, who trumpeted the results on their Web sites. But the disgruntled administrators did have a point. The U.S. News rankings depend heavily on a survey of college and university presidents and administrators, who are asked their opinions on the quality of their own and other institutions. Whether these individuals have enough knowledge to accurately judge the quality of other schools or are just circulating the conventional wisdom is a fair question (although if they are behaving ethically they are supposed to simply follow the magazine’s instructions and avoid rating any school they don’t know much about). The rankings also rely on a plethora of more objective measures, such as faculty salaries, average class size, graduation rates, and the SAT scores of incoming freshmen. But while these are defensible stand-ins for excellence (and I defended them) they are mostly measures of inputs—what human and financial resources go into a university rather than what educational results come out. To truly judge educational quality requires also measuring outputs—specifically, how much students actually learn in the classroom, particularly at the undergraduate level. Such data exists, albeit in limited quantities, and U.S. News has tried to get it. But colleges and universities jealously hoard the information, so U.S. News has had to make do with the data it can obtain.

Creating an accurate, useful ranking of schools is a daunting, and perhaps quixotic, task. But arguing against imperfect efforts to try to rank colleges is beside the point. The runaway popularity of the U.S. News rankings demonstrates the enormous appetite that exists for the product: unsurprisingly, people like to know something about the education they’re considering paying tens of thousands of dollars a year for, and in the absence of truly meaningful measures of what exactly a college produces in exchange for your tuition, halfway measures like the existing rankings are about as good as you’re going to get.

The potency of college rankings can be seen in their rapid spread internationally. Since the 1990s, more than forty countries, from Poland to Argentina to Kazakhstan, have developed some sort of ranking of their national universities—often called “league tables,” the same term used in Europe for sports rankings.

The current decade has seen the rise of rankings that compare colleges not just within countries but between them. In 2001, administrators at China’s Shanghai Jiao Tong University tasked a chemical engineering professor named Nian Cai Liu with assessing how their school compared with others around the world. China’s massive economic expansion was in full bloom, the government saw higher education as a potential source of innovation and economic growth, and the Shanghai Jiao Tong officials wanted to know where they stood in the international marketplace. Looking at a number of mostly research-oriented factors—how many frequently cited researchers schools employed, how many articles they published, and how many prizes they won—Liu and his staff spent two years compiling data for more than 2,000 institutions. Then they weighted the measures, converted them into an aggregate score for each institution, and sorted the world’s universities from top to bottom. Without really trying to, Liu had created the first comprehensive global college rankings—Shanghai Jiao Tong’s Academic Ranking of World Universities became an international sensation, much followed in the academic and government worlds.

A year after the debut of the Shanghai rankings, Times Higher launched its own, very different global effort to compare universities. Where the Shanghai rankings focused almost exclusively on research, the Times Higher rankings aimed for a more “rounded assessment” that would also appeal to consumers—prospective college students and their parents. In practice, this has meant that Times Higher is even more reliant than U.S. News on subjective “reputational” data from surveys of college administrators, though Times Higher also incorporates such measures as employer surveys, student/faculty ratios, and the percent of international students and faculty at a university—the latter on the grounds that it serves as a market test of an institution’s ability to attract brainpower in an ever more globalized world.

Ben Wildavsky is a senior fellow at the Ewing Marion Kauffman Foundation and a guest scholar at the Brookings Institution. He is working on a book about the globalization of higher education, forthcoming from Princeton University Press in 2010.

Comments

  • Phil Baty on December 18, 2009 10:03 AM:

    The Times Higher Education World University Rankings are changing. The magazine has ended its rankings partnership with QS and will now produce the annual rankings with Thomson Reuters. THE and TR are currently consulting on a new more rigorous and transparent methodology. For news on the development of the rankings, and to have yoru say, vist: http://bit.ly/ErAag
    Phil Baty