College Guide
Blog
U.S. News & World Report rankings of colleges has long been a source of concern for those interested in determining college quality. Part of the trouble comes from the way that 25 percent of the college ranking (more than any other factor) comes from a survey of a college’s reputation. Essentially this reputation survey, or peer assessment, is a measure of how people “feel” about a college. It turns out the reputation ranking is pretty much fixed from year to year. According to an article in Inside Higher Ed:
Two scholars evaluated changes in reputational scores of colleges and then looked for correlations between those changes and other factors that U.S. News declares are important and recalculates each year: graduation and retention, faculty resources, selectivity and financial resources. The theory behind the study was that if these are key measures of quality in the magazine’s view, institutions that change in these categories should also experience reputational changes over time. But they didn’t — while the correlation that was clear was reputation with the previous year’s rankings.
Basically, the reputation a college had one year was the only indicator of the reputation it would have the next year, regardless of any changes that actually occurred at the school.
Robert Morse, the director of data research for U.S. News & World Report, protests that this should not be much of a surprise, since schools change very slowly. “U.S. News believes that the peer assessment scores are measuring something valuable and help provide highly useful information about the relative merits of schools,” Morse told Inside Higher Ed.
Reputation surveys measure “something valuable” for sure. What’s odd is that U.S. News currently takes it to be the most valuable part of American college quality.
Facebook
Twitter
g on January 13, 2010 5:40 PM:
I'd read/heard that USNews uses Princeton/Yale/Harvard as a check on their rankings. As in, one of those school must be no. 1 or there is a problem with the formula. One year - and only one year - CalTech came out number 1. The formula was immediately changed to restore the Ivies to their birthright position at the top of the charts.
CapitalistImperialistPig on January 13, 2010 5:53 PM:
Some post college testing could help. At which schools do students learn more?
frank jones on January 13, 2010 7:07 PM:
I like when schools try to convince people they are prestigious by constantly calling themselves prestigious. Yale never needs to tell people this because everyone knows. But here in CT, Trinity and Wesleyan are always telling people that they're "prestigious," even though no one in Deadwood, North Dakota or Centerville, California has heard of them or cares.
I will now be referring to myself as "prestigious" in all communications and conversations. Hopefully, if enough people think I'm prestigious, I will move up in the rankings. Thank you, and please give to my endowment fund.
Texas Aggie on January 14, 2010 12:11 AM:
I would be surprised if the evaluations of reputation change much. First of all, the reputation depends on what the school has done in the past, and the past stays the past. Second, schools don't change that much from year to year. If it had good academics last year, there is little reason to suspect that it won't have good academics next year. Frankly, I can't think of a single thing that would merit a drastic change in reputation other than maybe a takeover by some cult group.