College Guide

Blog

August 22, 2010 11:00 PM A Note on Methodology: Community Colleges

By the Editors

The Washington Monthly ranking of America’s best community colleges is based on two sources of information: the Community College Survey of Student Engagement (CCSSE) and graduation rates published by the U.S. Department of Education.

The CCSSE survey is managed by a nonprofit organization of the same name housed at the University of Texas at Austin. The survey is given to a representative sample of students at community colleges that voluntarily choose to participate. Since 2003, CCSSE has been administered to 68 percent of all community colleges in America. All CCSSE results are published on the organization’s Web site, www.ccsse.org.

Because most community colleges do not administer the survey every year, we combined results from the three most recent available years: 2007, 2008, and 2009. For colleges that participated more than once, we used the most recent year. More than 60 percent of all community colleges—roughly 650 institutions— were included in the rankings, from which we published the top fifty.

The CCSSE survey is comprised of 104 questions on a range of topics including teaching practices, student workload, interaction with faculty, and student support. CCSSE combines the results of those questions into aggregate “benchmark” scores in five categories: “Active and Collaborative Learning,” “Student Effort,” “Academic Challenge,” “Student-Faculty Interaction,” and “Support for Learners.” The benchmark scores are standardized to range from 0 to 100 with an average score of 50.

CCSSE has published research examining the relationship between student responses on the survey and those students’ likelihood of succeeding in college, as measured by outcomes such as grades, credit attainment, dropout rates, and graduation rates. Even after controlling for student factors like race, income, gender, and prior academic achievement, the research found that all five benchmarks are positively correlated with student success. However, the degree of correlation was not identical. Some practices matter more than others. Accordingly, we gave more weight in our rankings to the more important practices. Eighty-five percent of each college’s ranking is based on its CCSSE benchmark scores, weighted as follows:

Active and Collaborative Learning: 29%
Student Effort: 12%
Academic Challenge: 19%
Student-Faculty Interaction: 14%
Support for Learners: 12%

These weights (which do not sum to 85 percent due to rounding) were based on the number and strength of statistically significant correlations between CCSSE benchmarks and various student outcomes in three separate studies published by CCSSE. The weights represent the Washington Monthly’s interpretation of CCSSE research. While CCSSE constructed the benchmark measures to facilitate comparison among institutions, it does not endorse the use of its data for rankings.

As with any sample-based survey, CCSSE results have statistical margins of error. Readers should be mindful of this when interpreting the rankings, particularly when differences between colleges are small.

The remaining 15 percent of each college’s ranking is based on graduation rates submitted by colleges to the U.S. Department of Education. In our 2007 version of America’s best community colleges, we used three-year graduation rates (the percentage of students who graduate within three years of enrolling for the first time as freshmen). Subsequently, the U.S. Department of Education has published four-year graduation rates. Because many community college students take longer than three years to graduate, we used four-year graduation rates for this year’s rankings.

Federal graduation rate measures do not include students who enroll part-time and do not give colleges credit for students who take longer than four years to graduate, or who transfer to another college without earning a degree. However, research published by the Community College Research Center at Columbia University Teachers College has found that when part-time students are included in graduation rates and colleges are given credit for transfers and long-term graduates, their ranking positions relative to one another are similar to rankings produced using U.S. Department of Education measures. Therefore, we believe federal graduation rates are valid for ranking purposes.

the Editors can be found on Twitter: @washmonthly.