Prospective students and their families use college rankings to navigate their higher education options. Rising tuition and fees have made the college decision more fraught. Recently, the major college ranking providers have revised their methodologies to reflect costs and other considerations. These revisions raise important questions about the precise qualities the rankings aim to measure. Vanderbilt University asked NORC to produce a report evaluating the methodological validity of five major college ranking systems: U.S. News & World Report (USNWR), Wall Street Journal (WSJ), Forbes' Top College list (Forbes), The Times Higher Education World University Ranking (THE), and the QS World University Ranking (QS). Drawing on well-established social scientific concepts like "construct validity," this report identifies many issues in the conceptualization and construction of college ranking lists and offers methodological improvements that might address these shortcomings. The aim of this report is to inform consumers of these rankings--including college-going students, their parents, and college leaders--of the limitations of existing college ranking systems. Findings from this project will guide the development of improved systems for informing college-goers.