This study presents the validation process of a listening test based on a communicative language test proposed by Bachman (Fundamental considerations in language testing, 1990). It was administered to third-grade high school students by the sixteen Korean Provincial Offices of Education for Curriculum and Evaluation in September 2012 to assess their listening ability at the end of high school learning and compare it with the standard of the CSAT (College Scholastic Ability Test). The research questions were the following. First, to what extent does the test measure the listening comprehension construct? Second, what sub-skills does the test measure? Third, to what extent does the test measure communicative ability? To answer these three questions, a study was designed to examine the test's construct validity using classical test theory (CTT). Then, an exploratory factor analysis (EFA) and a confirmatory factor analysis (CFA) were applied to find a model that would fit the score of 400 examinees to explain the correlation between six divisible sub-skills of English listening comprehension and twenty listening items. R-program is used as a tool to analyze the above data. The results show that this test is not discriminatory, as the purpose of a summative assessment is not to level students into different groups. With an acceptable measurement of the construct, this research concludes that the test does not have a clear division of listening sub-skills but, on the other hand, sufficiently measures communicative ability.