Cosmology concepts encompass complex spatial and temporal relations that are counterintuitive. Cosmology findings, because of their intrinsic interest, are often reported in the public domain with enthusiasm, and students come to cosmology with a range of conceptions some aligned and some at variance with the current science. This makes cosmology concepts challenging to teach, and also challenging to evaluate students' conceptual understanding. This study builds on previous research of the authors investigating the methodological challenges for characterising students' cosmology conceptions and the reasoning underlying these. Insights from student responses in two iterations of an open-ended instrument were used to develop a concept inventory that combined cosmological conceptions with reasoning levels based on the SOLO taxonomy. This paper reports on the development and validation of the Cosmology Concept Inventory (CosmoCI) for high school. CosmoCI is a 28-item multiple-choice instrument that was implemented with grade 10 and 11 school students (n = 234) in Australia and Sweden. Using Rasch analysis in the form of a partial credit model (PCM), the paper describes a validated progression in student reasoning in cosmology across four conceptual dimensions, supporting the utility of CosmoCI as an assessment tool which can also instigate rich discussions in the science classroom.