Research related to teaching climate change, system thinking, current reform in science education, and the research on reform-oriented assessment indicate that we need to explore student understanding in greater detail instead of only testing for an incremental gain in disciplinary knowledge. Using open-ended items we assessed details in student successes and challenges in thinking about climate change and climate system. Twelve teachers of the 7th and 8th grades and 457 students from four schools in a Midwestern state participated in this study. Statistical analysis of student responses on the pre- and post-test showed a significant (p<0.0001) gain in knowledge but more importantly, qualitative analysis of student responses showed that they learned that our climate is changing but had constructed a linear connection between variables such as surface temperature and drought. The students did not develop a connected body of knowledge or an understanding of climate as a system. Future research needs to focus on curriculum development, effective system pedagogy, assessment, and teacher development in the context of climate change as an interlinked system.