Current science education reform efforts emphasize teaching K-12 science using hands-on, inquiry activities. For maximum learning and probability of implementation among inservice teachers, these strategies must be modeled in college science courses for preservice teachers. About a decade ago, Morehead State University revised their science content courses to follow an inquiry approach. As part of the courses' assessment, a locally-made, diagnostic pre- and post-test was prepared. The main purpose of this "ex post facto" study was to demonstrate how concepts from Item Response Theory can be used to detect and remove psychometrically faulty items, and how the remaining items can be used by teachers to determine science learning gains in an inquiry-based physical science course that implemented two different curricula, "Physics and Everyday Thinking" and "Interactions in Physical Science".