In previous studies, problem solving appraisal has been identified as playing a key role in promoting positive psychological well-being. The Problem Solving Inventory is the most widely used measure of problem solving appraisal and consists of 32 items. The length of the instrument, however, may limit its applicability to large-scale surveys…

Purpose: The purpose of this paper is to find appropriate forms of analysis of multiple-choice questions (MCQ) to obtain an assessment method, as fair as possible, for the students. The authors intend to ascertain if it is possible to control the quality of the MCQ contained in a bank of questions, implemented in Moodle, presenting some evidence…

The examination of the testing mode effect has received increased attention as higher education has shifted to remote testing during the COVID-19 pandemic. We believe the testing mode effect consists of four components: the ability to physically write on the test, the method of answer recording, the proctoring/testing environment, and the effect…

The Fundamentals in Meteorology Inventory (FMI) concept inventory exam was recently developed to gauge understanding of fundamental meteorological concepts among introductory meteorology students. The exam was iteratively tested and refined over several years using a small sample of students. With a goal of evaluating the reliability and validity…

Currently there are no assessment instruments available for upper-division thermal physics, though several introductory assessments are currently available. Notably missing from these introductory assessment are items targeting statistical mechanics. This leaves a gap in the content that can be assessed by upper-division thermal physics faculty.…

Research on the test structure of the Force Concept Inventory (FCI) has largely ignored gender, and research on FCI gender effects (often reported as "gender gaps") has seldom interrogated the structure of the test. These rarely crossed streams of research leave open the possibility that the FCI may not be structurally valid across…

We developed and evaluated an Oceanography Concept Inventory (OCI), which used a mixed-methods approach to test student achievement of 11 learning goals for an introductory-level oceanography course. The OCI was designed with expert input, grounded in research on student (mis)conceptions, written with minimal jargon, tested on 464 students, and…

This article provides preliminary validity evidence for the shorter Mandarin version of the Vocabulary Size Test (VST) under the content aspect, technical quality, substantive and generalizability aspect of Messick's (1995) construct validity framework. The shorter version with 177 Chinese university students in three proficiency levels indicates…

This dissertation includes two independent research projects, one in astronomy education research and the other in planetary science/astrophysics research. In the first research effort, we investigate college students' conceptual and reasoning difficulties on the topic of planet formation pre-instruction. Through an analysis of over 1,000…

Student learning in instructional physics labs represents a growing area of research that includes investigations of students' beliefs and expectations about the nature of experimental physics. To directly probe students' epistemologies about experimental physics and support broader lab transformation efforts at the University of Colorado Boulder…

Concepts of second language proficiency and how proficiency may be assessed have changed considerably over the last 20 years. New notions of validity with respect to the interpretation and uses of test scores have begun to shape discussions about test validity and quality assurance in college world language departments, in government, and in…

The topic of Newtonian gravity offers a unique vantage point from which to investigate and encourage conceptual change because it is something with which everyone has daily experience, and because it is taught in two courses that reach a wide variety of students--introductory-level college astronomy ("Astro 101") and physics ("Phys…

Item response models (IRMs) were used to analyze Cross Racial Identity Scale (CRIS) scores. Rasch analysis scores were compared with classical test theory (CTT) scores. The partial credit model demonstrated a high goodness of fit and correlations between Rasch and CTT scores ranged from 0.91 to 0.99. CRIS scores are supported by both methods.…

This study assesses the Japanese translation of the Force and Motion Conceptual Evaluation (FMCE). Researchers are often interested in comparing the conceptual ideas of students with different cultural backgrounds. The FMCE has been useful in identifying the concepts of English-speaking students from different backgrounds. To identify effectively…

This study compares the psychometric utility of Classical Test Theory (CTT) and Item Response Theory (IRT) for scale construction with data from higher education student surveys. Using 2008 Your First College Year (YFCY) survey data from the Cooperative Institutional Research Program at the Higher Education Research Institute at UCLA, two scales…