Rater comments tend to be qualitatively analyzed to indicate raters' application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The…

Listening is widely regarded as the predominant language proficiency utilized in virtually all forms of communication. However, its intricacies often engender feelings of complexity and, at times, provoke anxiety and frustration among both foreign and second-language learners. The enhancement of successful communication fundamentally hinges upon…

We developed and evaluated an Oceanography Concept Inventory (OCI), which used a mixed-methods approach to test student achievement of 11 learning goals for an introductory-level oceanography course. The OCI was designed with expert input, grounded in research on student (mis)conceptions, written with minimal jargon, tested on 464 students, and…

The purpose of this study was to develop the Basic Literacy In Statistics (BLIS) assessment for students in an introductory statistics course, at the postsecondary level, that includes, to some extent, simulation-based methods. The definition of statistical literacy used in the development of the assessment was the ability to read, understand, and…

Contribution: This article illustrates a theoretical model and an analytical tool for understanding the influence of diverse elements of the project-based learning (PBL) environment on engineering identity development in students. Supportive elements in three dimensions that contribute to students' identities as future engineers are identified to…

This study examines the adaptations and experiences of college-level language learners in the post-pandemic era, providing valuable insights into the transformative nature of language learning in an evolving world. Within the framework of the socio-cultural perspective, this study explores the interplay between technology, self-directed learning,…

The use of surveys, questionnaires, and rating scales to measure important outcomes in higher education is pervasive, but reliability and validity information is often based on problematic Classical Test Theory approaches. Rasch Analysis, based on Item Response Theory, provides a better alternative for examining the psychometric quality of rating…

In recent years many universities switched from paper- to online-based student evaluation of teaching (SET) without knowing the consequences for data quality. Based on a series of three consecutive field experiments--a split-half design, twin courses, and pre-post-measurements--this paper examines the effects of survey mode on SET. First, all…

The purpose of the present study is developing a test to measure the numerical ability for students of education. The sample of the study consisted of (504) students from 8 universities in Jordan. The final draft of the test contains 45 items distributed among 5 dimensions. The results revealed that acceptable psychometric properties of the test;…

The Locus-of-Hope Scale (LHS) was developed as a measure of the locus-of-hope dimensions (Bernardo, 2010). The present study adds to the emerging literature on locus-of-hope by assessing the psychometric properties of the LHS using Rasch analysis. The results from the Rasch analyses of the four subscales of LHS provided evidence on the…

Classroom response systems (often referred to as "clickers") have slowly gained adoption over the recent decade; however, critics frequently doubt their pedagogical value starting with the validity of the gathered responses: There is concern that students simply "click" random answers. This case study looks at different…

Our goal in this investigation was to evaluate the reliability of scores from the Balanced Inventory of Desirable Responding (BIDR) more comprehensively than in prior research using a generalizability-theory framework based on both dichotomous and polytomous scoring of items. Generalizability coefficients accounting for specific-factor, transient,…

This study traces the development of a grounded theory of assessment in e-learning environments, a field in need of research to establish the parameters of an assessment that is both reliable and worthy of higher learning accreditation. Using grounded theory as a research method, we studied an e-assessment model that does not require physical…

Test-retest studies for assessing stability and change are widely used in different domains and allow improved or additional individual estimates of interest to be obtained. However, if these estimates are to be validly interpreted the responses given at Time-2 must be free of retest effects, and the fulfilment of this assumption must be…

The American public's interests are well-served by a strong, effective postsecondary education system. And yet the industry's predominant learning and service paradigm, one that credentials learning by measuring student's time on task and that treats all learners largely the same from a pacing and a requirements perspective is inconsistent with…