An application of Bayesian factor analysis for evaluation of scale reliability is discussed, which is developed within the framework of latent variable modeling. The method permits direct point and interval estimation of the reliability coefficient of multiple-component measuring instruments using Bayesian inference. The approach allows also point…

A Bayesian statistics-based approach is discussed that can be used for direct evaluation of the popular Cronbach's coefficient alpha as an internal consistency index for multiple-component measuring instruments, as well as for testing its identity to scale reliability. The method represents an application of confirmatory factor analysis within the…

Educational researchers often need to construct precise and reliable measurement scales of complex and varied representations such as participants' written work, videoed lesson segments and policy documents. Developing such scales using can be resource-intensive and time-consuming, and the outcomes are not always reliable. Here we present…

We describe best practices in reporting reliability estimates in counseling research with consideration to precision, generalization, and diverse populations. We provide a historical context to reporting reliability estimates, the limitations of past practices, and new methods to address reliability generalization. We highlight best practices…

The purpose of this article is to discuss reliability and precision through the lens of a modern measurement approach, item response theory (IRT). Reliability evidence in the field of counseling is primarily generated using Classical Test Theory (CTT) approaches, although recent studies in the field of counseling have shown the benefits of using…

This paper details how to effectively share three-dimensional geological models using data conversion between two mainstream mining software, Micromine and Surpac. It also discusses the impact of this conversion method on geological integrated exploration decision-making guidance. The current situation primarily manifests in the fact that both…

The population relationship between coefficient alpha and scale reliability is studied in the widely used setting of unidimensional multicomponent measuring instruments. It is demonstrated that for any set of component loadings on the common factor, regardless of the extent of their inequality, the discrepancy between alpha and reliability can be…

In this digital ITEMS module, Dr. Jacqueline Leighton and Dr. Blair Lehman review differences between think-aloud interviews to measure problem-solving processes and cognitive labs to measure comprehension processes. Learners are introduced to historical, theoretical, and procedural differences between these methods and how to use and analyze…

The remarkable strides in artificial intelligence (AI), exemplified by ChatGPT, have unveiled a wealth of opportunities and challenges in assessment. Applying cutting-edge large language models (LLMs) and generative AI to assessment holds great promise in boosting efficiency, mitigating bias, and facilitating customized evaluations. Conversely,…

Project management (PM) work readiness requires preparation beyond the knowledge of concepts and best practices to develop and practice soft skills needed for successful project management. This article proposes a methodology that combines real-world practitioner tools, PM processes, and a Microsoft Excel-based PM Simulator to create a…

The procedures of carrying out factorial invariance to validate a construct were well developed to ensure the reliability of the construct that can be used across groups for comparison and analysis, yet mainly restricted to the frequentist approach. This motivates an update to incorporate the growing Bayesian approach for carrying out the Bayesian…

Schools are typically highly structured institutions, so any shifts in processes can potentially generate anxiety, confusion, and even anger among staff. As such, when there is frequent change, initiatives often fail to achieve their intended goals. Well-intentioned initiatives can go awry when not consistently implemented correctly by all staff;…

Reading and literacy skills are essential to fostering a child's development and academic success. Research has consistently highlighted the importance of early literacy proficiency as a strong indicator of later academic achievement, cognitive ability, and social-emotional well-being (Espinosa, 2002; Gottlieb et al., 2022). Unfortunately, reading…

A majority of youth in residential facilities such as residential children's treatment centers and juvenile justice facilities have a history of exposure to traumatic events, contributing to a multitude of long-term mental and physical concerns. Residential facility teachers and staff have the unique opportunity to create trauma-informed spaces -…

Decision consistency (DC) is the reliability of a classification decision based on a test score. In professional credentialing, the decision is often a high-stakes pass/fail decision. The current methods for estimating DC are computationally complex. The purpose of this research is to provide a computationally and conceptually simple method for…