The second version of Kaiser's Measure of Sampling Adequacy (MSA[subscript 2]) has been widely applied to assess the factorability of data in psychological research. The MSA[subscript 2] is developed in the population and little is known about its behavior in finite samples. If estimated MSA[subscript 2]s are biased due to sampling errors,…

This study compares how the generative AI (GenAI) large language model (LLM) ChatGPT performs in grading university exams compared to human teachers. Aspects investigated include consistency, large discrepancies and length of answer. Implications for higher education, including the role of teachers and ethics, are also discussed. Three…

The current study examined the reliability of The Reading League Curriculum-Evaluation Guidelines (CEGs), which were developed to help school-based teams rate the presence of red flags when considering adopting specific literacy curricula. Coders (n = 30) independently used the CEGs to evaluate a free online English language arts curriculum. The…

The typical process for assessing inter-rater reliability is facilitated by training raters within a research team. Lacking is an understanding if inter-rater reliability scores "between" research teams demonstrate adequate reliability. This study examined inter-rater reliability between 16 researchers who assessed fundamental motor…

Evaluation of child-friendly school (CFS) policies is essential to determine the achievements of school efforts in reducing violence cases. This research aims to proving the reliability and validity of CFS policy evaluation instruments in elementary schools with different locations. This investigation uses the Context Input Process Product (CIPP)…

The aim of this study was to verify the concurrent validity and the biological error-free reliability of a novel low-cost commercial encoder (Ergonauta I). Validity protocol involved comparisons with a custom system and other encoder commercially available (Vitruve). Reliability protocols involved interdevices and interunit comparisons. No…

Purpose: The current literature on school teacher-created summative assessment lacks a clear consensus regarding its definition and key principles. The purpose of this research was therefore to arrive at a cohesive understanding of what constitutes effective summative assessment. Design/methodology/approach: Conducting a systematic literature…

When making inferences about the mental lives of others (e.g., others' preferences), it is critical to consider the extent to which the choices we observe are constrained. Prior research on the development of this tendency indicates a contradictory pattern: Children show remarkable sensitivity to constraints in traditional experimental paradigms,…

Measuring classroom behavior among young children is important to guide assessment and intervention decisions, yet there is limited literature on appropriate direct observation tools for this purpose. This article describes the psychometric properties of the Behavior Assessment System for Children, Student Observation System (BASC-3 SOS) with 135…

The development of computational thinking skills is attracting attention worldwide. The use of visual or block-based coding in primary schools has gained momentum. Yet, students' acceptance of such coding environments has been neglected in the literature. This study presents a measurement instrument that will allow pursuing such an endeavor. The…

Purpose: Auditory training has been shown to reduce rater variability in perceptual voice assessment. Because rater variability is also a central issue in the auditory-perceptual assessment of dysarthria, this study sought to determine if training produces a meaningful change in rater reliability, criterion validity, and scaling magnitude of four…

The Creative Achievement Questionnaire (CAQ) is widely used to measure the creative achievement levels of individuals. Previous studies reported a varying range of reliability coefficients for the CAQ. To this date, no study has investigated the variability in the reliability coefficients of the CAQ. A random-effects reliability generalization…

The purpose of this instructional piece was to provide a nontechnical synthesis of common internal consistency reliability estimates used in professional counseling and in related fields. The article begins with an overview of coefficients alpha, omega, omega hierarchical, and H, with guidelines for their selection. Next, I provide recommendations…

Studies have shown that raising L2 learners' metaphor awareness contributes to the acquisition of figurative language, which fosters students' development of language skills. However, the instruments measuring metaphor awareness, in the majority of relevant research, did not seem to have undergone proper methodological procedures for checking…

This research compares the performance measures reported by two bibliographic databases relevant to a set of authors who have published in predatory journals. The reliability of decision-making based on the information provided by uncontrolled bibliographic databases is examined to support rational decisions. A sample of authors who published in…