The second version of Kaiser's Measure of Sampling Adequacy (MSA[subscript 2]) has been widely applied to assess the factorability of data in psychological research. The MSA[subscript 2] is developed in the population and little is known about its behavior in finite samples. If estimated MSA[subscript 2]s are biased due to sampling errors,…

The current study examined the reliability of The Reading League Curriculum-Evaluation Guidelines (CEGs), which were developed to help school-based teams rate the presence of red flags when considering adopting specific literacy curricula. Coders (n = 30) independently used the CEGs to evaluate a free online English language arts curriculum. The…

Trend scoring constructed response items (i.e. rescoring Time A responses at Time B) gives rise to two-way data that follow a product multinomial distribution rather than the multinomial distribution that is usually assumed. Recent work has shown that the difference in sampling model can have profound negative effects on statistics usually used to…

The typical process for assessing inter-rater reliability is facilitated by training raters within a research team. Lacking is an understanding if inter-rater reliability scores "between" research teams demonstrate adequate reliability. This study examined inter-rater reliability between 16 researchers who assessed fundamental motor…

In this paper I try to 'rethink' consistency as an educational quality for the 3rd millennium, following Italo Calvino's choice to take it up in his lecture series Memos for the Next Millennium, and despite the fact that the (final) lecture devoted to this quality remained unwritten. After reflecting on how consistency already plays a certain role…

Evaluation of child-friendly school (CFS) policies is essential to determine the achievements of school efforts in reducing violence cases. This research aims to proving the reliability and validity of CFS policy evaluation instruments in elementary schools with different locations. This investigation uses the Context Input Process Product (CIPP)…

The aim of this study was to verify the concurrent validity and the biological error-free reliability of a novel low-cost commercial encoder (Ergonauta I). Validity protocol involved comparisons with a custom system and other encoder commercially available (Vitruve). Reliability protocols involved interdevices and interunit comparisons. No…

An application of Bayesian factor analysis for evaluation of scale reliability is discussed, which is developed within the framework of latent variable modeling. The method permits direct point and interval estimation of the reliability coefficient of multiple-component measuring instruments using Bayesian inference. The approach allows also point…

A Bayesian statistics-based approach is discussed that can be used for direct evaluation of the popular Cronbach's coefficient alpha as an internal consistency index for multiple-component measuring instruments, as well as for testing its identity to scale reliability. The method represents an application of confirmatory factor analysis within the…

Purpose: The current literature on school teacher-created summative assessment lacks a clear consensus regarding its definition and key principles. The purpose of this research was therefore to arrive at a cohesive understanding of what constitutes effective summative assessment. Design/methodology/approach: Conducting a systematic literature…

When making inferences about the mental lives of others (e.g., others' preferences), it is critical to consider the extent to which the choices we observe are constrained. Prior research on the development of this tendency indicates a contradictory pattern: Children show remarkable sensitivity to constraints in traditional experimental paradigms,…

Measuring classroom behavior among young children is important to guide assessment and intervention decisions, yet there is limited literature on appropriate direct observation tools for this purpose. This article describes the psychometric properties of the Behavior Assessment System for Children, Student Observation System (BASC-3 SOS) with 135…

The development of computational thinking skills is attracting attention worldwide. The use of visual or block-based coding in primary schools has gained momentum. Yet, students' acceptance of such coding environments has been neglected in the literature. This study presents a measurement instrument that will allow pursuing such an endeavor. The…

Purpose: Auditory training has been shown to reduce rater variability in perceptual voice assessment. Because rater variability is also a central issue in the auditory-perceptual assessment of dysarthria, this study sought to determine if training produces a meaningful change in rater reliability, criterion validity, and scaling magnitude of four…

The Creative Achievement Questionnaire (CAQ) is widely used to measure the creative achievement levels of individuals. Previous studies reported a varying range of reliability coefficients for the CAQ. To this date, no study has investigated the variability in the reliability coefficients of the CAQ. A random-effects reliability generalization…