Testing for differential item functioning (DIF) has undergone rapid statistical developments recently. Moderated nonlinear factor analysis (MNLFA) allows for simultaneous testing of DIF among multiple categorical and continuous covariates (e.g., sex, age, ethnicity, etc.), and regularization has shown promising results for identifying DIF among…

Purpose: Multisite mediation studies are a cornerstone in mapping out developmental processes because they probe the mechanisms of a treatment while creating key opportunities to learn from and about variation in those mechanisms across sites. Despite the prevalence of multisite studies, a significant gap in the literature is how to plan such…

Drawing on descriptions of research-into-practice initiatives, this article presents a new framework to aid understanding of how research findings influence educational practice at scale. The framework focuses upon five areas: trustworthiness of the findings and generalisability; implications and instructions for practice; support for…

Although generalizability theory (GT) designs have traditionally been analyzed within an ANOVA framework, identical results can be obtained with structural equation models (SEMs) but extended to represent multiple sources of both systematic and measurement error variance, include estimation methods less likely to produce negative variance…

Within physics education research (PER), resource theory has proven to be a useful framework for investigating knowledge and learning and informing instructional design. To analyze learning over longer timescales and across cases, PER scholars must first identify and describe the resources activated within and across physics contexts and domains.…

Conventional methods for evaluating the utility of subscores rely on traditional indices of reliability and on correlations among subscores. One limitation of correlational methods is that they do not explicitly consider variation in subtest means. An exception is an index of score profile reliability designated as [G], which quantifies the ratio…

The use of Bayesian Knowledge Tracing (BKT) models in predicting student learning and mastery, especially in mathematics, is a well-established and proven approach in learning analytics. In this work, we report on our analysis examining the generalizability of BKT models across academic years attributed to "detector rot." We compare the…

Rater comments tend to be qualitatively analyzed to indicate raters' application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The…

This article introduces generalizability theory (G-theory) to institutional research and assessment practitioners, and explains how it can be utilized to evaluate the reliability of assessment procedures in order to improve student learning outcomes. The fundamental concepts associated with G-theory are briefly discussed, followed by a discussion…

Estimation of parameters for the many-facets Rasch model requires that conditional on the values of the facets, such as person ability, item difficulty, and rater severity, the observed responses within each facet are independent. This requirement has often been discussed for the Rasch models and 2PL and 3PL models, but it becomes more complex…

Modeling within-person change over time and between-person differences in change over time is a primary goal in prevention science. When modeling change in an observed score over time with multilevel or structural equation modeling approaches, each observed score counts toward the estimation of model parameters equally. However, observed scores…

With the development of the Internet as a main source of information, teenagers are increasingly faced with multiple documents which may contain contradictory statements, and whose reliability must be assessed. One way to assess information reliability is to evaluate the source of the information (e.g., author expertise, intention). However,…

Objective: This study aimed to validate the Cultural Humility and Enactment Scale (CHES) through (a) examining its factor structure with multiple samples; (b) employing item response theory (IRT) analysis to examine its item-level characteristics; (c) reducing potential redundancies among items; and (d) conducting measurement invariance (MI)…

The present study aimed to examine how to improve the effectiveness of moral exemplar-applied interventions based on the pillars of the self-determination theory framework, autonomy, competence, and relatedness. Past research has mainly focused on the relatedness and attainability of moral exemplars for predicting motivation outcomes. The data for…

Research Findings: Given the variable nature of early childhood settings, practitioners and researchers need better guidance on what conditions influence observations conducted within early childhood settings (National Research Council, 2008). Using 230 observations from 23 three- and four-year-old children, we conducted a Generalizability study…