Testing for differential item functioning (DIF) has undergone rapid statistical developments recently. Moderated nonlinear factor analysis (MNLFA) allows for simultaneous testing of DIF among multiple categorical and continuous covariates (e.g., sex, age, ethnicity, etc.), and regularization has shown promising results for identifying DIF among…

The purpose of this article is to discuss reliability and precision through the lens of a modern measurement approach, item response theory (IRT). Reliability evidence in the field of counseling is primarily generated using Classical Test Theory (CTT) approaches, although recent studies in the field of counseling have shown the benefits of using…

Drawing on descriptions of research-into-practice initiatives, this article presents a new framework to aid understanding of how research findings influence educational practice at scale. The framework focuses upon five areas: trustworthiness of the findings and generalisability; implications and instructions for practice; support for…

Taking a sociological view, we can investigate the empirical consequences of variations in the rhetoric of sociological methodology. The standards advocated in Qualitative Literacy divide communities of qualitative researchers, as they are not explicitly connected to an understanding of social ontology, unlike previous qualitative methodologies;…

Although generalizability theory (GT) designs have traditionally been analyzed within an ANOVA framework, identical results can be obtained with structural equation models (SEMs) but extended to represent multiple sources of both systematic and measurement error variance, include estimation methods less likely to produce negative variance…

Within physics education research (PER), resource theory has proven to be a useful framework for investigating knowledge and learning and informing instructional design. To analyze learning over longer timescales and across cases, PER scholars must first identify and describe the resources activated within and across physics contexts and domains.…

Conventional methods for evaluating the utility of subscores rely on traditional indices of reliability and on correlations among subscores. One limitation of correlational methods is that they do not explicitly consider variation in subtest means. An exception is an index of score profile reliability designated as [G], which quantifies the ratio…

Rater comments tend to be qualitatively analyzed to indicate raters' application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The…

Estimation of parameters for the many-facets Rasch model requires that conditional on the values of the facets, such as person ability, item difficulty, and rater severity, the observed responses within each facet are independent. This requirement has often been discussed for the Rasch models and 2PL and 3PL models, but it becomes more complex…

Modeling within-person change over time and between-person differences in change over time is a primary goal in prevention science. When modeling change in an observed score over time with multilevel or structural equation modeling approaches, each observed score counts toward the estimation of model parameters equally. However, observed scores…

With the development of the Internet as a main source of information, teenagers are increasingly faced with multiple documents which may contain contradictory statements, and whose reliability must be assessed. One way to assess information reliability is to evaluate the source of the information (e.g., author expertise, intention). However,…

Objective: This study aimed to validate the Cultural Humility and Enactment Scale (CHES) through (a) examining its factor structure with multiple samples; (b) employing item response theory (IRT) analysis to examine its item-level characteristics; (c) reducing potential redundancies among items; and (d) conducting measurement invariance (MI)…

The present study aimed to examine how to improve the effectiveness of moral exemplar-applied interventions based on the pillars of the self-determination theory framework, autonomy, competence, and relatedness. Past research has mainly focused on the relatedness and attainability of moral exemplars for predicting motivation outcomes. The data for…

Course: Communication Research Methods. Objectives: This activity provides students with an experiential introduction to measurement theory and the methods for assessing measurement reliability. First, multiple measurements of a person's height are interpreted according to classical test theory. Second, the measurement of human height is used as…

Research Findings: Given the variable nature of early childhood settings, practitioners and researchers need better guidance on what conditions influence observations conducted within early childhood settings (National Research Council, 2008). Using 230 observations from 23 three- and four-year-old children, we conducted a Generalizability study…