Monitoring the fidelity with which implementers implement behavioral procedures is important, but individuals tasked with monitoring fidelity may have received little training. As a result, their data may be affected by variations in implementers' performance, such as the frequency of fidelity errors. To determine the extent to which the frequency…

Data extraction is a crucial, yet labor-intensive and error-prone part of evidence synthesis. To date, efforts to harness machine learning for enhancing efficiency of the data extraction process have fallen short of achieving sufficient accuracy and usability. With the release of large language models (LLMs), new possibilities have emerged to…

Recent approaches to the detection of cheaters in tests employ detectors from the field of machine learning. Detectors based on supervised learning algorithms achieve high accuracy but require labeled data sets with identified cheaters for training. Labeled data sets are usually not available at an early stage of the assessment period. In this…

The product Bayes factor (PBF) synthesizes evidence for an informative hypothesis across heterogeneous replication studies. It can be used when fixed- or random effects meta-analysis fall short. For example, when effect sizes are incomparable and cannot be pooled, or when studies diverge significantly in the populations, study designs, and…

With the vigorous development of intelligent campus construction, great changes have taken place in the development of information technology in colleges and universities from the previous digital to intelligent development. In the teaching process, the analysis of students' classroom learning has also changed from the previous manual observation…

Interrupted time series (ITS) studies are frequently used to examine the impact of population-level interventions or exposures. Systematic reviews with meta-analyses including ITS designs may inform public health and policy decision-making. Re-analysis of ITS may be required for inclusion in meta-analysis. While publications of ITS rarely provide…

Background: During systematic reviews, "data abstraction" refers to the process of collecting data from reports of studies. The data abstractors' level of experience may affect the accuracy of data abstracted. Using data from a randomized crossover trial in which different data abstraction approaches were compared, we examined the…

Educational data mining has allowed for large improvements in educational outcomes and understanding of educational processes. However, there remains a constant tension between educational data mining advances and protecting student privacy while using educational datasets. Publicly available datasets have facilitated numerous research projects…

Social sciences expose many cognitively complex, highly qualified, or fuzzy problems, whose resolution relies primarily on expert judgement rather than automated systems. One of such instances that we study in this work is a reflection analysis in the writings of student teachers. We share a hands-on experience on how these challenges can be…

This article provides a conceptual framework for considering both the theoretical and methodological factors that underpin the successful validation of a competency framework. Drawing on educational assessment literature, this article argues that a valid competency framework relates to an interpretive judgement of the credibility of the claims…

Many of the decisions made related to student progress and intervention selection rely on data. In their roles, paraeducators often implement academic interventions; therefore, they must be prepared to monitor students' progress accurately and reliably. This pilot randomized control trial tested the effects of a remote training package on…

Purpose: This tutorial outlines an approach for best practices for speech-language pathology assistants (SLPAs) to collect data and document services. The tutorial outlines methods for developing accurate and effective data collection skills and provides instructions and tools for collecting various types of data. The authors discuss the…

Research Context: In educational research, "context effects" are often of inferential interest to researchers as well as of evaluative interest to policymakers. While student education outcomes likely depend on individual-level influences like individual academic achievement, school contexts may also make a difference. Such questions are…

Students' academic performance prediction is one of the most important applications of Educational Data Mining (EDM) that helps to improve the quality of the education process. The attainment of student outcomes in an Outcome-based Education (OBE) system adds invaluable rewards to facilitate corrective measures to the learning processes.…

During data collection, field interviewers often append notes or comments to a case in open text fields to request updates to case-level data. Processing these comments can improve data quality, but many are non-actionable, and processing remains a costly manual task. This article presents a case study using a novel application of machine learning…