In any meta-analysis, it is critically important to report the dispersion in effects as well as the mean effect. If an intervention has a moderate clinical impact "on average" we also need to know if the impact is moderate for all relevant populations, or if it varies from trivial in some to major in others. Or indeed, if the…

Mediation analysis in Single Case Experimental Designs (SCEDs) evaluates intervention mechanisms for individuals. Despite recent methodological developments, no clear guidelines exist for maximizing power to detect the indirect effect in SCEDs. This study compares frequentist and Bayesian methods, determining (1) minimum required sample size to…

Well-chosen covariates boost the design sensitivity of individually and cluster-randomized trials. We provide guidance on covariate selection generating an extensive compilation of single- and multilevel design parameters on student achievement. Embedded in psychometric heuristics, we analyzed (a) covariate "types" of varying…

In education settings, treatments are often non-randomly assigned to clusters, such as schools or classrooms, while outcomes are measured for students. This research design is called the clustered observational study (COS). We examine the consequences of common support violations in the COS context. Common support violations occur when the…

Random encouragement designs evaluate treatments that aim to increase participation in a program or activity. These randomized controlled trials (RCTs) can also assess the mediated effects of participation itself on longer term outcomes using a complier average causal effect (CACE) estimation framework. This article considers power analysis…

To date, there have been four responses (Dumas & Edelsbrunner, "Educational Psychology Review," 35, 48, 2023; Grosz, "Educational Psychology Review," 35, 57, 2023; Mayer, "Educational Psychology Review," 35, 64, 2023; Zitzmann et al., "Educational Psychology Review," 35, 65, 2023) to the Brady et al.…

Evidence-based health interventions are frequently translated into real-world settings where practical needs drive changes to intervention protocols. Due to logistical and resource constraints, these naturally arising adaptations are rarely assessed for comparative effectiveness using a randomized trial. Nevertheless, when observational data are…

Background: Studying causal effects is an important aim in education. Causal relationships indicate how well implements (e.g., interventions) work for the target subjects. A good strategy to get the inference in such relationships is to conduct randomized experiments. However, random assignment is limited in education research, even is discouraged…

The randomized control trial (RCT) is the primary experimental design in education research due to its strong internal validity for causal inference. However, in situations where RCTs are not feasible or ethical, quasi-experiments are alternatives to establish causal inference. This paper serves as an introduction to several quasi-experimental…

Background: Randomized controlled trials (RCTs), or experiments, are the gold standard for intervention evaluation. However, the main appeal of RCTs--the clean identification of causal effects--can be compromised by interference, when one subject's actions can influence another subject's behavior or outcomes. In this paper, we formalize and study…

Previous simulation studies of randomization tests applied in single-case educational intervention research contexts have typically focused on A-to-B phase changes in means/levels. In the present simulation study, we report the results of two multiple-baseline investigations, one targeting between-phase changes in slopes/trends and the other…

Qualitative comparative analysis (QCA) is a hybrid method designed to bridge the gap between qualitative and quantitative research in a case-sensitive approach that considers each case holistically as a complex configuration of conditions and outcomes. QCA allows for multiple conjunctural causation, implying that it is often a combination of…

Random encouragement designs are randomized controlled trials (RCTs) that test interventions aimed at increasing participation in a program or activity whose take up is not universal. In these RCTs, instead of randomizing individuals or clusters directly into treatment and control groups to participate in a program or activity, the randomization…

The present simulation study examined the statistical properties (namely, Type I error and statistical power) of various novel randomized single-case multiple-baseline designs and associated randomized-test analyses for comparing the A- to B-phase immediate abrupt outcome changes in two independent intervention conditions. It was found that with…

Randomized controlled trials have proliferated in education, in part because they provide an unbiased estimator for the causal impact of interventions. It is increasingly recognized that many such trials in education have low power to detect an effect if indeed there is one. However, it is less well known that low powered trials tend to…