In a clustered observational study, a treatment is assigned to groups and all units within the group are exposed to the treatment. We develop a new method for statistical adjustment in clustered observational studies using approximate balancing weights, a generalization of inverse propensity score weights that solve a convex optimization problem…

Single-case design (SCD) is a quantitative experimental technique in which participants serve as their own control. The use of an effect size in SCD allows evaluation of outcomes as well as comparison of outcomes via meta-analyses. Characteristics of SCD research make the selection of an appropriate effect size complicated. Additionally, there are…

Recent interest in promoting replication efforts assumes that there is well-established methodological guidance for designing and implementing these studies. However, no such consensus exists in the methodology literature. This article addresses these challenges by describing design-based approaches for planning systematic replication studies. Our…

Linear panel models featuring unit and time fixed effects appear in many areas of empirical economics. An active literature studies the interpretation of the ordinary least squares estimator of the model, commonly called the two-way fixed effects (TWFE) estimator, in the presence of unmodeled coefficient heterogeneity. We illustrate some…

In 2020, the What Works Clearinghouse (WWC) for single-case experimental design (SCED) studies released an updated version of their standards and procedures handbook (Version 4.1). Because of these updates, there is a need to understand the implications for the field in terms of quality rating of the design, and subsequent synthesis of evidence at…

Single-case experimental designs (SCEDs) have been used with increasing frequency to identify evidence-based interventions in education. The purpose of this study was to explore how several procedural characteristics, including within-phase variability (i.e., measurement error), number of baseline observations, and number of intervention…

What is the science of reading? The Reading League, in their document "Science of Reading: Defining Guide" (TRL, 2022), proposes that "The science of reading is a vast, interdisciplinary body of scientifically-based* research about reading and issues related to reading and writing." (p. 11.) The asterisk is carrying a lot of…

Currently the design standards for single case experimental designs (SCEDs) are based on validity considerations as prescribed by the What Works Clearinghouse. However, there is a need for design considerations such as power based on statistical analyses. We compute and derive power using computations for (AB)[superscript k] designs with multiple…

The field of education relies heavily on instructional coaches to build teacher capacity in the implementation of evidence-based practices (EBPs). Although observation tools are commonly used to measure the fidelity of implementation by teachers, fewer tools are available to identify specific coaching behaviors used during in situ coaching…

The COVID-19 pandemic caused tremendous upheaval in schooling. In addition to its devasting effects on students' academic development, the disruptions to schooling had important consequences for researchers conducting effectiveness studies on educational programs during this era. Given the likelihood of future large-scale disruptions, it is…

Single case experimental designs are an important research design in behavioral and medical research. Although there are design standards prescribed by the What Works Clearinghouse for single case experimental designs, these standards do not include statistically derived power computations. Recently we derived the equations for computing power for…

The clustered observational study (COS) design is the observational study counterpart to the clustered randomized trial. In a COS, a treatment is assigned to intact groups, and all units within the group are exposed to the treatment. However, the treatment is non-randomly assigned. COSs are common in both education and health services research. In…

Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a…

Although statistical practices to evaluate intervention effects in SCEDs have gained prominence in the recent times, models are yet to incorporate and investigate all their analytic complexities. Most of these statistical models incorporate slopes and autocorrelations both of which contribute to trend in the data. The question that arises is…

In cluster randomized evaluations, a treatment or intervention is randomly assigned to a set of clusters each with constituent individual units of observations (e.g., student units that attend schools, which are assigned to treatment). One consideration of these designs is how many units are needed per cluster to achieve adequate statistical…