Background: The generalizability of a study's results continues to be at the forefront of concerns in evaluation research in education (Tipton & Olsen, 2018). Over the past decade, statisticians have developed methods, mainly based on propensity scores, to improve generalizations in the absence of random sampling (Stuart et al., 2011; Tipton,…

Background: Evidence from education randomized controlled trials (RCTs) in low- and middle-income countries (LMICs) demonstrates how interventions can improve children's educational achievement [1, 2, 3, 4]. RCTs assess the impact of an intervention by comparing outcomes--aggregate test scores--between treatment and control groups. A review of…

This paper examines the scalability of the results from the Tennessee Student-Teacher Achievement Ratio (STAR) Project, a prominent educational experiment. We explore how the misalignment between the experimental design and the econometric model affects researchers' ability to learn about the intervention's scalability. We document heterogeneity…

The purpose of this study is to develop a nonparametric DIF method that (a) compares focal groups directly to the composite group that will be used to develop the reported test score scale, and (b) allows practitioners to explore for DIF related to focal groups stemming from multicategorical variables that constitute a small proportion of the…

Inquiry learning is an effective learning approach if learners are properly guided. Its effectiveness depends on learners' prior knowledge, the domain, and their relationship. In a previous study we developed an Experiment Design Tool (EDT) guiding learners in designing experiments. The EDT significantly benefited low prior knowledge learners. For…

Minimum discriminant information adjustment (MDIA), an approach to weighting samples to conform to known population information, provides a generalization of raking and poststratification. In the case of simple random sampling with replacement with uniform sampling weights, large-sample properties are available for MDIA estimates of population…

Propensity score analysis is a statistical method that balances pre-existing differences across treatment conditions achieving a similar condition as randomization and thus, allowing the estimation of causal effects in non-randomized experimental designs. The four stages in propensity score analysis are (1) propensity score estimation, (2)…

In randomized control trials (RCT), the recent focus has shifted to how an intervention yields positive results on its intended outcome. This aligns with the recent push of implementation science in healthcare (Bauer et al., 2015) but goes beyond this. RCTs have moved to evaluating the theoretical framing of the intervention as well as differing…

This article reviews important statistical methods for estimating the impact of interventions on outcomes in education settings, particularly programs that are implemented in field, rather than laboratory, settings. We begin by describing the causal inference challenge for evaluating program effects. Then four research designs are discussed that…

This study investigated the impact of a multicomponent training program on the American School Counselor Association's National Model of school counseling. The model was designed to enhance school counselors' competence in implementing a comprehensive school counseling program. An ABA single case design was used to collect baseline, training, and…

Traditional pre-test (TpT)/post-test (PT) and retrospective pre-test (RpT)/post-test (PT) designs are used to collect data on self-reported measures to assess the magnitude of change that occurs from interventions. If measurement invariance does not exist across the measurement occasions within these research designs, it is inappropriate to…

To plan cluster-randomized trials with sufficient statistical power to detect intervention effects on student achievement, researchers need multilevel design parameters, including measures of between-classroom and between-school differences and the amounts of variance explained by covariates at the student, classroom, and school level. Previous…

Head Start is a national program that helps young children from families with low income get ready to succeed in school. It does this by working to promote their early learning and health and their families' well-being. The Head Start Family and Child Experiences Survey (FACES) provides national information about Head Start programs and…

Ratio and proportional relationships, along with the interrelated topics of fractions,decimals, and percent, provide a critical foundation for algebra (National Mathematics Advisory Panel, 2008). Proportional reasoning, which not only requires understanding the concept of ratios and that two or more ratios are equal, also requires the ability to…

The Programme for International Student Assessment (PISA) assesses the knowledge and skills in mathematics, reading and science of 15-year-old pupils in countries around the world. PISA is run by the Organisation for Economic Co-operation and Development (OECD), and assessment is typically undertaken every 3 years, allowing us to chart how…