This paper seeks to raise awareness among educational researchers and practitioners of some significant weaknesses and internal contradictions of randomised control trials (RCTs). Although critiques throughout the years from education scholars have pointed to the detrimental effects of this experimental approach on education practice and values,…

Background: Randomised controlled trials (RCTs) have long been considered the gold standard in education research. Federal funds are allocated to evaluations that meet What Works Clearinghouse standards; RCT designs are required in order to meet these standards without reservations. Schools seek out interventions that are research based, in other…

Evaluators of education interventions are increasingly designing studies to detect impacts much smaller than the 0.20 standard deviations that Cohen characterized as "small." While the need to detect smaller impacts is based on compelling arguments that such impacts are substantively meaningful, the drive to detect smaller impacts may…

Evaluators of education interventions increasingly need to design studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." For example, an evaluation of Response to Intervention from the Institute of Education Sciences (IES) detected impacts ranging from 0.13 to 0.17 standard…

Background: The Education Endowment Foundation (EEF) is an independent charity that was established in 2011 with the explicit aim of breaking the link between family income and educational achievement in England. Over the seven years since its inception, EEF has contributed to the existing evidence base by funding over one hundred randomised…

In recent years, underserved communities have received much attention within the field of positive youth development (PYD). While there are an increasing number of opportunities for underserved youth to join PYD-based programs, there is growing concern surrounding the lack of critical evaluation of these programs (Kidd, 2008). Unfortunately, there…

Background: Social experiments frequently exploit data from administrative records. However, most administrative data systems are designed to track earnings or benefit payments among residents within a single state. When an experimental participant moves across state lines, his entries in the data system of his state of origin consist entirely of…

Drawing on our five years of experience developing WWC evidence standards and reviewing studies against those standards as well as current literature on the design of impact studies, we highlight in this paper some of the most critical issues and common pitfalls in designing and conducting impact studies in education, and provide practical…

E-learning researchers face considerable challenges in creating meaningful and generalizable studies due to the complex nature of this dynamic training medium. Our experience in conducting workplace e-learning research led us to create this guide for planning research on e-learning. We share the unanticipated complications we encountered in our…

Background: Those involved in school health education programs generally believe that health-education programs can play an important role in helping young people make positive health decisions. Thus, it is to document the effects of such programs through rigorous evaluations published in peer-reviewed journals. Methods: This paper helps the…

This paper introduces an empirical strategy to estimate dynamic treatment effects in randomized trials that provide treatment in multiple stages and in which various noncompliance problems arise such as attrition and selective transitions between treatment and control groups. Our approach is applied to the highly influential four year randomized…

Background: Disability research data often exist in the form of individual records located within discrete registers that may extend across sensitive political boundaries. Method: This paper discusses the opportunities and challenges associated with using linked health and administrative data for disability research, with examples from research…

Data from the Sustaining Effects Study were used to investigate the impact of attrition. Four types of analyses were conducted: estimation of attrition rate by project, estimation of bias by project, prediction of bias at the project level, and estimation of bias associated with alternative aggregation procedures. Effects of attrition varied with…

Examines the problem of attrition from therapeutic treatment programs, discussing the scope and limitations of research about the causes of subject attrition. Examines the applicability of proposed methodological remedies for assessing and countering attrition. Such research often suffers from inadequate conceptualization and weak design. (SLD)

The study under review is the second-year evaluation report of the Milwaukee Parental Choice Program (MPCP), a publicly funded voucher program that allows low-income students in Milwaukee to attend secular and religious private schools in that city. Its primary finding is that there were no overall statistically significant differences in…