Many teacher education researchers have expressed concerns about the lack of rigorous impact evaluations of teacher preparation practices. I summarize these various concerns as they relate to issues of internal validity, measurement, and external validity. I then assess the prevalence of these issues by reviewing 166 impact evaluations of teacher…

The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

The problem of assessing whether experimental results can be replicated is becoming increasingly important in many areas of science. It is often assumed that assessing replication is straightforward: All one needs to do is repeat the study and see whether the results of the original and replication studies agree. This article shows that the…

The problem of assessing whether experimental results can be replicated is becoming increasingly important in many areas of science. It is often assumed that assessing replication is straightforward: All one needs to do is repeat the study and see whether the results of the original and replication studies agree. This article shows that the…

Treatment effect estimates from a "regression discontinuity design" (RDD) have high internal validity. However, the arguments that support the design apply to a subpopulation that is narrower and usually different from the population of substantive interest in evaluation research. The disconnect between RDD population and the…

The article by Davis, Engberg, Epple, Sieg, and Zimmer (2010) represents one of the recent research efforts from economists in evaluating the impact of gifted programs. It can serve as a worked example of the implementation of the regression discontinuity (RD) design method in gifted education research. In this commentary, we first illustrate the…

This paper reviews literature on the rationale, challenges, and recommendations for choosing a nonequivalent comparison (NEC) group design when evaluating intervention effects. After reviewing frequently addressed threats to validity, the paper describes recommendations for strengthening the research design and how the recommendations were…

This chapter provides a selective review of some contemporary approaches to program evaluation. One motivation for our review is the recent emergence and increasing use of a particular kind of "program" in applied microeconomic research, the so-called Regression Discontinuity (RD) Design of Thistlethwaite and Campbell (1960). We organize our…

The authors of this article consider how education researchers can improve school violence and school safety research by (a) examining gaps in theoretical, conceptual, and basic research on the phenomena of school violence; (b) reviewing key issues in the design and evaluation of evidence-based practices to prevent school violence; and (c)…

A single-case (B-C-B-C) experimental design was used to evaluate the effects of the Color Wheel classroom management system (CWS) on on-task (OT) behavior in an intact, general-education, 2nd-grade classroom during transitions. The CWS included three sets of rules, posted cues to indicate the rules students are expected to be following at that…

This article discusses the difficulties in conducting intervention research or evaluating intervention programs in a school setting. In particular, the problems associated with randomization and obtaining control groups are examined. The use of quasi-experimental designs, specifically a paired comparison design using the individual as his or her…

A design for evaluation studies of educational programs should provide valid and defensible inferences. Goals of evaluation are the identity of major components of inferences and specific validity concerns. Design problems may be resolved by creatively utilizing features of specific evaluations in designing unique conditions that permit valid…

In no-difference research, no differences are found among groups or conditions. This article summarizes the existing commentary on such research. The characteristics of no-difference research, its acceptance by the research community, strategies for conducting such studies, and its centrality within the experimental and nonexperimental paradigms…

We rely a great deal on the schematic descriptions that represent experimental and quasi-experimental design arrangements, as well as the discussions of threats to validity associated with these, provided by Campbell and his associates: Stanley, Cook, and Shadish. Some of these designs include descriptions of treatments removed, removed and then…

In an overview of single-case methodology, the potential utility of A-B-A and multiple baseline designs for evaluating social programs is discussed. Validity factors and cost-effectiveness are considered, showing that these designs are viable alternative methods where traditional randomized group designs are infeasible. (Author/CM)