The central roles of science in the field of remedial and special education are to (a) identify basic laws of nature and (b) apply those laws in the design of practices that achieve socially valued outcomes. The scientific process is designed to allow demonstration of specific (typically positive) outcomes, and to assist in the attribution of…

To counteract what we see as a growing research-reporting concern, we propose the following editorial-policy change regarding the content of primary research articles in educational research journals: Contributors should restrict their discussion and conclusions to their data and not offer recommendations for educational practice nor speculate…

Two previous studies examining 5 empirical educational psychology research journals (Hsieh et al., 2005; Robinson, Levin, Thomas, Pituch, & Vaughn, 2007) found that in the 21-year period from 1983 to 2004, there was a decrease in intervention and randomized experimental research, whereas in the 10-year period from 1994 to 2004, there was an…

In recent years, single-case designs have increasingly been used to establish an empirical basis for evidence-based interventions and techniques in a variety of disciplines, including psychology and education. Although traditional single-case designs have typically not met the criteria for a randomized controlled trial relative to conventional…

Outlines concerns that must be addressed by those who advocate replacing statistical hypothesis-testing with alternative data-analysis strategies. Suggests that commonly recommended alternatives are not perfect and that various hypothesis-testing modifications can be implemented to make the process and its conclusions more credible. Hypothesis…

Presents a general class of single-case statistical procedures derived from previously developed nonparametric randomization tests. Designs are illustrated that focus on the general and comparative effectiveness of alternative interventions, multiple units with differentiable characteristics, and multiple outcome measures. Provides operational…

This report presents the problem and a solution to the treatment of students by age interactions. It considers the plight of an educational researcher who wishes to demonstrate that a particular treatment effect changes from age to age. In analysis of variance language, he is seeking a treatment by age interaction. The report presents a rationale…

Proposes modifications to the recent suggestions by B. Thompson (1996) for an American Educational Research Association editorial policy on statistical significance testing. Points out that, although it is useful to include effect sizes, they can be misinterpreted, and argues, as does Thompson, for greater attention to replication in educational…

The use of analysis of variance (ANOVA), multivariate analysis of variance (MANOVA), and analysis of covariance (ANCOVA) by educational researchers was studied through review of several prominent research journals. The analyses suggest that researchers rarely verify that validity assumptions are satisfied, and typically use analyses that are…

Because of criticisms leveled at statistical hypothesis testing, some researchers have argued that measures of effect size should replace the significance-testing practice. We contend that although effect-size measures have logical appeal, they are also associated with a number of limitations that may result in problematic interpretations of them…

Journal editors respond to criticisms of reliance on statistical significance in research reporting. Joel R. Levin ("Journal of Educational Psychology") defends its use, whereas William D. Schafer ("Measurement and Evaluation in Counseling and Development") emphasizes the distinction between statistically significant and important. William Asher…