Given the American Psychological Association's strong recommendation to always report effect sizes in research, scholars have a responsibility to provide complete information regarding their findings. The purposes of this study were to (a) determine the frequencies with which different effect sizes were reported in published, peer-reviewed…

The present study was conducted to characterize (a) the features of the thousands of primary reports synthesized in 47 reliability generalization (RG) measurement meta-analysis studies and (b) typical methodological practice within the RG literature to date. With respect to the treatment of score reliability in the literature, in an astounding…

The purpose of the present study is to provide a historical account and metasynthesis of which statistical techniques are most frequently used in the fields of education and psychology. Six articles reviewing the "American Educational Research Journal" from 1969 to 1997 and five articles reviewing the psychological literature from 1948 to 2001…

The present article provides a primer on using effect sizes in research. A small heuristic data set is used in order to make the discussion concrete. Additionally, various admonitions for best practice in reporting and interpreting effect sizes are presented. Among these is the admonition to not use Cohen's benchmarks for "small," "medium," and…

Researchers have conducted studies of curriculum for doctoral students in various disciplines and subdisciplines. In this study, the authors examined methods (e.g., quantitative, qualitative, mixed methods) requirements for doctoral programs in education. Such requirements operationalize visions about the meaning of educational research and the…

This article reviews the literature on interventions targeting the academic performance of students with attention-deficit/hyperactivity disorder (ADHD) and does so within the context of the statistical significance testing controversy. Both the arguments for and against null hypothesis statistical significance tests are reviewed. Recent standards…

The present article provides a primer on (a) effect sizes, (b) confidence intervals, and (c) confidence intervals for effect sizes. Additionally, various admonitions for reformed statistical practice are presented. For example, a very important implication of the realization that there are dozens of effect size statistics is that "authors must…

Only true experiments offer definitive evidence for causal inferences, but not all educational interventions are readily amenable to experiments. Correlational evidence can at least tentatively inform evidence-based practice when sophisticated causal modeling or exclusion methods are employed. Correlational evidence is most informative when…

This article sets the context for the development of research quality indicators and guidelines for evidence of effective practices provided by different methodologies. The current conceptualization of scientific research in education and the complexity of conducting research in special education settings underlie the development of quality…

Arthur Levine's (2007) recent report on the academic preparation of education doctoral students, Educating Researchers, http://www.edschools.org/EducatingResearchers/educating_researchers.pdf is highly critical of both AERA and the scholarly training of education doctoral students. One criticism Levine (2007) offers is his view that educational…

Given some consensus that statistical significance tests are broken, misused, or at least have somewhat limited utility, the focus of discussion within the field ought to move beyond additional bashing of statistical significance tests, and toward more constructive suggestions for improved practice. Five suggestions for improved practice are…

Explains basic logic of canonical analysis, illustrates that canonical analysis is general parametric analytic method subsuming other methods, and offers guidelines regarding use of this analytic approach. Concludes that canonical analysis is potent because it does not require researcher to discard variance of any variables and because it honors…

T. Vacha-Haase (1998) proposed her "reliability generalization" methodology to characterize (a) typical score reliability for a measure across studies, (b) the variability of score reliabilities, and (c) what measurement protocol features predict the variability in score reliabilities across administration. The present article provides…

The present article presents a tutorial on how to estimate and interpret various effect sizes. The 5th edition of the Publication Manual of the American Psychological Association (2001) described the failure to report effect sizes as a "defect" (p. 5), and 23 journals have published author guidelines requiring effect size reporting. Although…

Three problems with stepwise research methods are explored. Computer packages may use incorrect degrees of freedom in stepwise computations. In addition, stepwise methods do not identify correctly the best variable set of a given size. A third problem is that stepwise methods tend to capitalize on sampling error. (SLD)