Given the American Psychological Association's strong recommendation to always report effect sizes in research, scholars have a responsibility to provide complete information regarding their findings. The purposes of this study were to (a) determine the frequencies with which different effect sizes were reported in published, peer-reviewed…

This article reviews the literature on interventions targeting the academic performance of students with attention-deficit/hyperactivity disorder (ADHD) and does so within the context of the statistical significance testing controversy. Both the arguments for and against null hypothesis statistical significance tests are reviewed. Recent standards…

Three problems with stepwise research methods are explored. Computer packages may use incorrect degrees of freedom in stepwise computations. In addition, stepwise methods do not identify correctly the best variable set of a given size. A third problem is that stepwise methods tend to capitalize on sampling error. (SLD)

Canonical correlation analysis is illustrated and three common fallacious interpretation practices are described. Simply, canonical correlation is an example of the bivariate case. Like all parametric methods, it involves the creation of synthetic scores for each person. It presumes at least two predictor variables and at least two criterion…

Three criticisms of conventional uses of structural significance testing are elaborated; and alternatives for augmenting statistical significance tests are reviewed, which include emphasizing effect size, evaluating statistical significance in a sample size context, and evaluating result replicability. Among ways of estimating result…

A study examined effect-size reporting in 23 quantitative articles reported in "Exceptional Children". Findings reveal that effect sizes are rarely being reported, although exemplary reporting practices were also noted. Reasons why encouragement by the American Psychological Association to report effect size has been ineffective are…

As an extension of B. Thompson's 1998 invited address to the American Educational Research Association, this paper cites two additional common faux pas in research methodology and explores some research issues for the future. These two errors in methodology are the use of univariate analyses in the presence of multiple outcome variables (with the…

Factor analysis can be a valuable tool for assessing validity; however, the reliabilities of identified factors must be attended to. This study examined the impacts of item variance on factor stability and structure. The attempt was to determine empirically whether unnumbered graphic scales scored using a larger number of scale steps produce more…

Three criticisms of overreliance on results from statistical significance tests are noted. It is suggested that: (1) statistical significance tests are often tautological; (2) some uses can involve comparisons that are not completely sensible; and (3) using statistical significance tests to evaluate both methodological assumptions (e.g., the…

Problems with using stepwise analytic methods are discussed, and better alternatives are illustrated. To make the illustrations concrete, an actual data set, involving responses of 91 medical school admissions directors to 30 variables, was used. The 30 variables involved perceptions of barriers to medical school with respect to characteristics of…

Dissertations are the cumulative, tangible "best evidence" of interests of doctoral faculty and students in serious and incisive scholarship. Thus, dissertations are thoroughly studied by the program review teams periodically hired by boards of higher education in most states. The present paper explores seven errors in quantitative…

The research literature provides important guidance to counselors working to keep abreast of the latest thinking regarding best practices and recently developed counseling tools. The purpose of this digest is to highlight a few errors that seem to recur within the literature, and to provide some helpful references that further explore these…

Too few researchers understand what statistical significance testing does and does not do, and consequently their results are misinterpreted. This Digest explains the concept of statistical significance testing and discusses the meaning of probabilities, the concept of statistical significance, arguments against significance testing,…

Hypothetical data sets are used to demonstrate how canonical correlation methods subsume other commonly utilized parametric methods. Analysis of variance, analysis of covariance, multiple analysis of variance, and multiple analysis of covariance are heavily used by educational researchers. It is concluded that researchers would do well to consider…

Researchers too frequently consider the reliability of the scores they analyze, and this may lead to incorrect conclusions. Practice in this regard may be negatively influenced by telegraphic habits of speech implying that tests possess reliability and other measurement characteristics. Styles of speaking in journal articles, in textbooks, and in…