Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences. Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational…

Background/Context: Many students in the United States need access to effective writing instruction, with some requiring intensive intervention. Results from the National Assessment of Education Progress (NAEP) in writing over the past 25 years revealed that most students in the U.S. have yet to attain a proficient level of written expression…

The field of education is increasingly committed to adopting evidence-based practices. Although randomized experimental designs provide strong evidence of the causal effects of interventions, they are not always feasible. For example, depending upon the research question, it may be difficult for researchers to find the number of children necessary…

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…

Systematic reviews and meta-analysis are techniques for synthesizing primary empirical studies to produce a summary of effects. To facilitate this goal, the Campbell Collaboration (C2) supports reviews within the disciplines of crime and justice, education, international development, and social welfare. At the annual Campbell Colloquium, experts…

This succinct and jargon-free introduction to effect sizes gives students and researchers the tools they need to interpret the practical significance of their results. Using a class-tested approach that includes numerous examples and step-by-step exercises, it introduces and explains three of the most important issues relating to the practical…

One of the main objectives in meta-analysis is to estimate the overall effect size by calculating a confidence interval (CI). The usual procedure consists of assuming a standard normal distribution and a sampling variance defined as the inverse of the sum of the estimated weights of the effect sizes. But this procedure does not take into account…

Objective: To clarify the nature and extent of differences in the ways that physically abusive, neglectful, and non-maltreating parents communicate during interactions with their children. Method: A meta-analysis was conducted of 33 observational studies comparing parent-child interactions in families where parents have a documented history of…

Multisite research designs involving cluster randomization are becoming increasingly important in educational and behavioral research. Researchers would like to compute effect size indexes based on the standardized mean difference to compare the results of cluster-randomized studies (and corresponding quasi-experiments) with other studies and to…

Using meta-analysis, randomized experiments in education that either clearly did or clearly did not experience student attrition were examined for the baseline comparability of groups. Results from 35 studies suggested that after attrition, the observed measures of baseline comparability of groups did not differ more than would be expected given…

Over the years, methodologists have been recommending that researchers use magnitude of effect estimates in result interpretation to highlight the distinction between statistical and practical significance (cf. R. Kirk, 1996). A magnitude of effect statistic (i.e., effect size) tells to what degree the dependent variable can be controlled,…

One purpose for combining research studies is to estimate a population treatment effect. The internal validity of a model for how effect size estimates should be computed and combined will hinge upon the homogeneity of the effect size variation. Effect size variation may be assessed in the form of a summary fit statistic, and a direct…

This manual and the accompanying software are intended to provide a step-by-step guide to conducting a meta-analytic study along with references for further reading and free high-quality software, "Meta-Stat.""Meta-Stat" is a comprehensive package designed to help in the meta-analysis of research studies in the social and behavioral sciences.…

The random variable p and its functions figure in several "tests of combined significance," meta-analysis summaries based on sample significance values, and ps have been used singly, as well as in other tests for evaluating the outcomes of individual research studies. In this work, asymptotic distributions of the sample one-sided…

Power is an indicator of the ability of a statistical analysis to detect a phenomenon that does in fact exist. The issue of power is crucial for social science research because sample size, effects, and relationships studied tend to be small and the power of a study relates directly to the size of the effect of interest and the sample size.…