We describe best practices in reporting reliability estimates in counseling research with consideration to precision, generalization, and diverse populations. We provide a historical context to reporting reliability estimates, the limitations of past practices, and new methods to address reliability generalization. We highlight best practices…

Network meta-analysis has been an essential methodology of systematic reviews for comparative effectiveness research. The restricted maximum likelihood (REML) method is one of the current standard inference methods for multivariate, contrast-based meta-analysis models, but recent studies have revealed the resultant confidence intervals of average…

Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences. Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational…

Meta-analyses are often used to synthesize the findings of studies examining the correlational relationship between two continuous variables. When only dichotomous measurements are available for one of the two variables, the biserial correlation coefficient can be used to estimate the product-moment correlation between the two underlying…

A network meta-analysis allows a simultaneous comparison between treatments evaluated in randomised controlled trials that share at least one treatment with at least one other study. Estimates of treatment effects may be required for treatments across disconnected networks of evidence, which requires a different statistical approach and modelling…

Launching biomedical innovations based on clinical demands instead of translating basic research findings to practice reduces the risk that the results will not fit the clinical routine. To realize this type of innovation, a meta-analysis of the body of research is necessary to reveal demand-matching concepts. However, both the data deluge and the…

Methods for meta-analyzing single-case designs (SCDs) are needed in order to inform evidence based practice in special education and to draw broader and more defensible generalizations in areas where SCDs comprise a large part of the research base. The most widely used outcomes in single-case research are measures of behavior collected using…

Evidence-based practices (EBPs) are supported as generally effective for populations of learners by bodies of high-quality and experimental research and, when aligned with stakeholder values and practical needs, should be prioritized for implementation. However, evidence-based practices are not currently available for all learner types in all…

Confidence intervals (CIs) and effect sizes are essential to encourage meta-analytic thinking and to accumulate research findings. CIs provide a range of plausible values for population parameters with a degree of confidence that the parameter is in that particular interval. CIs also give information about how precise the estimates are. Comparison…

There is general agreement that meta-analysis is an important tool for synthesizing study results in quantitative educational research. Yet, a shared feature of many meta-analyses is a failure to report sufficient information for readers to fully judge the reported findings, such as the populations to which generalizations are to be made,…

As meta-analytic studies have become essential in educational research, resampling methods have matured and are supported by an in-depth theoretical and applied literature base. The objective of this paper is to assist researchers in applying resampling methods to the results of meta-analysis. The rationale for resampling methods is reviewed, and…

This paper discusses the impact of sampling error on the construction of confidence intervals around effect sizes. Sampling error affects the location and precision of confidence intervals. Meta-analytic resampling demonstrates that confidence intervals can haphazardly bounce around the true population parameter. Special software with graphical…