In any meta-analysis, it is critically important to report the dispersion in effects as well as the mean effect. If an intervention has a moderate clinical impact "on average" we also need to know if the impact is moderate for all relevant populations, or if it varies from trivial in some to major in others. Or indeed, if the…

As a measure of heterogeneity in meta-analysis, the coefficient of variation (CV) has been recently considered, providing researchers with a complement to the very popular I[superscript 2] measure. While I[superscript 2] measures the proportion of total variance that is due to variance of the random effects, the CV is the ratio of the standard…

Extracting data from studies is the norm in meta-analyses, enabling researchers to generate effect sizes when raw data are otherwise not available. While there has been a general push for increased reproducibility in meta-analysis, the transparency and reproducibility of the data extraction phase is still lagging behind. Unfortunately, there is…

Statistical methods are increasingly being used to integrate findings from the ever-expanding universe of empirical research. Meta-analysis encompasses various techniques for synthesizing summary statistics, and mega-analysis pools raw data across studies. This paper offers an introduction to meta-analysis and mega-analysis that complements the…

Before embarking on an individual participant data meta-analysis (IPDMA) project, researchers should consider the power of their planned IPDMA conditional on the studies promising their IPD and their characteristics. Such power estimates help inform whether the IPDMA project is worth the time and funding investment, before IPD are collected. Here,…

Meta-analysis is a widely used methodology to combine evidence from different sources examining a common research phenomenon, to obtain a quantitative summary of the studied phenomenon. In the medical field, multiple studies investigate the effectiveness of new treatments and meta-analysis is largely performed to generate the summary (average)…

We present the R package and web app "statcheck" to automatically detect statistical reporting inconsistencies in primary studies and meta-analyses. Previous research has shown a high prevalence of reported p-values that are inconsistent--meaning a re-calculated p-value, based on the reported test statistic and degrees of freedom, does…

Toward the goal of more rapid knowledge accumulation via better meta-analyses, this article explores statistical approaches intended to increase the precision and comparability of effect sizes from education research. The featured estimate of the proposed approach is a standardized mean difference effect size whose numerator is a mean difference…

A systematic review and meta-analysis is an important step in evidence synthesis. The current paradigm for meta-analyses requires a presentation of the means under a random-effects model; however, a mean with a confidence interval provides an incomplete summary of the underlying heterogeneity in meta-analysis. Prediction intervals show the range…

School leaders today are making important decisions regarding education innovations based on published average effect sizes, even though few understand exactly how effect sizes are calculated or what they mean. This article explains how average effect sizes are determined in meta-analyses and the importance of including measures of variability…

Cost, cost-effectiveness, and benefit-cost analysis are methods used by economists to evaluate public policies. Essentially, these methods rely on impact evaluations, that is, research studies of efficacy and effectiveness. However, in most research in education, these cost and impact evaluations are performed separately. This separation creates…

This report aims to introduce the fundamental features of the free Jamovi software to academics in the field of educational measurement for use at undergraduate and graduate level research. As such, after introducing the R based interface and the integrated development environment, the core functions of Jamovi are presented, the installation for…

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If…

This paper proposes the use of the tools of statistical meta-analysis as a method of conflict resolution with respect to experiments in cognitive linguistics. With the help of statistical meta-analysis, the effect size of similar experiments can be compared, a well-founded and robust synthesis of the experimental data can be achieved, and possible…

Network meta-analysis (NMA) is gaining popularity for comparing multiple treatments in a single analysis. Generalized linear mixed models provide a unifying framework for NMA, allow us to analyze datasets with dichotomous, continuous or count endpoints, and take into account multiarm trials, potential heterogeneity between trials and network…