Publication bias is a well-known threat to the validity of meta-analyses and, more broadly, the reproducibility of scientific findings. When policies and recommendations are predicated on an incomplete evidence base, it undermines the goals of evidence-based decision-making. Great strides have been made in the last 50 years to understand and…

Recently, Picho-Kiroga (2021) published a meta-analysis on the effect of stereotype threat on females. Their conclusion was that the average effect size for stereotype threat studies was d = .28, but that effects are overstated because the majority of studies on stereotype threat in females include methodological characteristics that inflate the…

Publication bias threatens meta-analysis validity. It is often assessed via the funnel plot; an asymmetric plot implies small-study effects, and publication bias is one cause of the asymmetry. Egger's regression test is a widely used tool to quantitatively assess such asymmetry. It examines the association between the observed effect sizes and…

A decade after the publication of Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement, the influence of John Hattie's statistical work continues to grow. This paper critically examines one aspect of Hattie's 'synthesis' of education meta-analyses, his summation of research data for whole language literacy teaching. Of…

Educational materials focused on spatial data analysis often feature mathematical descriptions of methods and step-by-step instructions of software tools, but infrequently discuss the set of decisions involved in specifying a statistical model. Failing to consider model specification may lead to specification searching, or the process of repeating…

"Shifting Sands: Unsound Science and Unsafe Regulation" examines how irreproducible science affects select areas of government policy and regulation governed by different federal agencies. This first report on "PM[subscript 2.5] Regulation" focuses on irreproducible research in the field of environmental epidemiology, which…

Purpose: To examine the effects on anxiety of mindfulness-based interventions (MBIs) among youth with anxiety disorders. Method: Systematic review and meta-analytic procedures were employed to synthesize experimental and quasi-experimental studies authored between 1980 and 2015. Results: The search yielded five studies from five countries…

Background: Stratified medicine seeks to identify patients most likely to respond to treatment. Individual participant data (IPD) network meta-analysis (NMA) models have greater power than individual trials to identify treatment-covariate interactions (TCIs). Treatment-covariate interactions contain "within" and "across" trial…

Research syntheses are increasingly being conducted within the fields of ecology and environmental management. Information retrieval is crucial in any synthesis in identifying data for inclusion whilst potentially reducing biases in the dataset gathered, yet the nature of ecological information provides several challenges when compared with…

Small-study effects are a common threat in systematic reviews and may indicate publication bias. Their existence is often verified by visual inspection of the funnel plot. Formal tests to assess the presence of funnel plot asymmetry typically estimate the association between the reported effect size and their standard error, the total sample size,…

This article has two interrelated purposes. The first is to explain how various forms of bias, if introduced during any stage of a meta-analysis, can provide the consumer with a misimpression of the state of a research literature. Five of the most important bias-producing aspects of a meta-analysis are presented and discussed. Second, armed with…

This meta-analysis provides an extensive and organized summary of intervention studies in education that are grounded in motivation theory. We identified 74 published and unpublished papers that experimentally manipulated an independent variable and measured an authentic educational outcome within an ecologically valid educational context. Our…

This study was conducted to evaluate alternative analysis strategies for the meta-analysis method of reliability generalization when the reliability estimates are not statistically independent. Five approaches to dealing with the violation of independence were implemented: ignoring the violation and treating each observation as independent,…

A large number of studies have examined the finding that recognition memory for faces of one's own age group is often superior to memory for faces of another age group. We examined this "own-age bias" (OAB) in the meta-analyses reported. These data showed that hits were reliably greater for same-age relative to other-age faces (g = 0.23) and that…

This article reviews the research literature and theory concerned with accuracy of judgments in romantic relationships. We initially propose a model of cognition in (romantic) relationships that distinguishes between 2 forms of accuracy: mean-level bias and tracking accuracy. We then report the results of meta-analyses of research on heterosexual,…