ERIC Number: EJ1323642
Record Type: Journal
Publication Date: 2022-Jan
Pages: 21
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1759-2879
EISSN: N/A
Retrospective Median Power, False Positive Meta-Analysis and Large-Scale Replication
Stanley, T. D.; Doucouliagos, Hristos; Ioannidis, John P. A.
Research Synthesis Methods, v13 n1 p88-108 Jan 2022
Recent, high-profile, large-scale, preregistered failures to replicate uncover that many highly-regarded experiments are "false positives"; that is, statistically significant results of underlying null effects. Large surveys of research reveal that statistical power is often low and inadequate. When the research record includes selective reporting, publication bias and/or questionable research practices, conventional meta-analyses are also likely to be falsely positive. At the core of research credibility lies the relation of statistical power to the rate of false positives. This study finds that high (>50%-60%) median retrospective power (MRP) is associated with credible meta-analysis and large-scale, preregistered, multi-lab "successful" replications; that is, with replications that corroborate the effect in question. When median retrospective power is low (<50%), positive meta-analysis findings should be interpreted with great caution or discounted altogether.
Descriptors: Meta Analysis, Replication (Evaluation), Statistical Analysis, Research Problems, Credibility
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://bibliotheek.ehb.be:2191/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Data File: URL: https://osf.io/x7cf6/