Background/Context: Over the past twenty plus years, the What Works Clearinghouse (WWC) has reviewed over 1,700 studies, cataloging effect sizes for 189 interventions. Some 56% of these interventions include results from multiple, independent studies; on average, these include results of [approximately]3 studies, though some include as many as 32…

Background/Context: In meta-analyses examining educational interventions, researchers seek to understand the distribution of intervention impacts, in order to draw generalizations about what works, for whom, and under what conditions. One common way to examine equity implications in such reviews is through moderator analysis, which involves…

Background/Context: In meta-analysis examining educational intervention, characterizing heterogeneity and exploring the sources of variation in synthesized effects have become increasingly prominent areas of interest. When combining results from a collection of studies, statistical dependency among their effects size estimates will arise when a…

Background: A systematic review is a secondary research method that follows a predetermined protocol to identify, synthesize, and assess the quality of primary studies that meet pre-established eligibility criteria to answer a specific research question (Higgins & Green, 2011; Higgins & Thomas, 2019). It uses systematic and transparent…

Background/Context: The reckoning with racial injustice and growing inequality that have become hallmarks of the early 2020s in the United States has implications for impact analysis and the evidence it produced for public policy decision-making. Various researchers have highlighted the shortcomings that impact analyses have when estimating the…

Background: Strong classroom management skills are necessary for students' academic and behavioral success (e.g., Korpershoek et al., 2016), and group contingencies are an effective strategy teachers can use to help improve classroom behaviors (Little et al., 2015; Maggin et al., 2012). CW-FIT is one group contingency intervention that helps…

Background: Meta-analysis refers to the statistical methods employed to combine results of several empirical studies in a topic of interest (Hedges & Olkin, 1985). Meta-analysis is often included in literature review studies to quantitatively analyze data from a collection of studies (Valentine et al., 2010). The statistical power of a…

Background: Searching the literature and screening studies are two key stages to build a strong foundation for a quality systematic review. Low-quality handsearching and citation chasing processes may lead to missing relevant studies, while low-quality study screening may lead to human errors and a lengthy process (Haddaway et al., 2022; Zhang…

Context: A perennial problem in applied statistics is the inability to justify strong claims about cause-and-effect relationships without full knowledge of the mechanism determining selection into treatment. Few research designs other than the well-implemented random assignment study meet this requirement. Researchers have proposed partial…

Background: Systematic reviews and meta-analyses of interventions can provide critical evidence for educators and policymakers by providing insights into the effectiveness of related interventions. The quality of included studies builds the foundation for a high-quality review, while bias in the included studies increases the risks of unreliable…

Background: Despite the usefulness of systematic reviews and meta-analyses, they are time-consuming and labor-intensive (Michelson & Reuter, 2019). The technological advancements in recent years have led to the development of tools aimed at streamlining the processes of systematic reviews and meta-analyses. Innovations such as Paperfetcher…

Background: Meta-analysis is considered to be the gold standard for evidence synthesis. It involves combining data from multiple independent sources to produce a summary estimate with improved precision. Traditionally, meta-analysis methods have been applied to a large collection of studies, and past research efforts have indicated its numerous…

Introduction: The What Works Clearinghouse (WWC) reviews rigorous research on educational interventions with a goal of identifying "what works" and making that information accessible to educators and policymakers. The WWC has historically prioritized internal validity over external validity in rating the quality of research. One critique…

Why we need better reports of teacher professional development interventions for finding out what works where: Worldwide, billions of dollars are invested yearly in the professional development of in-service teachers ($14 billion in the USA alone, as of 2014; Bill & Melinda Gates Foundation, 2014). This makes the question of what effects TPD…

Background/Context: Methodological background: Meta-analysis typically depends on the assumption that true effects follow the normal distribution. While assuming normality of effect "estimates" is often supported by a central limit theorem, normality for the distribution of interventions' "true" effects is a computational…