This response aims to critically evaluate the methodology and aims of the meta-analytic review written by Maggin and Johnson (2014). The present authors systematically provide responses for each of the original criticisms and highlight concerns regarding Maggin and Johnson's methodology, while objectively describing the current state of evidence…

Augustine and Hemenover (2013) were right to state that meta-analyses should be accurate and generalizable. However, we disagree that our meta-analysis of emotion regulation strategies (Webb, Miles, & Sheeran, 2012) fell short in these respects. Augustine and Hemenover's concerns appear to have accrued from misunderstandings of our inclusion…

In the past decade, the federal government, some states, and numerous communities have initiated programs to help couples form and sustain healthy marriages and relationships in order to increase family stability for children. Thus, the authors value the attention given to this emerging policy area by the "American Psychologist" in a recent…

This paper provides a summary of cost-benefit and cost-effectiveness analyses and their potential importance for decision making. The extent to which these tools are actually used, the possible connections between meta-analysis and cost-effective analysis, and the connection between effect sizes and benefit-cost ratios are discussed. (JAZ)

This article describes conceptual and methodological limitations of existing research on the efficacy of early intervention programs. Recent studies using meta-analytic techniques to generate testable hypotheses are reviewed. Recommendations for greater precision in measurement and for improved research design are offered. (DB)

The purpose of this paper is to review some of the methods used for the impact evaluation of Demonstrator-Developer projects and to promote the meta-analysis approach. Four examples of attempts to collect and analyze impact data from adopters are presented, one of which used the meta-analysis approach. In this example, each adopter collected pre…

Using examples from government program evaluation studies, six areas where research synthesis is more effective than individual studies are presented. Research synthesis can: (1) help match treatment type with recipient type; (2) explain which treatment features matter; (3) explain conflicting results; (4) determine critical outcomes; (5) assess…

Reviews synthesis as an important task of evaluation, putting evidence together for a coherent assessment. Types of synthesis are described from the case of two variables to more complex situations, ending with metanalysis. Syntheses help identify common patterns and regularities and can help progress toward positive social change. (SLD)

Examines the problems affecting evaluation of distance education in developing countries in terms of economic, political, and social issues. Discusses one solution, a meta-analysis for synthesizing the results of research studies. (JOW)

In program evaluation and policy analyses, utility supplants rigor as the first and most important criterion in judging literature reviews. Although R. T. Ogawa and B. Malen emphasize rigor, a utilization-focused evaluation emphasizes purpose and audience, both of which affect the conduct of a multivocal literature review. (SLD)

Conclusion about site-specific determinants of educational outcomes of the controversial longitudinal evaluation of the Follow Through programs (1976) are challenged again by reexamining the Direct Instruction model data. Reanalysis confirms earlier critiques and supports more recent research funding on the relationship between types of…

This paper analyzes the 19 studies presented to the National Institute of Education's (NIE) panel on the effects of school desegregation on black achievement and discusses the author's own findings. The author concludes that desegregation did not cause any decrease in black achievement generally, nor did it cause any increase in math achievement.…

The decision by the National Institute of Education panel on the effects of school desegregation to select (for meta-analysis) a small group of preferred studies based upon criteria chosen in advance of examining the studies was, in principle, a mistake. One usually cannot know until the data have been examined which of several competing…

This paper analyzes 19 studies that were considered by the National Institute of Education's panel on the effects of school desegregation in order to determine desegregation's effect on black achievement, black self-esteem, and racial relations. It is stated that, overall, the results show that reading achievement improves somewhat as a result of…

Although school desegregation was initiated to address a social inequity--segregated schooling was seen as stigmatizing blacks as a social group--research has focused primarily on desegregation's effects on black academic achievement and self-esteem. Two problems have made this research difficult: the ambiguity of the term "school desegregation"…