In Bayesian random-effects meta-analysis, the use of weakly informative prior distributions is of particular benefit in cases where only a few studies are included, a situation often encountered in health technology assessment (HTA). Suggestions for empirical prior distributions are available in the literature but it is unknown whether these are…

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If…

Varying methods for evaluating the outcomes of single case research designs (SCD) are currently used in reviews and meta-analyses of interventions. Quantitative effect size measures are often presented alongside visual analysis conclusions. Six measures across two classes--overlap measures (percentage non-overlapping data, improvement rate…

Research synthesis of evaluation findings is a multistep process. An investigator identifies a research question, acquires the relevant literature, codes findings from that literature, and analyzes the coded data to estimate the average treatment effect and its distribution in a population of interest. The process of estimating the average…

Conducted a meta-analysis of methodologies used to evaluate after school programs and attempted a synthesis of evaluation findings. Results show that evaluation reports demonstrate moderate compliance with the Program Evaluation Standards of the Joint Committee on Standards for Educational Evaluation but evidence did not allow for meta-analysis of…

One of the major series of reviews in elementary and secondary education is the Best Evidence Encyclopedia, or the BEE. Up to now, findings for systematic reviews have largely been restricted to the reviews themselves, with few cases in which lessons learned across many reviews using similar methods can be synthesized. The completion of the Best…

There are numerous micro-level methods decisions associated with planning an impact evaluation. Quantitative synthesis methods can be used to construct an actuarial data base for establishing the likelihood of achieving desired sample sizes, statistical power, and measurement characteristics. (Author/BS)

The manner in which results and methods are reported influences the ability of the synthesis of prior studies for planning new evaluations. Confidence ratings, coding conventions, and supplemental evidence can partially overcome the difficulties. Planners must acknowledge the influence of their own judgement in using prior research. (Author)

Some guidelines are offered for interpreting and evaluating meta-analytic reviews of research. The choice of unit of analysis, the issue of fixed versus random effects, the meaning of heterogeneity, the determination of appropriate contrasts, and the choice of measures of central tendency are discussed. (SLD)

The six articles of this issue discuss the place of meta-analysis in the health professions and present two state-of-the-art meta-analyses using a variety of innovative methodologies, statistics, and secondary analytic procedures. A survey conducted for this issue and its sequel explores interventions to facilitate meta-analysis. (SLD)

This digest addresses the role of evaluation and its basic logic, and provides a description of how the field is structured. In order to reach evaluative conclusions it is usually necessary to establish or identify factual promises and value premises. To obtain the required kind of overall evaluative conclusion, it is typically necessary to…

Because research synthesis enables one to determine either the overall effectiveness of a particular treatment or the relative effectiveness of different types of treatments, it is becoming increasingly popular as a tool in program evaluation. Numerous methodological problems arise, however, when research synthesis is applied to studies conducted…

National evaluations of quality aspects of vocational training programs (VTPs) in five European countries--Belgium, Germany, France, Greece, and Portugal--were reviewed to determine how VTP quality is currently being evaluated and identify a more coherent approach to evaluating VTPs. Special attention was paid to how existing evaluation studies…