Research synthesis of evaluation findings is a multistep process. An investigator identifies a research question, acquires the relevant literature, codes findings from that literature, and analyzes the coded data to estimate the average treatment effect and its distribution in a population of interest. The process of estimating the average…

Background: Recent reviews suggest that many plausible programs are found to have at best small impacts not commensurate with their cost, and often have no detectable positive impacts at all. Even programs with initial rigorous impact evaluation (RIE) that show them to be effective often fail a second test with an expanded population or at…

Background: Many university departments use students' evaluations of teaching (SET) to compare and rank courses. However, absenteeism from class is often nonrandom and, therefore, SET for different courses might not be comparable. Objective: The present study aims to answer two questions. Are SET positively biased due to absenteeism? Do…

Regressions that control for confounding factors are the workhorse of evaluation research. When treatment effects are heterogeneous, however, the workhorse regression leads to estimated treatment effects that lack behavioral interpretations even when the selection on observables assumption holds. Regressions that use propensity scores as weights…

Complex survey data are collected by means other than simple random samples. This creates two analytical issues: nonindependence and unequal selection probability. Failing to address these issues results in underestimated standard errors and biased parameter estimates. Using data from the nationally representative Head Start Family and Child…

Growing interest in the institutionalization of evaluation in the public administration raises the question as to which institutional arrangement offers optimal conditions for the utilization of evaluations. "Institutional arrangement" denotes the formal organization of processes and competencies, together with procedural rules, that are…

Background: Injury is the leading cause of death for persons aged 1-44 years in the United States. Injuries have a substantial economic cost. For that reason, regional systems of trauma care in which the more acutely injured patients are transported to Level-I (L-I) trauma centers (TCs) has been widely advocated. However, the cost of TC care is…

The determinants of sentencing are of much interest in criminal justice and legal research. Understanding the determinants of sentencing decisions is important for ensuring transparent, consistent, and justifiable sentencing practice that adheres to the goals of sentencing, such as the punishment, rehabilitation, deterrence, and incapacitation of…

Many Medicaid programs have either fully or partially carved out mental health services. The evaluation of carve-out plans requires a case-mix model that accounts for differing health status across Medicaid managed care plans. This article develops a diagnosis-based case-mix adjustment system specific to Medicaid behavioral health care. Several…

This study is an exploratory examination of the projected sustainability of more than 100 projects funded by the Australian government. Using data collected by the body that evaluated the projects and data from a government database, it examines the predictors of various forms of sustainability. Findings show that some two thirds of the project…

A meta-program theory is proposed to overcome the limitations and improve the use of program theory as an approach to faith-based program evaluation. The essentials for understanding religious organizations, their various programs, and faith and spirituality are discussed to support a rationale for developing a faith-based program theory that…

After a brief discussion of the terms implementation and monitoring, this article illustrates a strategy for monitoring program implementation. The strategy, called model-guided method, offers a way to develop conceptual and empirical specifications for testing the adequacy of a program implementation. It also provides feedback to program…

The use of theoretical models in impact assessment can heighten the power of experimental designs and compensate for some deficiencies of quasi-experimental designs. Theoretical models of implementation processes are examined, arguing that these processes are a major obstacle to fully effective programs. (Author/CM)

Statistical identification of employees with unusual sick leave patterns can result in false-positive identifications. A computer simulation model was used to evaluate alternate methods of unusual sick leave detection for the New York State Department of Health. Results were used to refine identification parameters and modify supervisory…

Some questions about the relationship between program evaluation and forecasting are raised. Contrasting the two fields in terms of mind-set, purpose, problems, advantages, and use shows that although their modes of inquiry are indeed very different in many ways, there is interdependence between the two in several areas. (Author/JAZ)