Most existing solutions to the current replication crisis in science address only the factors stemming from specific poor research practices. We introduce a novel mechanism that leverages the experts' predictive abilities to analyze the root causes of replication failures. It is backed by the principle that the most accurate predictor is the most…

Homelessness is a serious and growing issue. Evaluations of systemic-level changes are needed to determine progress in reducing or ending homelessness. The report card methodology is one means of systems-level assessment. Rather than solely establishing an enumeration, homelessness report cards can capture pertinent information about structural…

The determinants of sentencing are of much interest in criminal justice and legal research. Understanding the determinants of sentencing decisions is important for ensuring transparent, consistent, and justifiable sentencing practice that adheres to the goals of sentencing, such as the punishment, rehabilitation, deterrence, and incapacitation of…

This study explores the possibility that any drug prevention program might be considered "evidence-based" given the use of data analysis procedures that optimize the chance of producing statistically significant results by reanalyzing data from a Drug Abuse Resistance Education (DARE) program evaluation. The analysis produced a number of…

The methodology and design of the process evaluation system of the Pawtucket Heart Health Program (an 11-year project for changing/documenting change in cardiovascular disease risk status of a northeastern city's residents) are described. Examples of the system's use are provided. The maintenance of such a system without substantial evaluation…

Examined six published papers using a theory-based evaluation (TBE) approach. Found that authors of the papers did not always make explicit the relation of their data to the theory of the program. Identified some ways in which TBE contributed to the merit of the evaluation. Outlines lessons for the future of TBE. (SLD)

When procedures for developing recommendations are integrated into evaluation methods, it is easier to produce useful recommendations for program improvement. (RB)

Four aspects of the relationship between auditing and evaluation in their approaches to program assessment are examined: (1) their different origins; (2) the definitions and purposes of both, and the questions they seek to answer; (3) contrasting viewpoints and emphases of auditors and evaluators; and (4) commonalities of interest and potential…

Evaluation design decisions for a child care provider training program are discussed as a function of three characteristics: decentralization, local autonomy and multiple delivery strategies. While decentralized process data were gathered, only centralized outcome data were collected. The design decisions made were appropriate for such diverse…

An overview of the evaluation issues associated with contraceptive social marketing programs is provided. Methodologies covered include survey techniques, cost-effectiveness analyses, retail audits of sales data, time series analysis, nested logit analysis, and discriminant analysis. (TJH)

This article explores the statistical methodologies used in demonstration and effectiveness studies when the treatments are applied across multiple settings. The importance of evaluating and how to evaluate these types of studies are discussed. As an alternative to standard methodology, the authors of this article offer an empirical binomial…

Proposes group mean centering as a solution to the problem of pooling data from multiple sites. Uses California's Work Pays Demonstration Project to illustrate three analysis strategies for data from multiple sites. Discusses considerations in choosing the centering strategy. (SLD)

A mixed-method program evaluation, concurrently using both qualitative interview data and quantitative questionnaire data, was conducted. It illustrated significant design and analysis issues related to integrating quantitative and qualitative methods, specifically between-method and cross-paradigm triangulation. (Author/GDC)

Strategies are presented for analyzing longitudinal research designs with many waves of data using hierarchical linear modeling. The approach defines well-focused parameters that yield meaningful effect size estimates and significance tests. It is illustrated with data from the Boys Town Follow-Up Study. (SLD)

Data analysis approaches for program evaluation under the conditions of natural experiments are explored, using evaluation of the DARE program of drug use prevention among elementary school students as an example. The suggested methodology uses multiple independent analyses to demonstrate the convergence or divergence of program outcomes. (SLD)