The use of peer review for federal research impact evaluation is described. Advanced review processes can improve the efficiency of a review, but the most important factors in a quality review are leader motivation and the competence and independence of review team members. No single method provides complete impact evaluation. (SLD)

Proposes and evaluates a method to test for mediation in multilevel data sets formed when an intervention administered to groups is designed to produce change in individual mediator and outcome variables. Applies the method to the ATLAS intervention designed to decrease steroid use among high school football players. (SLD)

The conventional way to measure program impacts is to compute the average treatment effect; that is, the difference between a treatment group that received some intervention and a control group that did not. Recently, scholars have recognized that looking only at the average treatment effect may obscure impacts that accrue to subgroups. In an…

This study describes a method for incorporating external estimates of intraclass correlation to improve the precision for the analysis of an existing group-randomized trial. The authors use a random-effects meta-analytic approach to pool the information across studies, which takes into account any interstudy heterogeneity that may exist. This…

A beverage-server intervention project at two West Coast Navy bases that attempted to reduce levels of alcoholic intoxication via policy changes and server training is described. Data obtained via interviews and structured observations of 1,511 club customers indicate methodological bias and self-selection effects. Bias adjustments were performed…

In 1980, five federal special education longitudinal qualitative impact evaluations were all prematurely terminated. What happened, why it happened, and what salvage decisions were made are examined in this article. The impact of these decisions on the design of RFPs and future evaluation plans are also discussed. (Author/EGS)

The concept of "preevaluative research" is examined in the context of a museum exhibition evaluation. It is viewed as distinct from an evaluability assessment. The exhibit preevaluative study indicates that instrumentation and implementation issues are likely to benefit from such activities, but that design and analysis may suffer.…

Net impact estimates of Comprehensive Employment and Training Act (CETA) programs vary widely and can be explained by the different evaluation methodologies used. Estimates are sensitive to the inclusion of recently unemployed persons in the comparison sample and assumptions about the time of decision to enroll in CETA. (GDC)

Microcomputer simulations in evaluation research are useful for (1) improving student understanding of research principles and analytic techniques; (2) investigating problems arising in research implementations; and (3) exploring the accuracy and utility of novel analytic techniques. This article describes these simulation uses for the context of…

Value judgments are central to the process of stakeholder-based evaluations. The selection of stakeholder participants involves a value judgment about the power and the legitimacy of the stakeholders. Consequences of stakeholder evaluation may include pseudoempowerment. Suggestions for evaluators for improving stakeholder evaluations are made.…

A controlled study of the impact of a juvenile education program on the recidivism rates of juveniles was performed. The program involved introducing the juveniles to prison life. Findings showed that youths categorized as more delinquent were affected differently by program attendance compared to youths categorized as less delinquent. (Author/LMO)

This article focuses on the distinction between needs assessment and demand assessment and presents a methodology for operationalizing and measuring demands. Results are reported of a survey of a national sample of 32 university and college administrators to assess their need and demand for an adult student opinion package. (Author/LMO)

This article presents a synopsis of each of seven presentations given at a conference on design and analysis in community trial studies. Papers identify problems with community trials and discuss strengths and weaknesses associated with design and analysis strategies. Areas of consensus are summarized. (SLD)

The validity of true experiments is threatened by a class of self-report biases that affect all respondents at pretest, but which are diminished by treatment. Four of these inaccurate self-evaluation biases are discussed. Means of detection include external criteria, special conditions of measurement, and retrospective pretests. (TJH)

A method is described for increasing residual degrees of freedom in a community experiment without substantially increasing cost or difficulty by dividing experimental subunits into batches. Theoretical advantages of batch sampling are described and illustrated with data from the Pawtucket Heart Health Program. (SLD)