Research synthesis of evaluation findings is a multistep process. An investigator identifies a research question, acquires the relevant literature, codes findings from that literature, and analyzes the coded data to estimate the average treatment effect and its distribution in a population of interest. The process of estimating the average…

Evaluations that combine social science and law have tremendous potential to illuminate the effects of governmental policies and yield insights into how effectively policy makers' efforts achieve their aims. This potential is infrequently achieved, however, because such interdisciplinary research contains often overlooked substantive and…

This article addresses the issue of falling response rates in telephone surveys. To better understand and maintain respondent goodwill, concepts of psychological contract and respondent expectations are introduced and explored. Results of the qualitative study show that respondent expectations are not only socially contingent but also…

Cross-cultural adaptation of study instruments is a difficult, time-consuming, but arguably cost-effective process. If conducted properly, it has the advantage that the translated study instruments are accurate, easy to understand, accessible, and culturally appropriate to the target audience and produce reliable and valid data. This article…

Risks of life on the street caused by inclement weather, harassment, and assault threaten the unsheltered homeless population. We address some challenges of enumerating the street homeless population by testing a novel capture-recapture (CR) estimation approach that models individuals' intermittent daytime visibility. We tested walking and…

Administered a survey to 606 National Science Foundation-funded principal investigators and their institutions' representatives (n=91) to study responses to unethical research behavior. Scientists appear to perceive that they uphold their responsibility to respond through disclosures in the research community; administrators propose to report such…

Active parental consent policies have been blamed for low participation rates and selection bias (i.e., loss of "high-risk" youths) in school-based studies. In this article, the authors describe active consent procedures that produced an overall active consent rate of 79% in a sample of more than 4,500 middle school students attending 29…

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions…

This article reviews the agricultural research evaluation literature in an effort to assess the extent of coverage in this area. Coverage breakdowns with respect to evaluation approaches, types of research, and products and production processes are presented. Results suggest that there has been limited coverage of research undertaken in…

The purposes of this article are, first, to argue that analyses based on level of treatment implementation can lead to biased estimates of treatment effects, and second, to discuss alternatives to this approach. (PN)

The normative implications of applying significance tests in evaluation research are examined. The authors conclude that evaluators often make normative decisions, based on the traditional .05 significance level in studies with small samples. Additional reporting of the magnitude of impact, the significance level, and the power of the test is…

Two studies of the cost-benefit analysis of correctional sentences are reviewed, which present conflicting results. Although such analysis has not determined a way to implement the selective incapacitation of criminals, it has identified the critical determinants of the decision to sentence criminals to prison or probation. (SLD)

Selectivity problems are discussed in terms of a general model that is estimated by the maximum likelihood method. Both single-group and multiple-group analyses are considered. An extension of the general model to latent variable models is discussed. (Author/PN)

Despite a growing literature on evaluation methods suitable for applied settings, few investigators address the separation of service and research. This article recognizes obstacles to field studies. The authors specify practical strategies for entering service environments and for designing, implementing, and measuring an experimental…

Correction for guessing does not fulfill its intended function when test takers who have nothing to gain from scoring will respond randomly when they could have answered correctly had they tried. Raw scores underestimate abilities. If random guessing is more prevalent in the control group, correction for guessing inflates treatment effects.…