This case study is based on a program evaluation of a professional certificate program that trains early childhood care and education providers to build and support young children's emotional literacy. The research project described in the case study will address approaches to methodological combination, justifying research design and changes to…

Our understanding of programs is enhanced when trained, skilled, and observant evaluators go "into the field"--the real world where programs are conducted--paying attention to what's going on, systematically documenting what they see, and reporting what they learn. The article opens by presenting and illustrating twelve reasons for…

In an effort to bring rigorous research into the education and social sciences field, the past decade had seen an increase in the advocacy of randomized controlled trials. The argument in favor of using randomized controlled trial is that one can evaluate the impact of an intervention in comparison to a control group with confidence during a…

The purpose of this paper is to provide a basis for judging the adequacy of evaluation plans or, as they are commonly called, evaluation designs. The authors assume that using the procedures suggested in this paper to determine the adequacy of evaluation designs in advance of actually conducting evaluations will lead to better evaluation designs,…

The youth mentoring program Big Brothers Big Sisters is one of the first social interventions involving youth in Ireland to be evaluated using a randomized controlled trial methodology. This article sets out the design process undertaken, describing how the research team came to adopt a concurrent embedded mixed methods design as a means of…

This final chapter summarizes the areas of consensus in the debate on method choice, including considering the nature of the primary evaluation questions, the nature of the phenomenon being evaluated, the constraints on the evaluation, and ethical issues. Pragmatic suggestions based on these areas as well as areas still in contention are offered.…

The design and conduct of a national evaluation study is discussed, demonstrating that a control group may not provide the no-cause baseline information expected. Resolution of this problem requires reexamination of paradigms, research practices, and policies, as well as the underlying real world constraints and views that generate them. (PN)

Is it ethical to select clients at random for a beneficial social service, then deny the benefits to a control group for the sake of science? Participation of control groups in planning, implementation and evaluation of social programs may resolve ethical issues. (Author/CP)

Seven qualitative issues associated with randomization that have the potential to weaken or destroy otherwise sound experimental designs are reviewed and illustrated via actual field experiments. Issue areas include ethics and legality, liability risks, manipulation of randomized outcomes, hidden bias, design intrusiveness, case flow, and…