The growing emphasis on using evidence to inform decisions in public sector policy making has created both opportunities for and demands on the American Evaluation Association (AEA) in pursuing its mission "to improve evaluation practices and methods, increase evaluation use, promote evaluation as a profession, and support the contribution of…

Drawing on a qualitative study of an educational reform and its external evaluation, I describe how a well-intentioned but poorly conceptualized evaluation helped perpetuate asymmetries in the generation and use of evaluation findings. This article explores this project's failure to clarify evaluator roles, identify intended users and expected…

As the field of evaluation continues to expand, so do evaluation course offerings. However, little is known about the backgrounds of evaluation educators, how they go about planning their courses, challenges experienced, or how they attain competency as teachers. This study offers an initial understanding of novice evaluation educators' experience…

Traditional methods of evaluation are limited in their ability to answer key questions often of interest to process evaluators, such as heterogeneity in the ways individuals adopt new programs. In this article, we demonstrate how a statistical approach, Latent Class Analysis, can help improve the quality of process evaluations and illustrate its…

This article discusses the nonuse, misuse, and proper use of pilot studies in experimental evaluation research. The authors first show that there is little theoretical, practical, or empirical guidance available to researchers who seek to incorporate pilot studies into experimental evaluation research designs. The authors then discuss how pilot…

Qualitative data offer advantages to evaluators, including rich information about stakeholders' perspectives and experiences. However, qualitative data analysis is labor-intensive and slow, conflicting with evaluators' needs to provide punctual feedback to their clients. In this method note, we contribute to the literature on rapid evaluation and…

Our understanding of programs is enhanced when trained, skilled, and observant evaluators go "into the field"--the real world where programs are conducted--paying attention to what's going on, systematically documenting what they see, and reporting what they learn. The article opens by presenting and illustrating twelve reasons for…

Within the context of intentional collaboration between doctoral-level program evaluation and counselor educators, this article addresses the need for advanced training in program evaluation for preservice doctoral-level counselor education (CE) students. To determine the current level of evaluation training provided to doctoral students attending…

In experimental evaluations of policy interventions, the so-called Bloom adjustment is commonly used to estimate the impact of the treatment on the treated. It does so by rescaling the estimated impact of the intention to treat--that is, the overall treatment-control group difference in outcomes for the entire experimental sample--by the…

This article reports findings from a research program exploring the role of mediation in an "adaptive learning" process through study of developmental evaluation (DE). Our study focuses on how mediators might influence the relationships between components of a social learning system and the implications for adaptive learning.…

Pressure on evaluators has been investigated recently by surveys in the USA, the UK, Germany, and Switzerland. This study compares the results of those studies regarding pressure on evaluators in different countries. The findings suggest that independence of evaluations does not exist for many respondents. Moreover, the person who commissioned the…

Despite political and rhetorical pronouncements of a reduction in racism, growing inequity in U.S. society continues to feature race as a prominent fault line with no evidence of reduction on the horizon. Of significant concern is the degree to which inequity among racially identified subgroups of the population link to policies and practices of…

Culturally and politically science is under attack. The core consequence of perceiving and asserting evaluation as science is that it enhances our credibility and effectiveness in supporting the importance of science in our world and brings us together with other scientists to make common cause in supporting and advocating for science. Other…

Traditional social sector and international development organizations have prioritized measuring the impacts of their work for decades. Understanding the ways in which they are bringing about change or helping people and communities has long been part of the traditional social sector ethos. The Global Impact Investing Network (GIIN) was founded by…

What is different about the conduct of evaluations in conflict zones compared to nonconflict zones--and how do these differences affect (if at all) the ethical calculations and behavior of evaluators? When are ethical issues too risky, or too uncertain, for evaluators to accept--or to continue--an evaluation? These are the core questions guiding…