Practical-participatory evaluation is often considered an effective approach to enhancing evaluation use. Despite this generally positive connotation, potential negative effects of participatory evaluation have also been discussed in the literature but never empirically investigated before. Drawing upon the social--psychological procedural justice…

This method note presents Q methodology as a useful tool for evaluators to add to their practice toolbox. Q methodology, which involves both quantitative and qualitative techniques, can help researchers and evaluators systematically understand subjectivity and the communicability of opinions and perspectives. We first provide an overview of Q…

Large-scale education interventions aimed at diminishing disparities and generating equitable learning outcomes are often complex, involving multiple components and intended impacts. Evaluating implementation of complex interventions is challenging because of the interactive and emergent nature of intervention components. Methods that build from…

Problem definition in program planning and evaluation is rarely problematized. In this article, I discuss why the lack of problem problematization is itself problematic--in other words, why treating problems as self-evident can pose a risk for evaluation practice. Then, to help avoid such risks, I suggest Carol Bacchi's "What's the Problem…

The second article in this series on the history of evaluation use has three sections. The first and longest develops a functional definition of the term "use," noting that a thorough definition of evaluation use includes the initial stimulus (i.e., evaluation findings or process), the user, the way people use the information, the aspect…

This article explores the utility of mindfulness in the field of evaluation. Mindfulness is a translation of the ancient Indian word, "Sati," which means awareness, attention, and remembering. While definitions vary, a practical definition of mindfulness is present-moment awareness in an open and nonjudgmental manner. Mindfulness-based…

This study was a formative evaluation of a retention program for nursing students from disadvantaged backgrounds underrepresented in the nursing workforce. The program was challenging, with multilevel interventions, some implemented sequential and some concurrently. The program team was interested to identify problematic components and address…

Despite enduring and unacceptable disparities in health outcomes for Aboriginal and Torres Strait Islander Australians as well as people living in rural and remote locations, evidence indicates that health services are not routinely evaluated. This article describes an exploration of a context where evaluators and community partners have achieved…

Social innovations (SIs) frequently bring previously unrelated actors, ideas, and practices together in new configurations with the goal of addressing social needs. However, the dizzying variety of definitions of SI and their dynamic, exploratory character raise dilemmas for evaluators tasked with their evaluations. This article is based on a…

We present this brief reflection on key aspects of the 2030 Agenda for Sustainable Development and the implications of the worldwide focus on achieving the sustainable development goals as external observers of the evaluation endeavor around the world. We have conducted and participated in evaluations, but it is not our primary field of work and…

Program participants have been largely excluded as an evidence source in realist evaluations. We test whether and how lived experience as described through life history interviews with pilot program participants can be used as a valid and unique source of data for elucidating context (C)--mechanism (M)--outcome (O) configurations and informing…

This article shares lessons learned in applying system evaluation theory (SET) to evaluate a Clinical and Translational Research Center (CTR) funded by the National Institutes of Health. After describing how CTR support cores are intended to work interdependently as a system, the case is made for SET as the best fit for evaluating this evaluand.…

Monitoring, evaluation, and research activities generate important data, but they often fail to change policies or programs. In addition, local program staff and partners often feel disconnected from these activities, which undermines their ownership of data and results. To bridge the gaps between monitoring, evaluation, and research and to give…

Evaluators have been slow to "plug" into the digital world, let alone engage with the strategic implications of the Big Data revolution for the evaluation discipline. Yet, Big Data holds enormous promise for extending the reach and improving the quality of evaluation practice. Equally, systematic recourse to evaluation would greatly…

Implementation fidelity is the degree of compliance with which the core elements of program or intervention practices are used as intended. The scientific literature reveals gaps in defining and assessing implementation fidelity in early intervention: lack of common definitions and conceptual framework as well as their lack of application. Through…