Social return on investment (SROI), an evaluation method that compares monetized social value generated to costs invested, is in ascendance. Conceptually akin to cost-benefit analysis, it shares some of its challenges; however, these are heightened due to the expressed promise of using SROI to compare programs and inform philanthropic and public…

In this study, we investigated the impact of the evaluation capacity building (ECB) efforts of an organization by examining the evaluation plans included in funding proposals over a 14-year period. Specifically, we sought to determine the degree to which and how evaluation plans in proposals to one National Science Foundation (NSF) program changed…

Narrative case studies have shown that, when people are involved in an evaluation of a program they are part of, it can change how they experience the program. This study used a quasi-experiment to test this proposition empirically in the context a participatory action research curriculum called Youth as Researchers. Half of all Youth as…

Program evaluations that lack experimental design often fail to produce evidence of impact because there is no available control group. Theory-based evaluations can generate evidence of a program's causal effects if evaluators collect evidence along the theorized causal chain and identify possible competing causes. However, few methods are…

Photovoice is an empowering, qualitative, community-based participatory research method that engages participants to take photographs and reflect on their meanings. We employed a novel application of a Photovoice-inspired methodology to evaluate perinatal home visiting services. This article describes the feasibility and utility of remote…

Power has always been an element of the program evaluation field, however, it has traditionally been attributed a negative connotation. With that in mind, power is seen as a force possessed to some extent by different stakeholders that can be imposed over others, and by which certain views, ideas, and decisions can be adopted. Given that program…

In this article, we discuss the methodological implications of data and theory integration for Theory-Based Evaluation (TBE). TBE is a family of approaches to program evaluation that use program theories as instruments to answer questions about whether, how, and why a program works. Some of the groundwork about TBE has expressed the idea that a…

This study aimed to develop and validate the Evaluator Competencies Assessment Tool (ECAT) Cultural Competencies Subscale, which measures cultural competence among evaluators. By addressing the scarcity of validated tools in this area, the study offers professionals a valuable resource to assess their strengths and areas for improvement. The…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

The purpose of this study was to determine professional evaluators' perceptions of reflective practice (RP) and the extent and manner in which they engage in RP behaviors. Nineteen evaluators with 10 or more years of experience in the evaluation field were interviewed to explore our understanding and practice of RP in evaluation. Findings suggest…

Empowerment is a concept that has become increasingly used over recent years. However, little research has been undertaken into how empowerment can be evaluated, particularly in the case of young people. The aim of this article is to present an inventory of dimensions and indicators of youth empowerment. The article describes the various phases in…

Although experts agree that diverse stakeholder inclusion enhances quality and equity in evaluation designs and implementation, diverse voices are often omitted. Particularly antithetical to principles of youth character development, evaluations for these programs should strive to include voices from various social, economic, community, and…

A method called multi-attribute utility analysis (MAUA) provides a decision-making framework that facilitates comparative analysis of multiple real-world decision alternatives with unique complex attributes. Utility analysis as a measure of effectiveness has been minimally used by educational researchers to date, despite clear relevance in complex…

Site visits are common in evaluation plans but there is a dearth of guidance about how to conduct them. This paper revisits site visit standards published by Michael Patton in 2017 and proposes a framework for evaluative site visits. We retrospectively examined documents from a series of site visits for examples of Patton's standards. Through this…

Psychological theory suggests that evaluators' individual values and traits play a fundamental role in evaluation practice, though few empirical studies have explored those constructs in evaluators. This paper describes an empirical study on evaluators' individual, work, and political values, as well as their personality traits to predict…