We investigated the use and validity of standardized achievement tests for summative evaluation of 78 educational intervention projects funded by the Institute of Education Sciences (IES) math and science education program. Investigators from 46 projects evaluated curricular interventions with standardized achievement tests as outcome measures.…

Evaluation occurs within a specific context and is influenced by the economic, political, historical, and social forces that shape that context. The culture of evaluation is thus very much embedded in the culture of accountability that currently prevails in public sector institutions, policies, and program. As such, our understanding of the…

Research on evaluation has mainly focused on the use of evaluation and has given little attention to the origins of evaluation demand. In this article, I consider the question of why parliamentarians demand evaluations with parliamentary requests. Building on the literature of delegation, I use a principal-agent framework to explain the origins of…

The "performance movement" has been a subject of enthusiasm and frustration for evaluators. Performance measurement, data analytics, and program evaluation have been treated as different tasks, and those addressing them speak their own languages in their own circles. We suggest that situating performance measurement and data analytics…

In a world faced with unprecedented rising levels of inequality and injustice, is there a responsibility for our evaluation organizations to take on a leadership role in promoting inclusive, evaluative dialog and deliberation about the state of our democracies in relation to key democratic principles and ideals? In this forum, I question whether…

Evaluators can work with brief units of text-based data, such as open-ended survey responses, text messages, and social media postings. Online crowdsourcing is a promising method for quantifying large amounts of text-based data by engaging hundreds of people to categorize the data. To further develop and test this method, individuals were…

The interest and effort in evaluation capacity building (ECB) continues to grow enormously among organizations, funders, and evaluators. Yet a research synthesis of the ECB literature (Labin, Duffy, Meyers, Wandersman, & Lesesne, 2012) revealed major limitations in the science and practice of ECB. This forum on ECB, which includes two past…

The 4-H youth development organization is a complex public--private partnership between the U.S. Department of Agriculture's National Institute of Food and Agriculture, the nation's Cooperative Extension system and National 4-H Council, a private, nonprofit partner. The current article is focused on a partnership approach to the…

Researcher--community partnerships are a necessary but not sufficient facet of effective research and evaluation with community-based projects and in clinical settings. This article describes two approaches that we have integrated into a multiyear, multiphase research and evaluation initiative supporting the health and well-being of vulnerable…

Although collaboration is recognized as an effective means to address multifaceted community issues, successful collaboration is difficult to achieve and failure is prevalent. To effectively collaborate, collaborators must recognize the strengths and weaknesses within their own efforts. Using Mattessich and colleagues' work as a springboard, a…

This article presents discussion and recommendations on approaches to retrospectively evaluating development interventions in the long term through a systems lens. It is based on experiences from the implementation of an 18-month study to investigate the impact of development interventions on economic and social change over a 40-year period in the…

Both speakers at the Eleanor Chelimsky Forum on Theory and Practice in Evaluation pointed out the complexity and messiness of evaluation practice, and thus potential limits on theory and generalizable knowledge. The concept of reflective practice offers one way forward to build evaluation theory. Building generalizable knowledge about practice…

To begin statistical analysis, Bayesians quantify their confidence in modeling hypotheses with priors. A prior describes the probability of a certain modeling hypothesis apart from the data. Bayesians should be able to defend their choice of prior to a skeptical audience. Collaboration between evaluators and stakeholders could make their choices…

There is a growing call for evaluation capacity building (ECB); although the area currently lacks a rich research base, there are few robust methods and practice through which to define it. The argument in this article is that the impact of evaluation is mediated by program stakeholders' engagement in evaluation activities. This mediation provides…

In this paper we share our reflections, as evaluators, on an evaluation where we encountered Excessive Evaluation Anxiety (XEA). The signs of XEA which we discerned were particularly evident amongst the program head and staff who were part of a new training program. We present our insights on the evaluation process and its difficulties, as well as…