A latent variable modeling-based procedure is discussed that permits to readily point and interval estimate the design effect index in multilevel settings using widely circulated software. The method provides useful information about the relationship of important parameter standard errors when accounting for clustering effects relative to…

Confirming multisite studies presents an ongoing challenge for researchers based in England. Those wishing to set up a multisite study must undergo local Research & Development review, in addition to ethical review by a Research Ethics Committee. However, both the documentation required for multisite review and the length of time it takes to…

This letter documents some problems in Ancaiani et al. (2015). Namely the evaluation of concordance, based on Cohen's kappa, reported by Ancaiani et al. was not computed on the whole random sample of 9,199 articles, but on a subset of 7,597 articles. The kappas relative to the whole random sample were in the range 0.07-0.15, indicating an…

In response to Eleanor Chelimsky's inspiring plenary address 3 years ago, and with generous support from the Robert Wood Johnson Foundation, the Eastern Evaluation Research Society launched the Eleanor Chelimsky Forum at its 2013 Annual Conference. The objective of this annual Forum, which has become a hallmark event in the evaluation world, is to…

William Dean Crano is the Stuart Oskamp Distinguished Professor of Psychology in the School of Behavioral and Organizational Sciences at Claremont Graduate University. A social psychologist, Crano received his MS and PhD degrees from Northwestern University, where Donald T. Campbell was his advisor and mentor. Dr. Crano has served on the faculties…

In this article, the author highlights a recurrent issue related to the historical measurement and use of attendance data: The challenge of producing uniform and reliable records on school attendance. Comparing this issue in the late 1800s to nearly two centuries later, he observes that while the capacity for record keeping and analysis has…

To help evaluation professionals better understand what an authentic attempt at Developmental Evaluation (DE) ought to look like and when it is appropriate to use, Michael Patton's provocative essay (this issue) offers evaluators eight sensitizing concepts to call on as guides. Patton states these concepts succinctly define DE and persuasively…

Insofar as the title of this piece might call for a straightforward answer, it seems obvious that EPMA professionals are servants. Viewed in this perspective, Paul E. Newton's analysis is carefully balanced, in that it respects the complex history of the concerns of the professionals, whilst moving towards conclusions that place the needs of the…

In this commentary I discuss how Getting to Outcomes and other empowerment strategies have potential to contribute to a theory of evaluation practice, address the epistemology of quality improvement, and inform the external validity of outcome evaluations.

In this commentary, Gary Henry explains that the Danes have got it right, again. Not only is their voter turnout high to begin with, but they are evaluating ways to increase voter turnout with "random variation" to accurately assess the effects of the innovations. In the Danish 2009 municipal elections, 65.8% turned out to vote, which…

The foundation upon which surgical residents are trained to work comprises more than just critical cognitive, clinical, and technical skill. In an environment where the synchronous application of expertise is vital to patient outcomes, the expectation for optimal functioning within a multidisciplinary team is extremely high. Studies have shown…

This chapter argues that the most important thing for a nonnative English-speaking teacher is to have an open and positive attitude toward students. A teacher's willingness to work out problems with individual students, coupled with openness to students' ideas and criticism, should help improve students' negative attitudes toward…

In this article, we critique two recent theoretical developments about collaborative inquiry in evaluation--using logic models as a means to understand theory, and efforts to compartmentalize versions of collaborative inquiry into discrete genres--as a basis for considering future direction for the field. We argue that collaborative inquiry in…

Haertel's argument is that one must "expand the scope of test validation to include indirect testing effects" because these effects are often the "rationale for the entire testing program." The author strongly agrees that this is essential. However, he maintains that Haertel's argument does not go far enough and that there are two additional…

Disagreements are an inevitable part of life, but what happens if a school issue becomes truly divisive? It's impossible to go through this world without coming into conflict with someone at some point, and music educators are no more immune to this than anyone else. However, there are details and procedures peculiar to the teaching world, and…