Many teacher education researchers have expressed concerns about the lack of rigorous impact evaluations of teacher preparation practices. I summarize these various concerns as they relate to issues of internal validity, measurement, and external validity. I then assess the prevalence of these issues by reviewing 166 impact evaluations of teacher…

This article addresses the debate on pre-evaluative choices of impact depictions and the forms of responses between applicants and funders. By adopting a reflexive perspective on the social impact of social sciences, this article explores researchers' vocabularies in the research proposals and mid-term reports of consortiums during the Strategic…

The purpose of this article is to identify proper uses for mixed methods in leadership research and assessment. This article will highlight research and assessment questions that are best served by mixed methods and will offer practitioner-friendly guides and examples for integrating quantitative and qualitative data to help eat the whole whale…

The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

Although experimental evaluations have been labeled the "gold standard" of evidence for policy (U.S. Department of Education, 2003), evaluations without an analysis of costs are not sufficient for policymaking (Monk, 1995; Ross et al., 2007). Funding organizations now require cost-effectiveness data in most evaluations of effects. Yet,…

Computation is foundational to learning many mathematics concepts, as well as a functional skill in everyday life. Yet students with autism spectrum disorder (ASD) often have challenges in learning computation skills. The current study aimed to provide quantitative and descriptive analyses of single-case experimental studies on computation…

Teachers can use a variety of classroom management practices to help foster a classroom environment in which all students can learn. "Good Behavior Game" is a specific classroom management strategy that aims to improve social skills, minimize disruptive behaviors, and create a positive learning environment. Teachers place students into…

Teachers can use a variety of classroom management practices to help foster a classroom environment in which all students can learn. "Good Behavior Game" is a specific classroom management strategy that aims to improve social skills, minimize disruptive behaviors, and create a positive learning environment. Teachers place students into…

Higher education professionals often are tasked with providing evidence to stakeholders that programs, services, and practices implemented on their campuses contribute to student success. Furthermore, in the absence of a solid base of evidence related to effective practices, higher education researchers and practitioners are left questioning what…

A randomized experiment that measures the impact of a social policy in a sample of the population reveals whether the policy will work on average with universal application. An experiment that includes only the subset of the population that volunteers for the intervention generates narrower "proof-of-concept" evidence of whether the…

Lottery-based identification strategies offer potential for generating the next generation of evidence on U.S. early education programs. The authors' collaborative network of five research teams applying this design in early education settings and methods experts has identified six challenges that need to be carefully considered in this next…

Lottery-based identification strategies offer potential for generating the next generation of evidence on U.S. early education programs. The authors' collaborative network of five research teams applying this design in early education settings and methods experts has identified six challenges that need to be carefully considered in this next…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

Regression discontinuity (RD) designs are commonly used for program evaluation with continuous treatment assignment variables. But in practice, treatment assignment is frequently based on ordinal variables. In this study, we propose an RD design with an ordinal running variable to assess the effects of extended time accommodations (ETA) for…

The geoscience community has begun to grapple with the whiteness of the community and the harm and erasure of Indigenous communities done by earth and environmental scientists. We have come to understand that to recruit and retain Indigenous students, geoscience education needs to be culturally responsive by explicitly centering Indigenous…