Many teacher education researchers have expressed concerns about the lack of rigorous impact evaluations of teacher preparation practices. I summarize these various concerns as they relate to issues of internal validity, measurement, and external validity. I then assess the prevalence of these issues by reviewing 166 impact evaluations of teacher…

The use of singular methodologies has dominated emotional and behavioral disorder (EBD) intervention research, and by extension, the field has not deployed mixed-methods research (MMR). The purpose of this article is to provide an overview of MMR and demonstrate its utility for conducting EBD intervention research. The basic tenants and design…

Professional standards related to outcomes assessment call for student affairs professionals to use research to inform programming. If professionals are to rely on research to build programs that positively impact student learning outcomes, the research should be credible. We examined the quality of program effectiveness research available for…

Treatment acceptability is an aspect of social validity that refers to participants' beliefs and perceptions about the intervention, such as the helpfulness of the strategies or the interventions' efficacy to improve performance. The purpose of this study was to conduct a systematic review of treatment acceptability measures administered during…

Tools for evaluating the quality and rigor of single case research designs (SCD) are often used when conducting SCD syntheses. Preferred components include evaluations of design features related to the internal validity of SCD to obtain quality and/or rigor ratings. Three tools for evaluating the quality and rigor of SCD (Council for Exceptional…

This paper reviews literature on the rationale, challenges, and recommendations for choosing a nonequivalent comparison (NEC) group design when evaluating intervention effects. After reviewing frequently addressed threats to validity, the paper describes recommendations for strengthening the research design and how the recommendations were…

Although randomized studies have high internal validity, generalizability of the estimated causal effect from randomized clinical trials to real-world clinical or educational practice may be limited. We consider the implication of randomized assignment to treatment, as compared with choice of preferred treatment as it occurs in real-world…

Evaluations should be issues driven, not methods driven. The starting point should be priority programs to be evaluated or policies to be tested. From this starting point, a list of evaluation questions is identified. For each evaluation question, the task is to identify the best available method for answering that question. Hence it is likely…

The research synthesis presented here reviewed the currently available evidence on the effectiveness of sibling-implemented interventions for their siblings with disabilities. The search for appropriate research literature spanned from 1975 to 2008 and resulted in a total of eight studies that met the inclusion criteria. All eight studies in this…

Child maltreatment affects 900 thousand children in the U.S. every year and impacts all areas of daily functioning. Behavioral parent training (BPT) programs have effectively taught parenting & demonstrated externally valid outcomes (i.e., lower recidivism rates). Skill maintenance assessments for BPTs have mixed results. The Behavior Management…

"Theory as Method: Small Theories of Treatments," by Mark W. Lipsey, is one of the most influential and highly cited articles to appear in "New Directions for Evaluation." It articulated an approach in which methods for studying causation depend, in large part, on what is known about the theory underlying the program. Lipsey discussed the benefits…

This investigation examined the effects of currently available word prediction software programs that support phonetic/inventive spelling on the quality of journal writing by six students with severe writing and/or spelling difficulties in grades three through six during a month-long summer writing program. A changing conditions single-subject…

To date, reports of reading interventions for students at risk for emotional/behavioral disorders (E/BD) that have been published in refereed journals have involved sustained support by university or school-site personnel. This study examined the efficacy and feasibility of a reading intervention that 2 general education teachers implemented in…

The major theme is that better and more productive studies could emerge if media research would move out of its traditional either-internal- or-external validity conflict. (Author/MV)

Issues pertaining to the documentation of positive outcomes for severely handicapped learners as a function of their educational programs are addressed, and the value of the general evaluative concept of educational validity is emphasized. Issues related to the current overreliance on the single-subject experimental model are also discussed. (SEW)