ERIC Number: ED567745
Record Type: Non-Journal
Publication Date: 2016
Pages: 7
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Updated Design Standards and Guidance from the What Works Clearinghouse: Regression Discontinuity Designs and Cluster Designs
Cole, Russell; Deke, John; Seftor, Neil
Society for Research on Educational Effectiveness
The What Works Clearinghouse (WWC) maintains design standards to identify rigorous, internally valid education research. As education researchers advance new methodologies, the WWC must revise its standards to include an assessment of the new designs. Recently, the WWC has revised standards for two emerging study designs: regression discontinuity designs (RDDs), and cluster designs where the clusters (e.g. schools) are of assignment, and data are collected from lower-level unites (e.g. students). Regression discontinuity designs (RDDs) are considered to be one of the strongest nonexperimental designs available (Shadish, Cook, & Campbell, 2002) for the purpose of identifying the effects of an intervention. These designs are applicable when a continuous "scoring" rule is used to assign the intervention to study units (for example, school districts, schools, or students). Units with scores below a pre-set cutoff value are assigned to the treatment group and units with scores above the cutoff value are assigned to the comparison group, or vice versa. In cluster designs, researchers may be interested in two different types of impacts: An intent-to-treat (ITT) effect and a place-based (PB) effect (Schochet 2013; Vuchinich et al. 2012). The ITT parameter pertains to students in the study clusters at the time the clusters were randomly assigned to research conditions. This population includes "stayers" who remained in the study clusters during the follow-up period and "leavers" who did not remain in the study clusters. The ITT parameter is the average difference in outcomes between stayers and leavers across research conditions. The WWC is revisiting the current guidance in the version 3.0 design standards for the review of cluster designs to address a number of limitations. In particular the revisions are intended to improve the review of cluster RCTs, which have been perceived as being too strict. The revised RDD standards expand and refine the pilot standards for RDD studies (June, 2010). The standards have been expanded to cover "fuzzy" RDDs (studies in which some treatment group members do not receive intervention services or some comparison group members receive services) and to cover RDDs that combine (through aggregation or pooling) multiple impacts (for example, from multiple sites or multiple assignment variables). The RDD standards have also been refined to reflect the evolution of the methodological literature (for example, the standards now favor studies that estimate impacts within a justified bandwidth around the cutoff value on the assignment variable). [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
Descriptors: Educational Research, Research Design, Regression (Statistics), Multivariate Analysis, Scores, Research Methodology, Standards, Randomized Controlled Trials, Statistical Analysis, Measurement, Guidelines
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A