ERIC Number: ED656769
Record Type: Non-Journal
Publication Date: 2021-Sep-27
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
A Systematic Review and Gap Analysis of English Learner Reclassification Research
Molly Faulkner-Bond; Ilana Umansky; Emily Tanner-Smith; Tran Keys; Ryan Lewis; Lorna Porter
Society for Research on Educational Effectiveness
Background/Context: This paper reports the process, progress, and initial findings from a systematic review of research on English learner (EL) reclassification in the United States. EL classification is federally protected status in U.S. public schools that identifies students whose core rights must be met to support their educational access (Lau v. Nichols, 1974). EL status is intended to be temporary; students reclassify to fluent English proficient if and when, based on a set of criteria, they are considered sufficiently proficient in English to access educational opportunities without EL services and supports (Takanishi & Le Menestrel, 2017). As our project's theory of action illustrates (see Figure 1), reclassification functions both as an important outcome for students, and as an important educational input that results in a change of services (Robinson, 2011). Our goal is to advance the field's understanding of two critical topics around EL reclassification: (1) how long it takes a student to reach reclassification, and (2) what the effects of reclassification are on student learning outcomes. There is a growing body of research on both of these questions, with varying conclusions regarding both. With this variation comes an opportunity to, through a systematic review, document key features of research on reclassification and uncover important gaps that may exist. Purpose/Objective/Research Question: In this study, we describe our process, progress, and initial findings for identifying and extracting information from studies that have explored reclassification-related questions in the 16-year period since No Child Left Behind (2001) nationally positioned reclassification as a high-stakes policy event for EL students. We have three research questions: 1. How can research on reclassification best be identified, categorized, and coded to support a future meta-analysis? 2. What aspects of reclassification require a different approach to the systematic review and data extraction process than is typically employed for reviews of intervention research? 3. How can research on reclassification best be summarized to support future research and policymaking? What are the strengths and limitations of a systematic review and meta-analysis? Our aim with these questions is to share problems of practice and lessons learned based on our progress to date in identifying and screening articles from which we are currently extracting data to support various meta-regression analyses. To the extent possible, we will share initial descriptive information about characteristics of the eligible studies, as well as information about the information we are extracting for future analysis. Setting: To be eligible for inclusion, articles must report on quantitative data from real students who attended public school in the United States in the 2002-03 school year or later. Population/Participants/Subjects: To be eligible for inclusion, articles must report on data from real K-12 students who are currently, or have ever been, identified as English learners according to their setting's criteria. Intervention/Program/Practice: As noted above reclassification can function as both an outcome and input. As an outcome, reclassification reflects that a given student has reached English proficiency, a critical goal for English learner students (Kieffer, 2008; Nawyn, Gjokaj, Agbényiga, & Grace, 2012). In this framing, the intervention in question is the services offered to students classified as English learners -- which vary in their form and quality both within and across settings, and are thus one of several areas of interest for our gap analysis. When reclassification is framed as an input, it functions as an intervention in itself (Robinson, 2011) -- one that triggers changes in students' educational opportunities and experiences that can impact downstream educational outcomes such as academic achievement and attainment. Research Design: Both research questions are based on a systematic search and review of academic and grey literature for articles that focus on time to reclassification, effects thereof, or both. A team of raters used a structured protocol (see Figure 2) to screen the titles and abstracts of 14,935 articles returned from a search of the academic literature, ultimately identifying 322 potentially eligible articles. The team then used a second systematic protocol (see Figure 3) to double-screen the full-text of these 322 articles to identify 115 studies that were eligible for inclusion based on their focus on time to reclassification, effects of reclassification, or both. A follow-up search of grey literature sites -- focusing particularly on federal-, state- and local-education agency websites -- identified an additional 130 reports, of which 101 met eligibility criteria for inclusion under at least one RQ. Data Collection and Analysis: Articles in the final sample are independently coded using a detailed protocol that guides coders to extract data at the study-, group-, outcome-, and effect-size levels. For the systematic review, analysis will consist of descriptively summarizing the sample with respect to the extracted variables, to give a sense of the prevalence of the various characteristics that have been extracted and coded. For the gap analysis, we will synthesize the features of the included studies to make conclusions about areas where there is either limited, weak, or no evidence. Findings/Results: We have conducted the full systematic review and will be coding these studies in the coming months. Findings to date will focus on lessons learned from the systematic search and screening processes, as well as the process to design protocols for systematic data extraction from eligible articles. To the extent possible, we will also share preliminary descriptive information about the characteristics of eligible articles. Conclusions: Although our analyses are in progress, we have learned several lessons about how to design and carry out a systematic review around a policy such as a reclassification that serves a specific subpopulation and whose outcomes are measured in ways that do not always align to common approaches in intervention research. We will share these lessons and considerations, as well as any preliminary descriptive information we can about the characteristics of our final article sample.
Descriptors: English Language Learners, Outcomes of Education, Classification, Public Schools, Elementary Secondary Education, Language Proficiency, Language Fluency, Evaluation Criteria, Educational Opportunities, English (Second Language), Second Language Learning, Second Language Instruction, Kindergarten, Intervention, Educational Policy, Access to Education, Academic Support Services
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Information Analyses
Education Level: Elementary Secondary Education; Early Childhood Education; Elementary Education; Kindergarten; Primary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A