ERIC Number: ED663432
Record Type: Non-Journal
Publication Date: 2024-Sep-21
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Evaluating the Effectiveness of Writing Interventions for Students in Grades K-5: A Comprehensive Meta-Analysis
Alyson Collins; Stephen Ciullo; Steve Graham; Joong won Lee
Society for Research on Educational Effectiveness
Background/Context: Many students in the United States need access to effective writing instruction, with some requiring intensive intervention. Results from the National Assessment of Education Progress (NAEP) in writing over the past 25 years revealed that most students in the U.S. have yet to attain a proficient level of written expression (e.g., National Center for Education Statistics, 2012; Persky et al., 2002). Moreover, subgroup analysis of NAEP outcomes suggests a critical need to improve writing outcomes historically marginalized populations, including English Learners, students from low-income school districts, and students with a disability (Mo & Troia, 2017). Extending beyond students' needs, classroom teachers have expressed challenges with accessing effective instructional methods for teaching writing, as general educators as well as special educators reported feeling under-prepared to teach writing (Graham et al., 2023). As such, a comprehensive meta-analysis on writing instruction in Grades K-5 is necessary for two reasons. First, researchers need to expand upon previous systematic reviews because most meta-analyses in writing have narrowly focused on a single writing outcome (e.g., writing quality; Graham et al., 2012) or limited findings to a specific student population (e.g., students with learning disabilities (Gillespie & Graham, 2014). Although evaluating specific outcomes of writing interventions and their impact on certain populations of learners may inform everyday educational practice, a comprehensive examination of the existing research in writing greatly expands our knowledge of the extent which instruction is effective on a range of skills (e.g., handwriting, spelling), cognitive factors (e.g., motivation), and types of writers. Second, a comprehensive meta-analysis provides comparisons and contrasts to overall effect sizes alongside deeper investigations of effects for subgroups of interventions, outcomes, and learners. Ultimately, this meta-analysis extends prior reviews writing intervention research in Grades K to 5 by evaluating current evidence to shape future directions for classroom instruction and research. Purpose/Objective/Research Question: The current meta-analysis involved a comprehensive and systematic review of the writing intervention literature for students in Grades K-5. The study aimed to analyze the extant writing research to understand which writing interventions are effective, for whom, and on which writing outcomes. By synthesizing this body of research, researchers can make informed recommendations to pre-service teacher education programs, professional development providers, educational policy makers, and writing intervention researchers. Our hypothesis was that writing interventions using experimental and/or quasi-experimental designs would result in greater performance for students in Grades K to 5 in the treatment conditions (with moderate effect sizes), including when writing intervention is delivered to specific populations of learners (e.g., students with disabilities). Two research questions were investigated: 1. What writing interventions are effective on what writing outcomes for students in Grades K to 5? 2. What writing interventions are effective for which learners (i.e., for whom)? Research Design & Data Analysis: This meta-analysis investigated the extent to which writing interventions are effective on writing outcomes (e.g., quality, spelling) for students in Grades K to 5. Researchers applied rigorous systematic search methods to locate studies including electronic database searches, reference harvesting, journal hand searches, inspection of previously conducted reviews, and an author contact effort. Inclusion criteria were: (a) participants in Grade K to 5, (b) the study investigated a writing intervention, (c) the study included at least one writing outcome measure, (d) studies used randomized control trials (RCT) or quasi-experimental designs with pretest. No restrictions were placed on date of the study, the country in which the study was implemented, or language of instruction; however, all studies were reported in English. Eligible studies were implemented in a typical school setting (e.g., hospitals and clinic settings were excluded). Approximately 20% of studies were independently screened and coded by a second researcher. Reliability for screening and coding exceeded 92% and 80%, respectively. For data analysis, researchers utilized Hedge's g (Hedges, 1981) standardized mean difference effect size (ES) to estimate the effects of writing interventions when compared to control conditions, with ESs adjusted when pretest scores were reported. R packages metafor and clubSandwich (Pustejovsky, 2022) were used to apply Robust Variance Estimation (RVE) procedures and to estimate the average ES for categories of writing interventions (e.g., strategy instruction, spelling interventions, computer-assisted instruction). Additionally, separate models were estimated for each writing outcome (e.g., holistic quality, spelling, writing output) and subgroup of learners, with a value of [rho] = 0.80 applied to estimate between-study variance (Hedges et al., 2010). Moderator analyses further explored if study characteristics including intervention duration, group size, or language of instruction explained heterogeneity in study ES. Results identified 362 eligible studies that included 3,727 ESs. Robust Variance Estimation (RVE) models were conducted. The full model (including all ES) produced moderate effects in favor of students receiving treatment (ES = 0.34). Subgroup analyses were also conducted. These additional RVE models revealed that writing interventions, on average, have large effects on outcomes of genre elements (ES = 0.89) and organizational processes (ES = 2.14). Conversely, on average, smaller effects were reported for writing mechanics outcomes (spelling ES = 0.41) and conventions (ES = 0.33). Another noteworthy findings was that simply providing students with time to write revealed a small ES that was not statistically significant. Further, large effects were reported across student populations for writing strategy instruction, including self-regulated strategy development (SRSD). In addition, computer-assisted interventions produced a small effect size (and not statistically significant), but the corpus of included technology-based studies was relatively small (n=12). Conclusions: Findings from this comprehensive meta-analysis has important implications for how educators use writing interventions to improve outcomes for different populations of learners. Discussion will consider implications for using different approaches to teaching writing, underscoring interventions with potential to have greater impact on elementary students. In addition, discussion consider areas needing further attention in pre- and in-service professional to influence on-the-ground teacher practice. Discussion will conclude with considerations for future research in writing.
Descriptors: Program Effectiveness, Writing Instruction, Elementary School Students, Intervention, Writing Skills, Meta Analysis, Effect Size, Writing Research, Research Methodology, At Risk Students, Students with Disabilities, English Language Learners, Low Income Students, Student Characteristics, Writing Strategies, Computer Assisted Instruction
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Information Analyses
Education Level: Elementary Education
Audience: Researchers
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Assessments and Surveys: National Assessment of Educational Progress
Grant or Contract Numbers: N/A