NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED656925
Record Type: Non-Journal
Publication Date: 2021-Sep-28
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Initial Impact Findings from a Digital Social Emotional Learning Platform Implemented at Third Grade amidst the Pandemic
Kylie Flynn; Linlin Li; Melissa DeRosier
Society for Research on Educational Effectiveness
Background/Context: Poor social emotional learning (SEL) development places students at heightened risk for bullying, teasing, and social isolation (Asher, et al., 1996; Solberg, et al., 2007). As students experience social failure, school becomes an aversive place which they may actively try to avoid (Gazelle & Druhen, 2009; Ladd, 2006). Despite many schools' attempts to improve SEL and the proven effectiveness of SEL interventions, many students who could bene t don't receive the help they need because traditional in-person SEL has limited reach or entail too many logistical barriers (Agron, et al., 2010; Center for Mental Health in Schools at UCLA, 2014). Competing priorities, lack of available staff, inadequate tools to support ongoing implementation, insufficient training, and instructional drift over time can erode fidelity and sustainability (Agron et al., 2010). Technology-based interventions provide the opportunity to reduce these barriers to treatment and effectively deliver evidence-based SEL content with fidelity (Thomas, 2014). Technology- and game-based programs can be delivered at a much lower cost and on a flexible schedule as time allows, increasing opportunities for exposure to the program and maximizing the number of students who can benefit. However, for technology- and game-based SEL to be effective and result in observable improvement in real world settings, the instructional content must be built on a solid theoretical and research foundation. The "INTERVENTION" was specifically designed to translate the content and cognitive-behavioral strategies of an established, evidence-based SEL program into a game-based environment (DEVELOPER, 2004). Purpose/Objective/Research Question: The purpose of this study (funded by the U.S. Department of Education's Institute of Education Sciences) is to determine the effectiveness of "INTERVENTION" in the school setting for third grade students by answering our primary research question--Is the "INTERVENTION" program treatment more effective than the control for improving students' social-emotional skills? Setting: This study took place in 37 third grade classrooms across three school districts in Santa Clara and San Bernardino Counties in California. Population/Participants/Subjects: The original study sample consisted of 823 third-grade students with 543 (242 treatment students vs. 301 control students) completing at least one direct SEL outcome measure. Teachers provided SEL ratings for a subsample of 480 students (228 treatment students vs. 252 control students). More than 50% of the final analytic sample were qualified for Free/Reduced-Price Lunch program and 46.8% are Latinx. There were no statistical differences between treatment and control on their ethnicity, Free/Reduced-Price Lunch status, English language learner status, gender, or baseline performance. Intervention/Program/Practice: The "INTERVENTION" is intended to be implemented for nine weeks in typical third grade classes. The nine instructional episodes are set within an appealing story narrative: the player is a recruit on the sailing ship who joins the crew and travels around an island, interacting with a host of characters and engaging in social problem solving to address plot conflicts and save friendship on the island. The social problems encountered in the game are true-to-life (e.g., entering group social situations, staying calm in emotionally charged situations, expressing emotions positively, cooperating and compromising with peers) to increase ease of skill transfer to real life. The "INTERVENTION" is delivered once per week for 30-45 minutes each. The episodes focus on using respect, social planning, responsibility, communication, assumptions, perspective taking, building friendships, cooperation, and emotion regulation. Most treatment students were on episode 7 when schools closed during the pandemic. Teachers encouraged students to finish the remaining episodes remotely. Research Design: This study utilized a randomized controlled trial where classrooms were randomly assigned to one of two conditions--a treatment condition using the "INTERVENTION" program or a control condition that uses the school's business-as-usual program. Data Collection and Analysis: The "Behavioral and Emotional Rating Scale Second Edition" (Epstein, 2004) and "Social Skills and Behavioral Index" (DeRosier, 2011) are teacher rating scales that were collected from teachers on a subsample of students utilizing Qualtrics, an online survey platform. SELweb (McKown, et al., 2016) and ZooU (DeRosier, et al., 2012) are online performance-based assessments of social-emotional skills. For pretesting, teachers provided links to the SELweb and ZooU platforms and students navigated through the assessments independently in the classroom. For post-testing, teachers provided links via virtual learning environments (e.g., Google classroom) and students completed assessments independently at home with teacher support and/or supervision. To address whether the "INTERVENTION" program treatment is more effective than the control for improving students' social-emotional skills, researchers used a set of fixed block effects and fixed treatment effect models, where the dependent variable in each model is a separate SEL outcome. In each model, the effect of treatment on an outcome measure was estimated at the classroom level, controlling for student pretests, baseline student covariates (i.e., gender, ethnicity, and eligibility for Free and/or Reduced-Price lunch programs). The stepwise multiple hypothesis testing procedure was applied to account for conducting multiple comparisons (Benjamini & Hochberg, 1995). Findings/Results: Even though only 68.5% of treatment students completed the entire game, the "INTERVENTION" program was significantly and positively associated with gains in students' social-emotional skills. The effect sizes of the treatment impact as measured by different outcome measures ranged from 0.29 to 0.63 (based on Hedges' g), which is medium to large for educational interventions for a sample of this size (Kraft, 2020). Conclusions: This study is important for advancing the research on the impact of COVID-19 on students for four reasons. First, it addresses SEL, which is even more critical given the increased stressors on students related to the pandemic. Second, the "INTERVENTION" allows teachers and students the flexibility to engage in SEL activities remotely, making it a feasible option during ongoing distance learning. Third, even with the transition to remote implementation, the findings indicate significant and meaningful impacts on student outcomes. Fourth, despite the encouraging results, more than 20% of the overall sample were unable to complete the project activities due to a lack of devices, internet, or in-person supervision by their teacher, raising an important equity issue related to remote learning.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education; Early Childhood Education; Grade 3; Primary Education
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
IES Funded: Yes
Grant or Contract Numbers: N/A