NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED663420
Record Type: Non-Journal
Publication Date: 2024-Sep-19
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
The Effects of Automated Personalized Messages on Rural Student Attendance: Evidence from a Student-Level Randomized Control Trial
Sativa Thompson; Hayley Didriksen; Elise Swanson; Jennifer Ash
Society for Research on Educational Effectiveness
Background: Addressing chronic absenteeism is a policy priority, with 36 states and Washington D.C. referencing attendance in ESSA accountability plans (Jordan & Miller, 2017) and a body of research establishing the connections between attendance, achievement, and social development (e.g., Gottfried, 2014; Ginsburg, 2014). The need to find practices that increase attendance has become more urgent given rising absenteeism following the COVID-19 pandemic (Bay, 2023). Identifying effective, low-cost solutions to absenteeism is particularly critical for rural districts, which often face financial and staff capacity constraints (Hartman et al., 2023). Nudges can provide a low-effort, low-cost way of improving student outcomes by increasing the availability and accuracy of information (Damgaard & Nielsen, 2018). Prior work in urban settings demonstrated that texting parents could significantly reduce chronic absenteeism (Heppen et al., 2020). An initial pilot of a personalized messaging intervention similarly showed promising effects in rural settings, with students of caregivers assigned to receive messages accruing about 2% fewer absences than their non-treated peers (Swanson, 2023). This paper shares results from a national (46 rural districts in 18 states) randomized controlled trial (RCT) replication of this personalized messaging intervention. Specifically, we estimate the effect of families being assigned to receive messages containing information on students' cumulative days absent, providing a goal or contextual information about absences, and encouraging the caregiver to connect with the school. Purpose and Research Questions: The personalized messaging intervention was first piloted and evaluated through an RCT in the 2020-21 school year in 8 rural districts with over 8,000 K-12 students. We scale and replicate this intervention in a national RCT conducted over two years. This study addresses the following research questions: 1. Do students of caregivers assigned to receive personalized messages about their children's attendance accrue fewer absences compared to students of caregivers not assigned to receive messages? 2. Does the effect vary by district or student characteristics? Setting: This study takes place in 46 rural districts in 18 states with roughly 33,000 K-12 students. All geographic regions of the US are represented in our sample. All participating districts were using the student information system (SIS) and messaging platform prior to randomization. Population and Participants: Our population of interest is rural K-12 students. Year 1 (2022-23) included 12,203 students in 9 rural districts; year 2 (2023-24) included about 21,000 students in 37 rural districts. Intervention: Rural schools use a SIS with messaging (text, phone, and email) capabilities to send messages to treated families with students' cumulative absences since the last message, set an attendance goal or provide a reference point about average absences in the student's grade, and encourage the caregiver to connect with the school. Initial messages were sent in the fall, with subsequent messages sent every 4-6 weeks. Research Design: We estimate intent-to-treat (ITT) effects through a student-level RCT, with randomization based on randomly assigned, anonymous student IDs in the SIS. Data Collection and Analysis: We obtain demographic, outcome and select implementation data directly from the SIS. Student-level data include current and prior-year demographics and absences. District-level implementation data shows the date and number of messages sent. We use district surveys to collect information on the amount of time spent by district staff to set up and send the messages, as well as programmatic costs associated with developing the messaging materials and training district staff. We leverage a two-stage regression framework to estimate ITT effects. The first stage is a Poisson model in which students' days absent are regressed on treatment status with controls included for students' district, race/ethnicity, gender, special education services, and free- or reduced-price lunch (FRPL) receipt. The exposure term is the number of days enrolled in the district. Subgroup effects are estimated through interaction terms. Standard errors are clustered by student, the unit of randomization. First-stage coefficients are attenuated towards a prior mean of 0 using Bayesian shrinkage with a prior standard deviation of 0.05. We estimate per-pupil cost using the ingredients method, leveraging statewide data from the Bureau of Labor Statistics with a rural scalar. Results: 2022-23 results are shown below. Bayesian estimates suggest being assigned to have a caregiver receive messages led to a 1% reduction in absences, with a 70.1% chance that being assigned to the messages led to some reduction in absences. Perhaps unsurprisingly, this varies based on district implementation fidelity. We find a 2.3% reduction in absences among students in districts that sent three or more messages during the school year, compared to virtually no reduction among students in districts that sent fewer than three messages. The effect among students in districts meeting this minimum threshold of implementation fidelity was more than 80% certain. We find heterogeneous effects across student populations. Estimated effects were largest for students in high school (-2.7%) and smallest for students in middle school (-1%). We find suggestive evidence of differential impacts across demographic characteristics, including race/ethnicity (larger and more certain declines among White and Hispanic students), gender (larger and more certain declines among male students), family income (larger and more certain declines among students receiving FRPL), and receipt of special education services (larger and more certain decreases among students with disabilities). We also find larger and more certain effects for students chronically absent in the prior year (a 3.7% reduction with at least 90% certainty). Initial cost estimates suggest the program costs $3-$5 per student, including development and operating costs. Conclusions: In the first year, we do not find certain estimates of reduced absenteeism among K-12 students whose families were randomized to receive messages with personalized information about their attendance. Exploratory analyses suggest that effects were stronger (and certain) when minimum implementation fidelity benchmarks were met. Subgroup analyses also suggest that the intervention may have larger impacts among specific populations. We will present results from the full, two-year replication study and offer valuable insights into the process of developing and launching large, randomized trials in the rural context.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A