ERIC Number: ED663487
Record Type: Non-Journal
Publication Date: 2024-Sep-19
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Text4College: Impacts of a Text Message Intervention on Community College Enrollment and Persistence
Christina LiCalsi; Elisabeth Davis; Jill Bowdon; Alberto Guzman-Alvarez
Society for Research on Educational Effectiveness
Background/Context: The transition from high school to college marks a critical juncture for students planning to pursue higher education, particularly those from historically marginalized backgrounds. Unfortunately, many of these students are left to navigate complex financial aid requirements, registration processes, and academic placement with little or no support during the summer before college (Arnold et al., 2009). As a result, they often experience "summer melt," a phenomenon wherein up to 40% of these students fail to enroll in college in the Fall immediately following high school graduation, potentially exacerbating socioeconomic disparities in postsecondary access and attainment (Castleman & Page, 2014). In recent years, behavioral "nudging" through text messaging has emerged as a potential solution to this challenge, with Castleman and Page (2015) demonstrating its effectiveness in increasing college enrollment. In their randomized controlled study (RCT), Castleman and Page used a personalized text messaging campaign to remind students of key college-related tasks while offering students the option to seek follow-up support from a counselor via text message. This intervention led to a seven percentage point increase in college enrollment among students with limited access to college-going support (Castleman & Page, 2015). However, large-scale implementations of this kind of intervention have produced mixed results, with some studies reporting minimal to null impacts on college-related outcomes (Bird et al., 2021; Oreopoulos, 2020). Purpose: The present study aims to build on the nudge literature by replicating and scaling up Castleman and Page's (2015) study to students attending lower-resourced high schools in Alabama, Minnesota, Arizona, and Kentucky. This intervention is unique in leveraging partnerships with K-12 state education agencies, state higher-education agencies, and community college partners across four states. Moreover, we extend the nudge literature by examining long-term postsecondary impacts. Research Questions: (1) What is the impact of the Text4College intervention on students' community college enrollment and persistence?; (2) What is the impact of the Text4College intervention for different student subgroups, such as gender, race, economic status, first-generation status, and locale? Data: Our primary data source is student-level administrative records from Alabama, Minnesota, Arizona, and Kentucky, offering comprehensive insights into student demographics, academic performance, and postsecondary outcomes. Additionally, we leveraged platform data provided by Signal Vine, an AI-driven messaging platform tailored to support students in their postsecondary educational journey, to gain a detailed understanding of students' interactions with text messages. Subjects/Setting: This study includes two cohorts of students. Cohort 1 (n = 937) consists of graduating high school seniors from the class of 2020 in Alabama and Minnesota. In Alabama, we surveyed 1857 students, of which 803 met our eligibility criteria. In Minnesota, we surveyed 728 students, of which 134 met our eligibility criteria. As a result of complications caused by the COVID-19 pandemic, recruitment numbers from the spring of 2020 were lower than desired. The study team included a second cohort of students recruited during the 2020-21 school year to increase the sample size. Cohort 2 (n = 1,963) included graduated high school seniors from the class of 2021 in Alabama, Minnesota, Kentucky, and Arizona. In Alabama, we surveyed 2,083 students, of whom 844 met our eligibility criteria. In Minnesota, we surveyed 1,121 students, of whom 207 met our eligibility criteria. In Kentucky, we surveyed 2,517 students, of whom 503 met our eligibility criteria. Finally, we surveyed 992 students in Arizona, of which 409 met our eligibility criteria. Intervention: Students in the treatment group received between 12-17 tailored text messages over the summer. These messages, personalized for each student, served as reminders for critical college enrollment actions such as FAFSA submission. The messages were developed in collaboration with higher education partners to ensure relevance. This intervention was facilitated by Signal Vine, enabling two-way communication. Students could respond to messages seeking additional information or support, which was promptly provided by trained college staff. Research Design: Our study employed a blocked randomized control trial (RCT) design. At participating high schools, we administered a brief survey to seniors in spring 2020 and spring 2021 to determine student eligibility for the study. We randomized students within college blocks after establishing eligibility criteria (including consent, valid cell phone number, and intent to attend a partnering college). We initially intended to perform block randomization within high schools. However, we later randomized students within college blocks due to complications arising from recruiting students during the COVID-19 pandemic. Rerandomization was implemented to ensure covariate balance. The primary goal of our analysis was to estimate the intent-to-treat (ITT) effect of receiving a text messaging intervention compared to business as usual using linear probability models of the following form: Yi=[beta]0+[beta]1Ti+X'i[beta]StudCov+SchoolFE+ei. Where Yi is the outcome for student i (coded 1 = enrollment; 0 = did not enroll); Ti is an indicator for treatment status (coded 1 = assigned to text messaging intervention; 0 = not assigned to text messaging intervention); X'i is a vector of student baseline covariates that include gender, race/ethnicity, economic disadvantage status, first-generation college student status, and the extent of support received (to what); SchoolFE is a vector of dummy indicators for the college that the students enrolled in. ß1, our coefficient of interest, is the estimate of the percentage-point difference in college enrollment between those who were assigned to the intervention and those who were not. Findings: We observed a slightly higher rate of immediate enrollment in community college among students in the treatment group, about 1.7 percentage points higher than students in the control group. However, this effect was not statistically significant (Table 1). Notably, treatment students in Alabama enrolled in community colleges at higher rates than control students, with a difference of around 4.2 percentage points. However, this effect was only marginally significant. We found no significant effects in Arizona, Kentucky, or Minnesota. While not achieving statistical significance, our sub-group analysis showed encouraging positive trends among female, Black, and first-generation students (Table 2). This suggests that the Text4College intervention may benefit these historically underrepresented groups. We are currently analyzing whether the Text4College intervention impacted students' community college persistence and plan to present those results at the SREE conference.
Descriptors: Academic Persistence, Synchronous Communication, College Attendance, Comparative Analysis, Student Adjustment, Disadvantaged, Minority Group Students, Social Differences, Socioeconomic Status, Educational Attainment, Academic Advising, Intervention, Access to Information, High School Students, Partnerships in Education, State Agencies, Higher Education, Community Colleges, Gender Differences, Race, First Generation College Students, Student Attitudes, Student Financial Aid, College Applicants
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education; High Schools; Secondary Education; Two Year Colleges
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Alabama; Minnesota; Arizona; Kentucky
Grant or Contract Numbers: N/A