NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED663249
Record Type: Non-Journal
Publication Date: 2024-Sep-18
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Impact of STEM Mentored Undergraduate Research Program at a Community College: Results from an RCT
Laura Rosof; Julie Edmunds; Dora Gicheva
Society for Research on Educational Effectiveness
Background: Undergraduate research programs have been shown to have positive effects on student outcomes like retention and academic achievement. This study contributes to the very limited body of causal research on mentored undergraduate research programs (also known as apprentice-style research experiences, or AREs) by measuring the short- and long-term impacts of participation in a community-college based ARE using a randomized controlled trial. Only one study has used an RCT to study an ARE (Nagda et al., 1998); this was at a four-year university. There are no published RCT studies of AREs in a community college setting. Intervention and Setting: The START (STEM Academic Research and Training) program at Wake Technical Community College in Raleigh, North Carolina program provides the college's students with opportunities to do STEM projects through an extracurricular program. Participating students are assigned to a semester-long internship in which they conduct original research under the guidance of a faculty mentor. Students also complete training modules that cover STEM research skills, general career skills, and information about transferring to a four-year university, and they have opportunities to participate in START community events and share their research through a poster presentation. Students receive a stipend for their work. There are no academic pre-requisites for program eligibility and program recruitment focuses on students from populations historically underrepresented in STEM. Research Questions: START is intended to increase students' retention at Wake Tech, build their transfer capital, and ultimately increase the likelihood that they will transfer to and persist in STEM bachelor's programs. Three research questions guide our study of these and related impacts: What is the effect of participating in START... 1. On the percentage of students who remain enrolled in Wake Tech and/or transfer to a four-year institution? 2. On the percentage of students who major in or complete a terminal credential in a STEM field at Wake Tech? 3. On students' self-assessment of their scientific literacy, research skills, academic confidence, STEM identity, and STEM awareness? Research Design: The study is an RCT, capitalizing on the fact that more students applied to the program each semester than there were available spots. The research team randomly ordered the list of applicants and students were invited to participate in the order they appeared on the list; invitations were extended until all the program spots were filled. The study used an intent-to-treat approach, where all students who were offered a spot were considered part of the treatment group even if they did not participate; students who were not offered a spot were in the control group. 622 students in five semester cohorts participated in the RCT. Data Collection and Analysis: The study uses three data sources to answer the above questions: an original survey, student administrative data from Wake Tech, and National Student Clearinghouse (NSC) data. The NSC data provides information about students transferring to four-year institutions. The college's administrative data enables us to examine impacts on persistence (continued enrollment) and degree attainment at the college. In addition to these longer-term academic outcomes, we also investigate shorter-term outcomes through a survey of applicants which provides insight into some of the mechanisms potentially driving the longer-term outcomes. With validated items from Entering Research Learning Assessment (Butz & Branchaw, 2020), as well as original items that reflect the goals and context of START, the student survey is composed of seven scales that measure students' attitudes and skills related to STEM research and careers. Students take the survey pre- and posttreatment: when they apply to START and at the end of their internship semester. There was no attrition for outcomes measured using administrative data. There was high and differential attrition for the survey data; as a result, we assessed equivalence on baseline measures of the survey outcomes and all differences at baseline were an effect size of 0.06 or less. Differences in outcomes were assessed using multiple regression with an indicator for treatment and baseline student characteristics included as covariates in the model. Findings: The impact analysis using administrative data is preliminary at this time with data only available so far for the first two cohorts for the first semester after treatment. We used this data to look at a variety of academic outcomes (e.g., persistence, credits earned, GPA). Impacts on the treatment students were positive, but most were not statistically significant (see Table 1). However, we hypothesize that we will see statistically significant differences in treatment and control outcomes as we receive more data about more students and for longer timeframes after treatment. By the time of the conference, we expect to have received administrative data that will extend our analyses to one year after treatment for the first two cohorts and one semester after treatment for the first three cohorts. We have data from the Student Survey for the first three cohorts of students, which allows us to look at shorter-term outcomes. The Student Survey analysis shows significant positive effects (effect sizes of 0.21-0.44) on all but one of the scales measuring self-reported skills and attitudes related to STEM for students in the treatment group (see Table 2). They are significant at the 1 percent level for Comfort with STEM Research; at the 5 percent level for STEM Career Awareness, Scientific Literacy, and Diversity & Inclusion in STEM; and at the 10 percent level for Transfer Confidence and STEM Self-Efficacy. The impact estimate for Grit & Perseverance is also positive but not statistically significant. By the time of the conference, we expect to have survey data from the fourth cohort. Conclusions: The preliminary evidence suggests an overall positive effect on students' STEM academic careers. Our next steps for the study are to complete data collection and analysis for all five cohorts. Collection of administrative data will continue for two years after the fifth cohort's treatment semester in order to examine impacts two years posttreatment for all cohorts. We will also do subgroup analyses to understand the differences in impact by race/ethnicity, gender, and economic disadvantage status.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education; Two Year Colleges
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: North Carolina (Raleigh)
Grant or Contract Numbers: N/A