NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
ERIC Number: ED660098
Record Type: Non-Journal
Publication Date: 2024-Sep
Pages: 26
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Evaluation of the Impact of STEM-CS on High School Students' Mathematics Achievement in Rural South-Central Kentucky
Stephen Hiller; Jessica Campbell; Patricia Muller; Anne-Maree Ruddy
Grantee Submission
The Green River Regional Educational Cooperative (GRREC) and its partners administered the STEM-CS intervention to support the professional learning of middle and high school STEM teachers in rural southcentral Kentucky counties. The goals of the STEM-CS intervention were to improve teacher STEM-CS knowledge, practice, implementation, and effectiveness, while also improving opportunities, access, and outcomes for high-need rural students in STEM-CS fields. GRREC recruited rural, disadvantaged schools from among their member school districts to participate in the intervention funded by the Education Innovation and Research (EIR) grant. The evaluation was designed to provide the most rigorous possible study of the impact of the STEM-CS intervention given the parameters presented by program implementation. Unfortunately, the most rigorous possible study design was underpowered for detecting hypothesized effect sizes of interest. The pre-registration of the study indicated a minimal detectable effect size (MDES) of 0.50, meaning that even if STEM-CS had moderate to large effect on students' mathematics proficiency, the study was unlikely to detect these changes. Despite the poor sensitivity to even very large effect sizes, the study was conducted to meet evaluation requirements. This impact evaluation focuses on the high school portion of the intervention, using a cluster quasi-experimental design (QED), with non-random assignment to treatment at the teacher level to estimate the impact of the STEM-CS intervention in high schools on student academic proficiency in mathematics at grade 10. The six high schools participating in the STEM-CS intervention were propensity score matched with 12 rural high schools from GRREC-member school districts, with the final analytic sample of students in grade 10 in the 2021-2022 academic year consisting of 380 students in treatment high schools and 1,120 students in comparison high schools. Though school-level characteristics after propensity score matching showed inadequate baseline balance on gender and percentage of economically disadvantaged students (likely due to the purposive recruitment), final baseline equivalency on students' grade 7 academic proficiency in mathematics was satisfied, with additional student characteristics showing adequate baseline balance as well. Using a multilevel regression model with blocking to account for the propensity score matching, final results indicated that the STEM-CS intervention was not statistically significantly related to differences in grade 10 students' academic proficiency in mathematics. [This document was produced by the Center for Evaluation, Policy, & Research (CEPR).]
Publication Type: Reports - Research
Education Level: Secondary Education; High Schools; Grade 10; Elementary Education; Grade 7; Junior High Schools; Middle Schools
Audience: N/A
Language: English
Sponsor: Office of Elementary and Secondary Education (OESE) (ED), Education Innovation and Research (EIR)
Authoring Institution: N/A
Identifiers - Location: Kentucky
Grant or Contract Numbers: U411C190146