ERIC Number: ED663425
Record Type: Non-Journal
Publication Date: 2024-Sep-19
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
The Evaluation of a State Education Agency's Grant Program to Support the Advancement of Emergency Licensed Educators as a Response to the COVID-19 Pandemic
Raifu Durodoye Jr.; Shannon Stackhouse; Ione Heigham; George Lolashvili
Society for Research on Educational Effectiveness
Background: The closures resulting from the COVID-19 pandemic significantly impacted teacher licensure pathways. During that time, prospective teachers had less access to student teaching experiences, fewer program and course completion opportunities, and a reduced ability to prepare for and pass state licensing examinations. In response, a State Education Agency (SEA) was specially authorized to provide emergency licenses to eligible educators as a stop-gap measure to address teacher vacancies. The SEA developed a grant program to bolster licensure pathway programs that supported emergency licensed educators pursuing a provisional or initial license. The grant was intended to fund program design and/or redesign efforts, wrap-around supports for candidates, and/or program participation incentives. Ten educator preparation programs were ultimately funded and began receiving awards in July 2022. Purpose: To evaluate the effectiveness of the "Grant Program," the SEA partnered with WestEd, a non-profit education policy research firm, to conduct a mixed-methods evaluation of the program. The purpose of the evaluation is threefold: (1) quantify the effectiveness of the "grant program," (2) document and describe implementation and implementation challenges, and (3) assess the viability of the program moving forward. Based on focal areas defined by the SEA, WestEd developed a set of core evaluation questions addressing five key domains: (1) Participant demographics and representativeness, (2) Retention and its correlation with participation, (3) Licensure advancement and its correlation with participation, (4) Experiences of participants, and (5) Program sustainability. Setting: Educators that received grant-funded support came from 56 school districts across the state. The evaluation team collected rosters from educator preparation programs located across the state. Population: The study's population of interest was emergency licensed educators in the state that received grant-funded support via the educator preparation program. The purpose of the evaluation was to assess whether grant funded programming aided educators in matriculating from an emergency license to an initial or provisional license. Therefore, the study team collected data, provided by grantees, on the licensed state educators that participated in the grant-funded activities or benefited from grant-funded supports. Research Design: The evaluation team used a mixed-methodological approach to address the research questions. To address Domain 1, participant demographics and representativeness, the team conducted a descriptive analysis of the state's administrative data. For Domain 2, retention and its correlation with participation, WestEd conducted a matching analysis (quasi-experimental designs (QED)). The QED used a robust set of teacher characteristics to identify a set of teachers that resemble program participants for the purposes of comparing outcomes across groups. For Domain 3, licensure advancement and its correlation with participation, WestEd used the same matching design to compare advancing from emergency to initial and/or provisional licenses as an outcome. To address Domain 4, experiences of participants, the qualitative analysis incorporated findings from a participant survey and two participant focus groups with emergency licensed educators who participated in and received supports from the grant funded programs ("participants"). For Domain 5, program sustainability, the qualitative analysis incorporated findings from one focus group with administrators of educator preparation programs that received the grant ("program administrators") and grantee program data. The cost study analysis used the financial reports and approved budgets from the grantees. Data Collection and Analysis: The SEA provided administrative data to support the quantitative analysis, including the state's licensure database and demographic and work placement database. To measure the association between program participation and retention, WestEd examined the probability of retention for program participants vs. compared to non-participants using logistic regression. The effect size was measured by the mean difference in the predicted probability of retention between grant program participants and non-participating educators. This same framework was applied to the research question examining the relationship between grant program participation and achievement of a license as an advancement after the emergency licensure program. The survey and participant focus groups collected data on the experience of program participants, including the perceived quality of programming. The administrators' focus group captured the experience of administrators and their perspectives on program implementation. Survey data were disaggregated and analyzed by demographic group and institution. Focus groups were open-coded to identify major themes across groups and where participants and administrators differed in their interpretations of the grant program. Findings: The analysis of survey data and focus groups identified three major challenges the program participants faced: 1) passing the subject matter exams; 2) the lack of time to complete the requirements and prepare for exams while working; and 3) financial costs of exams. Administrators' challenges included the short length of time the grant covered and difficulty accessing the funds. Administrators felt that emergency licensed educators needed support over a longer time period in order to really achieve the goals set out by the grant and the institutions themselves. Based on the preliminary quantitative analysis, the effect of being in the program was positive in case of both educator retention and educator licensure advancement: The odds of advancement of the license was 38% higher in the program participants than in the comparison group, and the odds of retention was 15% higher. The results for both outcomes were statistically significant at a 95% confidence interval. The preliminary results are based on the Spring 2023 school term data. WestEd is waiting on the Fall 2023 data transfer from the SEA in April after the latest database update to run the analysis on the most recent retention and licensure data and update the outcome models. Conclusions: The program evaluation results are instrumental for the SEA in strategizing the increase of program effect on educators. Investing resources in educator preparation programs can result in increase in teacher retention and matriculation from emergency to advanced licenses. The qualitative analysis shows that the SEA and the programs need to be cognizant of the implementation process in terms of allocation of time and financial resources and accessibility of supports the programs offer. Information gleaned from evaluation can be used by the state to increase the positive effect of the program on educators pursuing advancement of their licenses.
Descriptors: COVID-19, Pandemics, State Departments of Education, Grants, Emergency Programs, Alternative Teacher Certification, Teacher Education Programs, Program Effectiveness, Program Implementation, Program Evaluation, Barriers, Persistence, Probability, Administrator Attitudes, Teacher Attitudes
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A