ERIC Number: ED628193
Record Type: Non-Journal
Publication Date: 2022
Pages: 12
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Heterogeneity of Treatment Effects of a Video Recommendation System for Algebra
Leite, Walter L.; Kuang, Huan; Shen, Zuchao; Chakraborty, Nilanjana; Michailidis, George; D'Mello, Sidney; Xing, Wanli
Grantee Submission, Paper presented at the Annual Meeting of the International Learning Analytics and Knowledge Conference (LAK2022) (Online, Mar 12-15, 2022)
Previous research has shown that providing video recommendations to students in virtual learning environments implemented at scale positively affects student achievement. However, it is also critical to evaluate whether the treatment effects are heterogeneous, and whether they depend on contextual variables such as disadvantaged student status and characteristics of the school settings. The current study extends the evaluation of a novel video recommendation system by performing an exploratory search for sources of heterogeneity of treatment effects. This study's design is a multi-site randomized controlled trial with an assignment at the student level across three large and diverse school districts in the southeast United States. The study occurred in Spring 2021, when some students were in regular classrooms and others in online classrooms. The results of the current study replicate positive effects found in a previous field experiment that occurred in Spring 2020, at the onset of the COVID-19 pandemic. Then, causal forests were used to investigate the heterogeneity of treatment effects. This study contributes to the literature on content sequencing systems and recommendation systems by showing how these systems can disproportionally benefit the groups of students who had higher levels of previous algebra ability, followed more recommendations, learned remotely, were Hispanic, and received free or reduced-price lunch, which has implications for the fairness of implementation of educational technology solutions.
Descriptors: Algebra, Teaching Methods, Mathematics Instruction, COVID-19, Pandemics, Video Technology, Instructional Effectiveness, Mathematics Achievement, School Districts, Comparative Analysis, In Person Learning, Online Courses, Causal Models, Prior Learning, Educational Benefits, Distance Education, Student Characteristics, Ethnicity, Lunch Programs, Ethics, Intelligent Tutoring Systems, Computer Simulation, Program Descriptions, Outcomes of Education
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
Identifiers - Location: Florida
IES Funded: Yes
Grant or Contract Numbers: R305C160004