ERIC Number: EJ1309772
Record Type: Journal
Publication Date: 2021-Sep
Pages: 19
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1560-4292
EISSN: N/A
Assessing the Effectiveness of Student Advice Recommender Agent (SARA): The Case of Automated Personalized Feedback
Mousavi, Amin; Schmidt, Matthew; Squires, Vicki; Wilson, Ken
International Journal of Artificial Intelligence in Education, v31 n3 p603-621 Sep 2021
Greer and Mark's (2016) paper suggested and reviewed different methods for evaluating the effectiveness of intelligent tutoring systems such as Propensity score matching. The current study aimed at assessing the effectiveness of automated personalized feedback intervention implemented via the Student Advice Recommender Agent (SARA) in a first-year biology class by means of statistical matching and by reviewing and comparing four different statistical matching methods (i.e., exact matching, nearest neighbor matching using the Mahalanobis distance, propensity score matching, and coarsened exact matching). Data from 1026 (73% female and 27% male) students who took a first-year biology course at a Western Canadian university were used. Two different measures for balance assessment of the matched data (i.e., % of balance improvement and standardized bias) were used to choose the best performing statistical matching method. Nearest neighbor matching using the Mahalanobis distance was found to be the most appropriate method for this study and results showed a statistically significant but small treatment effect for the group who received personalized feedback. Research and practical considerations were discussed and suggestions for future research are provided.
Descriptors: Automation, Feedback (Response), Intervention, College Freshmen, Biology, Intelligent Tutoring Systems, Science Instruction, College Science, Foreign Countries, Introductory Courses, Evaluation Methods, Statistical Analysis
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://bibliotheek.ehb.be:2123/
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Canada
Grant or Contract Numbers: N/A