ERIC Number: ED629711
Record Type: Non-Journal
Publication Date: 2023
Pages: 7
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Using Large Language Models to Provide Formative Feedback in Intelligent Textbooks
Grantee Submission, Paper presented at the International Conference on Artificial Intelligence in Education (AIED) (2023)
As intelligent textbooks become more ubiquitous in classrooms and educational settings, the need arises to automatically provide formative feedback to written responses provided by students in response to readings. This study develops models to automatically provide feedback to student summaries written at the end of intelligent textbook sections. The study builds on Botarleanu et al. (2022), who used the Longformer Large Language Model, a transformer Neural Network, to build a summary grading model. Their model explains around 55% of holistic summary score variance when compared to scores assigned by human raters on an analytic rubric. This study uses a principal component analysis to distill scores from the analytic rubric into two principal components -- content and wording. When training the models on the summaries and the sources using these principal components, we explained 79% and 66% of the score variance for content and wording, respectively. The developed models are freely available on HuggingFace and will allow formative feedback to users of intelligent textbooks to assess reading comprehension through summarization in real-time. The models can also be used for other summarization applications in learning systems. [This paper was published in: "AIED 2023, CCIS 1831," edited by N. Wang et al., Springer Nature Switzerland, 2023, pp. 484-489.]
Descriptors: Textbooks, Electronic Publishing, Feedback (Response), Formative Evaluation, Scores, Comparative Analysis, Scoring Rubrics, Factor Analysis, Reading Comprehension, Computational Linguistics, Intelligent Tutoring Systems, Writing Evaluation, High School Students, College Students, Adult Education
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: High Schools; Secondary Education; Higher Education; Postsecondary Education; Adult Education
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF); Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: 2112532; R305A180261