NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1378446
Record Type: Journal
Publication Date: 2023-Jun
Pages: 16
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0266-4909
EISSN: EISSN-1365-2729
Harnessing the Potential of Trace Data and Linguistic Analysis to Predict Learner Performance in a Multi-Text Writing Task
Rakovic, Mladen; Iqbal, Sehrish; Li, Tongguang; Fan, Yizhou; Singh, Shaveen; Surendrannair, Surya; Kilgour, Jonathan; Graaf, Joep; Lim, Lyn; Molenaar, Inge; Bannert, Maria; Moore, Johanna; Gaševic, Dragan
Journal of Computer Assisted Learning, v39 n3 p703-718 Jun 2023
Background: Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and by the characteristics of learning processes learners enacted while developing the product. However, formative feedback in writing tasks based on multiple texts has almost exclusively focused on essay product and rarely included SRL processes. Objectives: We explored the viability of using product and process features to develop machine learning classifiers that identify low- and high-performing essays in a multi-text writing task. Methods: We examined learning processes and essay submissions of 163 graduate students working on an authentic multi-text writing assignment. We utilised learners' trace data to obtain process features and state-of-the-art natural language processing methods to obtain product features for our classifiers. Results and Conclusions: Of four popular classifiers examined in this study, Random Forest achieved the best performance (accuracy = 0.80 and recall = 0.77). The analysis of important features identified in the Random Forest classification model revealed one product (coverage of reading topics) and three process (elaboration/organisation, re-reading and planning) features as important predictors of writing quality. Major Takeaways: The classifier can be used as a part of a future automated writing evaluation system that will support at scale formative assessment in writing tasks based on multiple texts in different courses. Based on important predictors of essay performance, a guidance can be tailored to learners at the outset of a multi-text writing task to help them do well in the task.
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://bibliotheek.ehb.be:2191/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A