NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1433237
Record Type: Journal
Publication Date: 2024-Jul
Pages: 27
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1360-2357
EISSN: EISSN-1573-7608
Engagement Assessment in Project-Based Education: A Machine Learning Approach in Team Chat Analysis
Sabah Farshad; Evgenii Zorin; Nurlybek Amangeldiuly; Clement Fortin
Education and Information Technologies, v29 n10 p13105-13131 2024
Project-based Learning (PBL) provides an effective environment for collaborative engineering design education. However, it is difficult to assess students' engagement and provide process-oriented feedback on their collaboration due to limited resources and scalability challenges. This paper presents an empirical study examining the application of Machine Learning (ML) to evaluate student engagement in PBL. A supervised ML model (SML) was developed using Natural Language Processing (NLP) to predict engagement levels using conversation data in online chat groups and comparing them to the ground truth metrics. Eleven teams of engineering students participated in the PBL courses on systems engineering while employing text-based discussions. Communication messages were manually labelled for active engagement metrics collected in our prior research. The SML was constructed with Gradient Boosting to classify engagement levels. The model was trained using more than 5,000 manually classified text messages, and it demonstrated an accuracy range of 0.75 to 0.81 on held-out test data. The model's predictions for complete team communication showed a meaningful association with the actual levels of engagement as determined by log data and surveys. Moreover, the model reproduces general patterns of team engagement and collaboration. According to the findings, conversational data can be a scalable source for ML-driven engagement assessment. This shows how AI has the potential to scale up process-oriented assessment in PBL. Future research should look towards instructor-AI dashboards and individualized feedback systems. The work adds an empirical evaluation of AI for automated engagement analysis, extending earlier research on feedback systems to tackle PBL issues.
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://bibliotheek.ehb.be:2123/
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A