NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1440094
Record Type: Journal
Publication Date: 2024
Pages: 16
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-1939-1382
AI-Based Automatic Detection of Online Teamwork Engagement in Higher Education
Alejandra J. Magana; Syed Tanzim Mubarrat; Dominic Kao; Bedrich Benes
IEEE Transactions on Learning Technologies, v17 p2091-2106 2024
Fostering productive engagement within teams has been found to improve student learning outcomes. Consequently, characterizing productive and unproductive time during teamwork sessions is a critical preliminary step to increase engagement in teamwork meetings. However, research from the cognitive sciences has mainly focused on characterizing levels of productive engagement. Thus, the theoretical contribution of this study focuses on characterizing active and passive forms of engagement, as well as negative and positive forms of engagement. In tandem, researchers have used computer-based methods to supplement quantitative and qualitative analyses to investigate teamwork engagement. Yet, these studies have been limited to information extracted primarily from one data stream. For instance, text data from discussion forums or video data from recordings. We developed an artificial intelligence (AI)-based automatic system that detects productive and unproductive engagement during live teamwork sessions. The technical contribution of this study focuses on the use of three data streams from an interactive session: audio, video, and text. We automatically analyze them and determine each team's level of engagement, such as productive engagement, unproductive engagement, disengagement, and idle. The AI-based system was validated based on hand-coded data. We used the system to characterize productive and unproductive engagement patterns in teams using deep learning methods. Results showed that there were > 91% prediction accuracy and < 7% mismatches between predictions for the three engagement detectors. Moreover, Pearson's r values between the predictions of the three detectors were > 0.844. On a scale of -1 (unproductive engagement) to 1 (productive engagement), the scores for all teams were 0.94 ± 0.04, suggesting high productive engagement. In addition, teams tended to mostly be in productive engagement before transitioning to disengagement (> 90.34% of the time) and to idle (> 93.69% of the time). Before transitioning to productive engagement, we noticed almost equal fractions of teams being in idle and disengagement modes. These results show that the system effectively detects engagement and can be a viable tool for characterizing productive and unproductive engagement patterns in teamwork sessions.
Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://bibliotheek.ehb.be:2578/xpl/RecentIssue.jsp?punumber=4620076
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF)
Authoring Institution: N/A
Grant or Contract Numbers: 2113991; 2219271; 2417510