ERIC Number: EJ1375352
Record Type: Journal
Publication Date: 2022
Pages: 13
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-1929-7750
Multimodal Data Fusion to Track Students' Distress during Educational Gameplay
Journal of Learning Analytics, v9 n3 p75-87 2022
Using multimodal data fusion techniques, we built and tested prediction models to track middle-school student distress states during educational gameplay. We collected and analyzed 1,145 data instances, sampled from a total of 31 middle-school students' audio- and video-recorded gameplay sessions. We conducted data wrangling with student gameplay data from multiple data sources, such as individual facial expression recordings and gameplay logs. Using supervised machine learning, we built and tested candidate classifiers that yielded an estimated probability of distress states. We then conducted confidence-based data fusion that averaged the estimated probability scores from the unimodal classifiers with a single data source. The results of this study suggest that the classifier with multimodal data fusion improves the performance of tracking distress states during educational gameplay, compared to the performance of unimodal classifiers. The study finding suggests the feasibility of multimodal data fusion in developing game-based learning analytics. Also, this study proposes the benefits of optimizing several methodological means for multimodal data fusion in educational game research.
Descriptors: Learning Analytics, Stress Variables, Educational Games, Middle School Students, Data Collection, Prediction, Identification, Cognitive Processes, Affective Behavior, Video Technology, Videoconferencing, Nonverbal Communication, Problem Solving
Society for Learning Analytics Research. 121 Pointe Marsan, Beaumont, AB T4X 0A2, Canada. Tel: +61-429-920-838; e-mail: info@solaresearch.org; Web site: https://learning-analytics.info/index.php/JLA/index
Publication Type: Journal Articles; Reports - Research
Education Level: Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: Department of Education (ED)
Authoring Institution: N/A
Grant or Contract Numbers: U411C190179