ERIC Number: EJ1279988
Record Type: Journal
Publication Date: 2020
Pages: 14
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1939-1382
EISSN: N/A
Available Date: N/A
DEEPSTEALTH: Game-Based Learning Stealth Assessment with Deep Neural Networks
Min, Wookhee; Frankosky, Megan H.; Mott, Bradford W.; Rowe, Jonathan P.; Smith, Andy; Wiebe, Eric; Boyer, Kristy Elizabeth; Lester, James C.
IEEE Transactions on Learning Technologies, v13 n2 p312-325 Apr-Jun 2020
A distinctive feature of game-based learning environments is their capacity for enabling stealth assessment. Stealth assessment analyzes a stream of fine-grained student interaction data from a game-based learning environment to dynamically draw inferences about students' competencies through evidence-centered design. In evidence-centered design, evidence models have been traditionally designed using statistical rules authored by domain experts that are encoded using Bayesian networks. This article presents DEEPSTEALTH, a deep learning-based stealth assessment framework, that yields significant reductions in the feature engineering labor that has previously been required to create stealth assessments. DEEPSTEALTH utilizes end-to-end trainable deep neural network-based evidence models. Using this framework, evidence models are devised using a set of predictive features captured from raw, low-level interaction data to infer evidence for competencies. We investigate two deep learning-based evidence models, long short-term memory networks (LSTMs) and n-gram encoded feedforward neural networks (FFNNs). We compare these models' predictive performance for inferring students' knowledge to linear-chain conditional random fields (CRFs) and naïve Bayes models. We perform feature set-level analyses of game trace logs and external pre-learning measures, and we examine the models' early prediction capacity. The framework is evaluated using data collected from 182 middle school students interacting with a game-based learning environment for middle grade computational thinking. Results indicate that LSTM-based stealth assessors outperform competitive baseline approaches with respect to predictive accuracy and early prediction capacity. We find that LSTMs, FFNNs, and CRFs all benefit from combined feature sets derived from both game trace logs and external pre-learning measures.
Descriptors: Game Based Learning, Student Evaluation, Artificial Intelligence, Models, Predictor Variables, Middle School Students, Thinking Skills, Computation, Educational Environment, Bayesian Statistics
Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://bibliotheek.ehb.be:2578/xpl/RecentIssue.jsp?punumber=4620076
Publication Type: Journal Articles; Reports - Research
Education Level: Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF)
Authoring Institution: N/A
Grant or Contract Numbers: CNS1138497; DRL1640141
Author Affiliations: N/A