ERIC Number: ED607993
Record Type: Non-Journal
Publication Date: 2020-Jul
Pages: 6
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Iterative Feature Engineering through Text Replays of Model Errors
Slater, Stefan; Baker, Ryan S.; Wang, Yeyu
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (13th, Online, Jul 10-13, 2020)
Feature engineering, the construction of contextual and relevant features from system log data, is a crucial component of developing robust and interpretable models in educational data mining contexts. The practice of feature engineering depends on domain experts and system developers working in tandem in order to creatively identify actions and behaviors of interest. In this paper we outline a method of iterative feature engineering using the misclassifications of earlier models. By selecting cases where earlier models and ground truth disagree, we can focus attention on specific behaviors, or patterns of behavior, that a model is not using in its predictions. We show that iterative feature engineering on cases of false positives and false negatives improved a model predicting quitting in an educational video game by 15%. We close by discussing applications of this method for addressing model performance gaps across different classes of learners, as well as precautions against model overfitting with using this method of feature engineering. [For the full proceedings, see ED607784.]
Descriptors: Data Analysis, Engineering, Classification, Models, Computer Uses in Education, Prediction, Educational Games, Video Games, Physics, Science Instruction, Middle School Students
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Pennsylvania; Florida
Grant or Contract Numbers: N/A