ERIC Number: ED637012
Record Type: Non-Journal
Publication Date: 2018
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Predicting Question Quality Using Recurrent Neural Networks
Stefan Ruseti; Mihai Dascalu; Amy M. Johnson; Renu Balyan; Kristopher J. Kopp; Danielle S. McNamara
Grantee Submission, Paper presented at the International Conference on Artificial Intelligence in Education (19th, London, United Kingdom, Jun 27-30, 2018)
This study assesses the extent to which machine learning techniques can be used to predict question quality. An algorithm based on textual complexity indices was previously developed to assess question quality to provide feedback on questions generated by students within iSTART (an intelligent tutoring system that teaches reading strategies). In this study, 4,575 questions were coded by human raters based on their corresponding depth, classifying questions into four categories: 1-very shallow to 4-very deep. Here we propose a novel approach to assessing question quality within this dataset based on Recurrent Neural Networks (RNNs) and word embeddings. The experiments evaluated multiple RNN architectures using GRU, BiGRU and LSTM cell types of different sizes, and different word embeddings (i.e., FastText and Glove). The most precise model achieved a classification accuracy of 81.22%, which surpasses the previous prediction results using lexical sophistication complexity indices (accuracy = 41.6%). These results are promising and have implications for the future development of automated assessment tools within computer-based learning environments. [This is a paper in: Penstein Rosé, C., et al. "Artificial Intelligence in Education." AIED 2018. Lecture Notes in Computer Science(), vol 10947. Springer, Cham.]
Descriptors: Questioning Techniques, Artificial Intelligence, Networks, Classification, Accuracy, Feedback (Response), Reading Instruction, Reading Strategies, Intelligent Tutoring Systems, Computer Assisted Instruction, Prediction, Educational Quality, Taxonomy, Computational Linguistics, Natural Language Processing, Distance Education, Adult Literacy
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED); Office of Naval Research (ONR) (DOD)
Authoring Institution: N/A
Identifiers - Location: California
IES Funded: Yes
Grant or Contract Numbers: R305A130124; N00014140343; N000141712300