Abstract:
In an intelligent tutoring system (ITS), it can be useful to know when a student has disengaged from a task and might benefit from a particular intervention. However, pre...Show MoreMetadata
Abstract:
In an intelligent tutoring system (ITS), it can be useful to know when a student has disengaged from a task and might benefit from a particular intervention. However, predicting disengagement on a trial-by-trial basis is a challenging problem, particularly in complex cognitive domains. In the present work, data-driven methods were used to address two aspects of this problem: identification of predictive features at the single-trial level, and selection of accurate and robust models. Experiment data were collected in a middle-school classroom using a vocabulary training ITS. On each trial, the ITS presented a low-frequency (Tier 2 or frontier) word and prompted students to type in the word's meaning. Single-trial measuresincluding the orthographic and semantic accuracy of each response, and context-sensitive measures such as interaction patterns across trialswere computed throughout the task. There were two key findings. First, as expected, different features predicted when a student was likely to be more engaged (e.g., high semantic accuracy) or less engaged (e.g., repetition of same or similar words across consecutive trials). Second, there was added value in representing context-sensitive information, which captures patterns of performance over time, as well as trial-specific information. These findings provide useful insights into effective methods for representing and modeling temporal patterns of student engagement in an ITS, especially those related to language learning. Such models may be useful in the design and implementation of adaptive tutors in complex cognitive domains like language learning.
Published in: IEEE Transactions on Learning Technologies ( Volume: 11, Issue: 3, 01 July-Sept. 2018)
Funding Agency:

University of Michigan, Ann Arbor, MI, US
SungJin Nam received the BA degree in psychology and information and
culture technology from Seoul National University and the MS degree in information from the University of Michigan.
Currently, he is working toward the doctoral degree in the School of Information, University of Michigan. With
backgrounds in both psychology and data science, he is interested in interpreting human behavior using data-dr...Show More
SungJin Nam received the BA degree in psychology and information and
culture technology from Seoul National University and the MS degree in information from the University of Michigan.
Currently, he is working toward the doctoral degree in the School of Information, University of Michigan. With
backgrounds in both psychology and data science, he is interested in interpreting human behavior using data-dr...View more

University of Oregon School of Law, Eugene, OR, US
Gwen Frishkoff is a research associate professor of psychology with
the University of Oregon. She combines her expertise in cognitive psychology, linguistics, and neuroscience with an
interest in vocabulary acquisition. She is co-Principal Investigator, with Dr. Kevyn Collins-Thompson, on the Dynamic
Support of Contextual Vocabulary Acquisition for Reading (DSCoVAR) project, which was funded by the Inst...Show More
Gwen Frishkoff is a research associate professor of psychology with
the University of Oregon. She combines her expertise in cognitive psychology, linguistics, and neuroscience with an
interest in vocabulary acquisition. She is co-Principal Investigator, with Dr. Kevyn Collins-Thompson, on the Dynamic
Support of Contextual Vocabulary Acquisition for Reading (DSCoVAR) project, which was funded by the Inst...View more

University of Michigan, Ann Arbor, MI, US
Kevyn Collins-Thompson is an associate professor of information and
computer science with the University of Michigan. His expertise lies in human language technology, data mining, and
computational methods for connecting users with information. He is co-PI, with Dr. Frishkoff, on the DSCoVAR project.
In related work, he pioneered the development of computational methods for assessing reading difficulty,...Show More
Kevyn Collins-Thompson is an associate professor of information and
computer science with the University of Michigan. His expertise lies in human language technology, data mining, and
computational methods for connecting users with information. He is co-PI, with Dr. Frishkoff, on the DSCoVAR project.
In related work, he pioneered the development of computational methods for assessing reading difficulty,...View more

University of Michigan, Ann Arbor, MI, US
SungJin Nam received the BA degree in psychology and information and
culture technology from Seoul National University and the MS degree in information from the University of Michigan.
Currently, he is working toward the doctoral degree in the School of Information, University of Michigan. With
backgrounds in both psychology and data science, he is interested in interpreting human behavior using data-driven
methods.
SungJin Nam received the BA degree in psychology and information and
culture technology from Seoul National University and the MS degree in information from the University of Michigan.
Currently, he is working toward the doctoral degree in the School of Information, University of Michigan. With
backgrounds in both psychology and data science, he is interested in interpreting human behavior using data-driven
methods.View more

University of Oregon School of Law, Eugene, OR, US
Gwen Frishkoff is a research associate professor of psychology with
the University of Oregon. She combines her expertise in cognitive psychology, linguistics, and neuroscience with an
interest in vocabulary acquisition. She is co-Principal Investigator, with Dr. Kevyn Collins-Thompson, on the Dynamic
Support of Contextual Vocabulary Acquisition for Reading (DSCoVAR) project, which was funded by the Institutes of
Education Sciences (IES, Department of Education).
Gwen Frishkoff is a research associate professor of psychology with
the University of Oregon. She combines her expertise in cognitive psychology, linguistics, and neuroscience with an
interest in vocabulary acquisition. She is co-Principal Investigator, with Dr. Kevyn Collins-Thompson, on the Dynamic
Support of Contextual Vocabulary Acquisition for Reading (DSCoVAR) project, which was funded by the Institutes of
Education Sciences (IES, Department of Education).View more

University of Michigan, Ann Arbor, MI, US
Kevyn Collins-Thompson is an associate professor of information and
computer science with the University of Michigan. His expertise lies in human language technology, data mining, and
computational methods for connecting users with information. He is co-PI, with Dr. Frishkoff, on the DSCoVAR project.
In related work, he pioneered the development of computational methods for assessing reading difficulty, and developed
an algorithmic approach, called Markov Estimation of Semantic Association (MESA), which is used for fine-grained and
incremental assessment of partial word knowledge.
Kevyn Collins-Thompson is an associate professor of information and
computer science with the University of Michigan. His expertise lies in human language technology, data mining, and
computational methods for connecting users with information. He is co-PI, with Dr. Frishkoff, on the DSCoVAR project.
In related work, he pioneered the development of computational methods for assessing reading difficulty, and developed
an algorithmic approach, called Markov Estimation of Semantic Association (MESA), which is used for fine-grained and
incremental assessment of partial word knowledge.View more