ERIC Number: ED593198
Record Type: Non-Journal
Publication Date: 2018-Jul
Pages: 7
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Towards a Model-Free Estimate of the Limits to Student Modeling Accuracy
Chen, Binglin; West, Matthew; Ziles, Craig
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (11th, Raleigh, NC, Jul 16-20, 2018)
This paper attempts to quantify the accuracy limit of "nextitem-correct" prediction by using numerical optimization to estimate the student's probability of getting each question correct given a complete sequence of item responses. This optimization is performed without an explicit parameterized model of student behavior, but with the constraint that a student's likelihood of getting a problem correct only increases or remains unchanged with additional practice (i.e., no forgetting). We present results for this method for the Assistments 2009-2010 data where it suggests that there is only modest opportunity for improvement beyond the state of the art predictors. Furthermore, we describe a framework for applying this method to datasets where problems can be tagged with multiple skills and problem difficulties. Lastly, we discuss the limitations of this method, specifically its inability to give tight bounds on short sequences. [For the full proceedings, see ED593090.]
Descriptors: Accuracy, Probability, Student Behavior, Test Items, Models, Prediction, Item Response Theory, Error Patterns, Knowledge Level, Sequential Approach, Computer Software, Problem Solving, Computer Simulation, Difficulty Level
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF)
Authoring Institution: N/A
Grant or Contract Numbers: DUE1347722; CMMI1150490