ERIC Number: EJ1381268
Record Type: Journal
Publication Date: 2023-Jun
Pages: 16
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-1939-1382
Toward Human-AI Collaboration: A Recommender System to Support CS1 Instructors to Select Problems for Assignments and Exams
Pereira, Filipe Dwan; Rodrigues, Luiz; Henklain, Marcelo Henrique Oliveira; Freitas, Hermino; Oliveira, David Fernandes; Cristea, Alexandra I.; Carvalho, Leandro; Isotani, Seiji; Benedict, Aileen; Dorodchi, Mohsen; de Oliveira, Elaine Harada Teixeira
IEEE Transactions on Learning Technologies, v16 n3 p457-472 Jun 2023
Programming online judges (POJs) have been increasingly used in CS1 classes, as they allow students to practice and get quick feedback. For instructors, it is a useful tool for creating assignments and exams. However, selecting problems in POJs is time consuming. First, problems are generally not organized based on topics covered in the CS1 syllabus. Second, assessing whether problems require similar effort to be completed and map onto the same topic is a subjective and expert-dependent task. The difficulty increases if the instructor must create variations of these assessments, e.g., to avoid plagiarism. Thus, here, we research how to support CS1 instructors in the task of selecting problems, to compose one-size-fits-all or personalized assignments/exams. Our solution is to propose a novel intelligent recommender system, based on a fine-grained data-driven analysis of the students' effort on solving problems in the integrated development environment of a POJ system, and automatic detection of topics for CS1 problems, based on problem descriptions. Data collected from 2714 students are processed to support, via our artificial intelligence (AI) method recommendations, the instructors' decision-making process. We evaluated our method against the state of the art in a simple blind experiment with CS1 instructors (N = 35). Results show that our recommendations are 88% accurate, surpassing our baseline (p = 0.05). Finally, our work paves the way for novel POJ smart learning environments, wherein instructors define learning tasks (assignments/exams) supported by AI.
Descriptors: Artificial Intelligence, Man Machine Systems, Educational Technology, Technology Uses in Education, Problem Solving, Accuracy, Assignments, Tests
Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://bibliotheek.ehb.be:2578/xpl/RecentIssue.jsp?punumber=4620076
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A