ERIC Number: EJ1452776
Record Type: Journal
Publication Date: 2024
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1449-3098
EISSN: EISSN-1449-5554
AI-Assisted Marking: Functionality and Limitations of ChatGPT in Written Assessment Evaluation
Joan Li; Nikhil Kumar Jangamreddy; Ryuto Hisamoto; Ruchita Bhansali; Amalie Dyda; Luke Zaphir; Mashhuda Glencross
Australasian Journal of Educational Technology, v40 n4 p56-72 2024
Generative artificial intelligence technologies, such as ChatGPT, bring an unprecedented change in education by leveraging the power of natural language processing and machine learning. Employing ChatGPT to assist with marking written assessment presents multiple advantages including scalability, improved consistency, eliminating biases associated with human subjectivity. This work aimed to evaluate the usefulness, reliability and accuracy of ChatGPT in marking written assessments of varied types and to identify its limitations and challenges. ChatGPT was instructed using a set of prompts to mark the assessment based on a rubric. ChatGPT was able to evaluate and assess both coding and reflective assessments and to distinguish between assignments of different quality, demonstrating high consistency and accuracy for higher quality assessments, comparable to a human marking. ChatGPT was also able to generate textual detailed justifications based on the rubric and assessment task description. There was a significant difference in the outcomes generated by different prompts. These preliminary findings suggest that utilising ChatGPT as a marking assistant can increase written assessment marking efficiency, reduce cost and potentially decrease the unfairness and bias by providing a moderating perspective.
Descriptors: Higher Education, Artificial Intelligence, Grading, Scoring Rubrics, Student Evaluation, Writing Evaluation, Reliability, Accuracy
Australasian Society for Computers in Learning in Tertiary Education. Ascilite Secretariat, P.O. Box 44, Figtree, NSW, Australia. Tel: +61-8-9367-1133; e-mail: info@ascilite.org.au; Web site: https://ajet.org.au/index.php/AJET
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A