ERIC Number: EJ1434964
Record Type: Journal
Publication Date: 2024
Pages: 15
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1049-4820
EISSN: EISSN-1744-5191
The Guts of Assessment: A Digital Architecture for Machine Learning and Analogue Judgement
Interactive Learning Environments, v32 n5 p1988-2002 2024
Educational assessment is inherently uncertain, where physiological, psychological and social factors play an important role in establishing judgements which are assumed to be "absolute". AI and other algorithmic approaches to grading of student work strip-out uncertainty, leading to a lack of inspectability in machine judgement and consequent problems of trust and reliability. The technique of Adaptive Comparative Judgement (ACJ), in focusing on small-stakes binary comparisons provides an alternative approach to dealing with uncertainty in grading. Rankings can be produced which codify uncertainty, rendering machine judgements inspectable. However, ACJ demands resources in terms of time and the number of people making comparisons. Machine learning trained to make binary comparisons can help to make the process of human comparison more efficient. In combining ACJ and AI, we argue that the result is an analogue-digital system where the physiological/analogue processes of assessment can be coordinated with digital services which steer human assessment to those judgements which are most uncertain requiring most human deliberation. Drawing on our design of such a system developed for medical judgements, we describe a general architecture for human-machine assessment in education and discuss its potential in bridging the gap between analogue human cognition and digital machine learning.
Descriptors: Artificial Intelligence, Evaluation Methods, Technology Uses in Education, Man Machine Systems, Coding, Algorithms, Conventional Instruction, Comparative Analysis
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A