Abstract:
In peer assessment, students assess a task done by their peers, provide feedback and usually a grade. The extent to which these peer grades can be used to formally grade ...Show MoreMetadata
Abstract:
In peer assessment, students assess a task done by their peers, provide feedback and usually a grade. The extent to which these peer grades can be used to formally grade the task is unclear, with doubts often arising regarding their validity. The instructor could supervise the peer assessments, but would not then benefit from workload reduction, one of the most appealing features of peer assessment for instructors. Our proposal uses a probabilistic model to estimate a grade for each test, accounting for the degree of precision and bias of grading peers. The grade that the instructor would assign to a test can help enhance the model. Our main hypothesis is that guiding the instructor through supervision of a peer-assessed task by pointing out to them which test to evaluate next can lead to improvement in the validity of the model-estimated grades at an early stage. Moreover, the instructor can decide how many tests to grade based on their own criteria of tolerable uncertainty, as measured by the model. We validate the method using both synthetically generated data and real data collected in an actual class. Models that link the roles of the student as grading peer and as test-taker appear to better exploit available information, although simpler models are more appropriate in specific conditions. The best performing technique for guiding the instructor is that which selects the test with the highest expected entropy reduction. In general, empirical results are in line with the hypothesis of this study.
Published in: IEEE Transactions on Learning Technologies ( Volume: 16, Issue: 6, December 2023)
Funding Agency:

Department of Computer Science, Applied Mathematics and Statistics, University of Girona, Girona, Spain
Jerónimo Hernández-González received the Ph.D. degree in computer science from the University of the Basque Country, Donostia, Spain, in 2015.
He is currently a Lecturer with the Department of Computer Science, Applied Mathematics and Statistics, University of Girona, Girona, Spain. His research interests include learning and inference with probabilistic graphical models and their application to biomedicine and education.
Jerónimo Hernández-González received the Ph.D. degree in computer science from the University of the Basque Country, Donostia, Spain, in 2015.
He is currently a Lecturer with the Department of Computer Science, Applied Mathematics and Statistics, University of Girona, Girona, Spain. His research interests include learning and inference with probabilistic graphical models and their application to biomedicine and education.View more

Department of Software Engineering and Computer Systems, National University of Distance Education, Madrid, Spain
Pedro Javier Herrera received the Ph.D. degree in computer science from the Complutense University of Madrid, Madrid, Spain, in 2010.
He is currently an Associate Professor with the Software Engineering and Computer Systems Department, National University of Distance Education, Madrid. His research interests include computer vision, pattern recognition, artificial intelligence, and robotics with interests in continuous tra...Show More
Pedro Javier Herrera received the Ph.D. degree in computer science from the Complutense University of Madrid, Madrid, Spain, in 2010.
He is currently an Associate Professor with the Software Engineering and Computer Systems Department, National University of Distance Education, Madrid. His research interests include computer vision, pattern recognition, artificial intelligence, and robotics with interests in continuous tra...View more

Department of Computer Science, Applied Mathematics and Statistics, University of Girona, Girona, Spain
Jerónimo Hernández-González received the Ph.D. degree in computer science from the University of the Basque Country, Donostia, Spain, in 2015.
He is currently a Lecturer with the Department of Computer Science, Applied Mathematics and Statistics, University of Girona, Girona, Spain. His research interests include learning and inference with probabilistic graphical models and their application to biomedicine and education.
Jerónimo Hernández-González received the Ph.D. degree in computer science from the University of the Basque Country, Donostia, Spain, in 2015.
He is currently a Lecturer with the Department of Computer Science, Applied Mathematics and Statistics, University of Girona, Girona, Spain. His research interests include learning and inference with probabilistic graphical models and their application to biomedicine and education.View more

Department of Software Engineering and Computer Systems, National University of Distance Education, Madrid, Spain
Pedro Javier Herrera received the Ph.D. degree in computer science from the Complutense University of Madrid, Madrid, Spain, in 2010.
He is currently an Associate Professor with the Software Engineering and Computer Systems Department, National University of Distance Education, Madrid. His research interests include computer vision, pattern recognition, artificial intelligence, and robotics with interests in continuous training and teaching innovation.
Pedro Javier Herrera received the Ph.D. degree in computer science from the Complutense University of Madrid, Madrid, Spain, in 2010.
He is currently an Associate Professor with the Software Engineering and Computer Systems Department, National University of Distance Education, Madrid. His research interests include computer vision, pattern recognition, artificial intelligence, and robotics with interests in continuous training and teaching innovation.View more