ERIC Number: EJ1381249
Record Type: Journal
Publication Date: 2023-Jun
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-1939-1382
Peer Grading Eliciting Truthfulness Based on Autograder
IEEE Transactions on Learning Technologies, v16 n3 p353-363 Jun 2023
Peer grading has diverse applications in many fields, including the peer grading of open assignments in online courses. The major challenge in peer grading is improving the seriousness (reviewing carefully) of reviewers. Previous studies have proposed several incentive reward mechanisms intended to reward or punish reviewers. Although these mechanisms are effective in massive open online courses environments with a large number of students, they are not suitable for small private online courses (SPOCs) environments with a small number of students. This article analyzes the characteristics of reviewers in a small private online course environment and designs an incentive and reward mechanism based on a hybrid human-machine peer grading framework, including an autograder (reviewed by the machine). The main component of the proposed mechanism is the autograder whose function is to evaluate the reviewers' reports (reviewers refer to students) and provide feedback information on reports to reviewers. The framework can incentivize reviewers to review carefully and give their reports truthfully when the reports between reviewers' and the autograder' are consistent. The proposed framework is verified experimentally. The experimental results demonstrate that the hybrid human-machine peer grading framework can effectively incentivize reviewers to review submissions carefully and improve the overall effect of peer grading.
Descriptors: Grading, Peer Evaluation, Online Courses, Small Classes, Incentives, Rewards, Automation, Educational Technology, Man Machine Systems
Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://bibliotheek.ehb.be:2578/xpl/RecentIssue.jsp?punumber=4620076
Publication Type: Journal Articles; Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A