ERIC Number: EJ1311586
Record Type: Journal
Publication Date: 2021-Nov
Pages: 15
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0007-1013
EISSN: N/A
Exploring the Reliability and Its Influencing Factors of Peer Assessment in Massive Open Online Courses
Li, Hongxia; Zhao, ChengLing; Long, Taotao; Huang, Yan; Shu, Fengfang
British Journal of Educational Technology, v52 n6 p2263-2277 Nov 2021
As an innovative evaluation tool, peer assessment is essential in Massive Open Online Courses (MOOCs). In both formative and summative peer assessments in MOOCs, providing reliable feedback is crucial in enhancing learning outcomes. Peer assessment has been highlighted as a reliable tool in both traditional classrooms and small-scale online learning contexts by a wealth of studies. Factors impacting the result of peer assessment have also been identified. However, such exploration in MOOCs is very limited. Through an examination of 5,722 assignment submissions and 56,794 review scores collected from 18 peer assessments in three courses on Chinese University MOOC platform, this study investigated the inter-reliability of peer assessment in MOOCs utilizing Krippendorff's alpha and ICC. It also revealed factors affecting peer assessment reliability in MOOCs. The results of this study show that students tended to mark extreme scores, and peer assessment in MOOCs seemed unreliable. In addition, the reliability of the peer assessment in these MOOCs was inversely correlated to the number of reviewers per assignment and reviews completed per reviewer. The assignment type also matters; peer assessment using e-portfolios format was found more reliable than those using papers and proposals. Suggestions on how to improve the reliability of peer assessments in MOOCs are also provided.
Descriptors: Peer Evaluation, Online Courses, Open Education, Feedback (Response), Scores, Assignments, Portfolio Assessment, Correlation, College Students, Interrater Reliability, Foreign Countries
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://bibliotheek.ehb.be:2191/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: China
Grant or Contract Numbers: N/A