NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1404294
Record Type: Journal
Publication Date: 2023
Pages: 37
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1560-4292
EISSN: EISSN-1560-4306
Assessing Teamwork Skills: Can a Computer Algorithm Match Human Experts?
Kotlyar, Igor; Sharifi, Tina; Fiksenbaum, Lisa
International Journal of Artificial Intelligence in Education, v33 n4 p955-991 2023
Teamwork skills are commonly evaluated by human assessors, which can be logistically challenging and resource intensive. Technological advancements provide an opportunity for a new assessment method -- virtual behavioural simulations with self-scoring algorithms. This study explores whether a rule-based algorithm can match human assessors at evaluating teamwork skills. 206 undergraduate students completed a virtual simulation assessment, where they interacted with "teammates" (represented by chatbots) using natural language. In this study, students' teamwork skills were assessed independently by a computer algorithm and two human experts based on the transcripts of their conversations with "teammates" (chatbots). The relative accuracy of these assessments was evaluated against peer- and self-evaluations of teamwork. The assessment scores generated by the algorithm and human experts were highly correlated with each other and were comparable in their ability to predict teamwork. The scores generated by the algorithm were slightly more correlated with peer-evaluations than those generated by human experts (r = 0.25 and r = 0.17, respectively; p = 0.21). The results indicate that AI-based techniques offer a promising method of skill assessment to support learning and acquisition teamwork skills.
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://bibliotheek.ehb.be:2123/
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A