ERIC Number: EJ1376572
Record Type: Journal
Publication Date: 2022
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1055-3096
EISSN: EISSN-2574-3872
Quality Assurance of Learning Assessments in Large Information Systems and Decision Analysis Courses
Ugray, Zsolt; Dunn, Brian K.
Journal of Information Systems Education, v33 n4 p405-415 Fall 2022
As Information Systems courses have become both more data-focused and student numbers have increased, there has emerged a greater need to assess technical and analytical skills more efficiently and effectively. Multiple-choice examinations provide a means for accomplishing this, though creating effective multiple-choice assessment items within a technical course context can be challenging. This study presents an iterative quality improvement framework based on Plan-Do-Study-Act (PDSA) quality assurance cycle for developing and improving such multiple-choice assessments. Integral to this framework, we also present a rigorous, reliable, and valid measure of assessment and item quality using discrimination efficiency and the KR-20 assessment reliability measure. We demonstrate the effectiveness of our approach across exams developed and administered for two courses -- one, a highly technical Information Systems introductory course and the other, an introductory data analytics course. Using this approach, we show that assessment quality iteratively improves when instructors measure items and exams rigorously and apply this PDSA framework.
Descriptors: Quality Assurance, Information Systems, Computer Science Education, Student Evaluation, Multiple Choice Tests, Test Items, Item Analysis, Data Analysis, Introductory Courses, Difficulty Level
Journal of Information Systems Education. e-mail: editor@jise.org; Web site: http://www.jise.org
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A