ERIC Number: EJ1115353
Record Type: Journal
Publication Date: 2012
Pages: 54
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2157-2100
EISSN: N/A
Putting ECD into Practice: The Interplay of Theory and Data in Evidence Models within a Digital Learning Environment
Rupp, Andre A.; Levy, Roy; Dicerbo, Kristen E.; Sweet, Shauna J.; Crawford, Aaron V.; Calico, Tiago; Benson, Martin; Fay, Derek; Kunze, Katie L.; Mislevy, Robert J.; Behrens, John T.
Journal of Educational Data Mining, v4 n1 p49-102 2012
In this paper we describe the development and refinement of "evidence rules" and "measurement models" within the "evidence model" of the "evidence-centered design" (ECD) framework in the context of the "Packet Tracer" digital learning environment of the "Cisco Networking Academy." Using "Packet Tracer" learners design, configure, and troubleshoot computer networks within an interactive interface. This leads to "product data," which result from the students' final submitted network configurations, and "process data," which are log file entries detailing how they got to the final configurations. We discuss how an iterative cycle of empirical analyses and discussions with subject-matter experts is essential for identifying and accumulating evidence about skill profiles of learners and their development. We present results from descriptive, exploratory, and confirmatory diagnostic modeling analyses for both data types, which required bringing to bear a diversity of tools from multivariate statistics, modern psychometrics, and educational data mining. We close the paper with a discussion of the implications of this work for evidence-based argumentation guided by ECD principles within digital learning environments more generally.
Descriptors: Computer Networks, Evidence Based Practice, Design, Instructional Design, Theory Practice Relationship, Models, Information Retrieval, Data Processing, Measurement, Troubleshooting, Computer Interfaces, Bayesian Statistics, Electronic Learning, Case Studies, Simulated Environment, Visualization, Computer Software, Engineering, Data Analysis, Multivariate Analysis, Psychometrics, Classification
International Working Group on Educational Data Mining. e-mail: jedm.editor@gmail.com; Web site: http://www.educationaldatamining.org/JEDM/index.php/JEDM/index
Publication Type: Journal Articles; Tests/Questionnaires; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED); National Science Foundation (NSF)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305C080015; DRL0918409; DRL0946372