Abstract:
With the widespread application of online education platforms, the necessity for identifying learners’ mental states from webcam videos is increasing as it can be potenti...Show MoreMetadata
Abstract:
With the widespread application of online education platforms, the necessity for identifying learners’ mental states from webcam videos is increasing as it can be potentially applied to artificial intelligence-based automatic identification of learner states. However, the behaviors that elementary school students frequently exhibit during online learning particularly when they are in a low attention state have rarely been investigated. This study employed electroencephalography (EEG) to continuously track changes in the learner's attention state during online learning. A new EEG index reflecting elementary students’ attention level was developed using an EEG dataset acquired from 34th graders during a computerized d2 test of attention. Characteristic behaviors of 24 elementary students in a low attention state were then identified from the webcam videos showing their upper bodies captured during 40-min online lectures, with the proposed EEG index being used as a reference to determine their attention level at the time. Various characteristic behaviors were identified regarding the participant's mouth, head, arms, and torso. For example, opening the mouth or leaning back was observed more frequently in a low-attention state than in a high-attention state. It is expected that the characteristic behaviors reflecting a learner's low attention state would be utilized as a useful reference in developing more interactive and effective online education systems.
Published in: IEEE Transactions on Learning Technologies ( Volume: 17)
Funding Agency:
Department of Electronic Engineering, Hanyang University, Seoul, South Korea
Suhye Kim (Graduate Student Member, IEEE) received the B.S. degree in psychology from the Department of Psychology, Chungbuk National University, Cheongju, South Korea, in 2020. She is currently working toward the Ph.D. degree in electronic engineering with the Department of Electronic Engineering, Hanyang University, Seoul, South Korea.
Her research interests include brain and human–computer interface, affective computing...Show More
Suhye Kim (Graduate Student Member, IEEE) received the B.S. degree in psychology from the Department of Psychology, Chungbuk National University, Cheongju, South Korea, in 2020. She is currently working toward the Ph.D. degree in electronic engineering with the Department of Electronic Engineering, Hanyang University, Seoul, South Korea.
Her research interests include brain and human–computer interface, affective computing...View more
Department of Artificial Intelligence, Hanyang University, Seoul, South Korea
Jung-Hwan Kim (Graduate Student Member, IEEE) received the B.S. degree in electrical engineering and the M.S. degree in electronic engineering from Hanyang University, Seoul, South Korea, in 2019 and 2021, respectively, where he is currently working toward the Ph.D. degree in artificial intelligence with the Department of Artificial Intelligence.
His research interests include brain–computer interfaces, biomedical signal p...Show More
Jung-Hwan Kim (Graduate Student Member, IEEE) received the B.S. degree in electrical engineering and the M.S. degree in electronic engineering from Hanyang University, Seoul, South Korea, in 2019 and 2021, respectively, where he is currently working toward the Ph.D. degree in artificial intelligence with the Department of Artificial Intelligence.
His research interests include brain–computer interfaces, biomedical signal p...View more
Department of Artificial Intelligence, Hanyang University, Seoul, South Korea
Wooseok Hyung (Graduate Student Member, IEEE) received the B.S. degree in digital imaging engineering from the School of Integrative Engineering, Chung-Ang University, Seoul, South Korea, in 2022. He is currently working toward the M.S. degree in artificial intelligence with the Department of Artificial Intelligence, Hanyang University, Seoul.
His research interests include brain and human–computer interface and biomedical...Show More
Wooseok Hyung (Graduate Student Member, IEEE) received the B.S. degree in digital imaging engineering from the School of Integrative Engineering, Chung-Ang University, Seoul, South Korea, in 2022. He is currently working toward the M.S. degree in artificial intelligence with the Department of Artificial Intelligence, Hanyang University, Seoul.
His research interests include brain and human–computer interface and biomedical...View more
Department of Education, Hanyang University, Seoul, South Korea
Suhkyung Shin received the Ph.D. degree in instructional systems technology from Indiana University, Bloomington, IN, USA, in 2016.
From 2016 to 2018, she was a Postdoctoral Research Associate of Instructional Technology at the College of Education, Texas Tech University, Lubbock, TX, USA. She is currently an Assistant Professor with the School of Education, Hanyang University, Seoul, South Korea. Her expertise lies in the...Show More
Suhkyung Shin received the Ph.D. degree in instructional systems technology from Indiana University, Bloomington, IN, USA, in 2016.
From 2016 to 2018, she was a Postdoctoral Research Associate of Instructional Technology at the College of Education, Texas Tech University, Lubbock, TX, USA. She is currently an Assistant Professor with the School of Education, Hanyang University, Seoul, South Korea. Her expertise lies in the...View more
Samsung SDS, Seoul, South Korea
Myoung Jin Choi received the B.A. degree in economics from Underwood International College, Yonsei University, Seoul, South Korea, in 2014.
She was a Software Engineer with the Samsung SDS, Seoul, South Korea. From 2021 to 2022, she was a Manager of R&D for an EdTech corporate venture at the goal of assessing students’ attention/focus levels using vision artificial intelligence in the context of online learning. She is als...Show More
Myoung Jin Choi received the B.A. degree in economics from Underwood International College, Yonsei University, Seoul, South Korea, in 2014.
She was a Software Engineer with the Samsung SDS, Seoul, South Korea. From 2021 to 2022, she was a Manager of R&D for an EdTech corporate venture at the goal of assessing students’ attention/focus levels using vision artificial intelligence in the context of online learning. She is als...View more
Center for Bionics, Biomedical Research Institute, Korea Institute of Science and Technology, Seoul, South Korea
Dong Hwan Kim (Member, IEEE) received the B.S. degree in electrical engineering and the M.S. and Ph.D. degrees in electrical engineering and computer science from Seoul National University, Seoul, South Korea, in 1999, 2001, and 2006, respectively.
From 2006 to 2007, he was a Senior Engineer at Samsung Electronics, Company Ltd., Suwon, South Korea. In 2007, he joined Korea Institute of Science and Technology (KIST), Seoul,...Show More
Dong Hwan Kim (Member, IEEE) received the B.S. degree in electrical engineering and the M.S. and Ph.D. degrees in electrical engineering and computer science from Seoul National University, Seoul, South Korea, in 1999, 2001, and 2006, respectively.
From 2006 to 2007, he was a Senior Engineer at Samsung Electronics, Company Ltd., Suwon, South Korea. In 2007, he joined Korea Institute of Science and Technology (KIST), Seoul,...View more
Department of Biomedical Engineering, HY-KIST Bioconvergence, and the Department of Electronic Engineering, Hanyang University, Seoul, South Korea
Chang-Hwan Im (Member, IEEE) received the B.S. degree in electrical engineering from the School of Electrical Engineering, Seoul National University, Seoul, South Korea, in 1999, and the M.S. and Ph.D. degrees in electrical and computer engineering from Seoul National University, in 2001 and 2005, respectively.
From 2005 to 2006, he was a Postdoctoral Associate at the Department of Biomedical Engineering, University of Min...Show More
Chang-Hwan Im (Member, IEEE) received the B.S. degree in electrical engineering from the School of Electrical Engineering, Seoul National University, Seoul, South Korea, in 1999, and the M.S. and Ph.D. degrees in electrical and computer engineering from Seoul National University, in 2001 and 2005, respectively.
From 2005 to 2006, he was a Postdoctoral Associate at the Department of Biomedical Engineering, University of Min...View more
Contents Department of Electronic Engineering, Hanyang University, Seoul, South Korea
Suhye Kim (Graduate Student Member, IEEE) received the B.S. degree in psychology from the Department of Psychology, Chungbuk National University, Cheongju, South Korea, in 2020. She is currently working toward the Ph.D. degree in electronic engineering with the Department of Electronic Engineering, Hanyang University, Seoul, South Korea.
Her research interests include brain and human–computer interface, affective computing, neurofeedback, cognition, and emotion.
Suhye Kim (Graduate Student Member, IEEE) received the B.S. degree in psychology from the Department of Psychology, Chungbuk National University, Cheongju, South Korea, in 2020. She is currently working toward the Ph.D. degree in electronic engineering with the Department of Electronic Engineering, Hanyang University, Seoul, South Korea.
Her research interests include brain and human–computer interface, affective computing, neurofeedback, cognition, and emotion.View more
Department of Artificial Intelligence, Hanyang University, Seoul, South Korea
Jung-Hwan Kim (Graduate Student Member, IEEE) received the B.S. degree in electrical engineering and the M.S. degree in electronic engineering from Hanyang University, Seoul, South Korea, in 2019 and 2021, respectively, where he is currently working toward the Ph.D. degree in artificial intelligence with the Department of Artificial Intelligence.
His research interests include brain–computer interfaces, biomedical signal processing, and artificial intelligence.
Jung-Hwan Kim (Graduate Student Member, IEEE) received the B.S. degree in electrical engineering and the M.S. degree in electronic engineering from Hanyang University, Seoul, South Korea, in 2019 and 2021, respectively, where he is currently working toward the Ph.D. degree in artificial intelligence with the Department of Artificial Intelligence.
His research interests include brain–computer interfaces, biomedical signal processing, and artificial intelligence.View more
Department of Artificial Intelligence, Hanyang University, Seoul, South Korea
Wooseok Hyung (Graduate Student Member, IEEE) received the B.S. degree in digital imaging engineering from the School of Integrative Engineering, Chung-Ang University, Seoul, South Korea, in 2022. He is currently working toward the M.S. degree in artificial intelligence with the Department of Artificial Intelligence, Hanyang University, Seoul.
His research interests include brain and human–computer interface and biomedical signal processing based on artificial intelligence.
Wooseok Hyung (Graduate Student Member, IEEE) received the B.S. degree in digital imaging engineering from the School of Integrative Engineering, Chung-Ang University, Seoul, South Korea, in 2022. He is currently working toward the M.S. degree in artificial intelligence with the Department of Artificial Intelligence, Hanyang University, Seoul.
His research interests include brain and human–computer interface and biomedical signal processing based on artificial intelligence.View more
Department of Education, Hanyang University, Seoul, South Korea
Suhkyung Shin received the Ph.D. degree in instructional systems technology from Indiana University, Bloomington, IN, USA, in 2016.
From 2016 to 2018, she was a Postdoctoral Research Associate of Instructional Technology at the College of Education, Texas Tech University, Lubbock, TX, USA. She is currently an Assistant Professor with the School of Education, Hanyang University, Seoul, South Korea. Her expertise lies in the use of instructional technologies to enhance learning performances and establish learning communities. Her research interests include designing effective learning environments and fostering problem-solving processes in various learning contexts.
Suhkyung Shin received the Ph.D. degree in instructional systems technology from Indiana University, Bloomington, IN, USA, in 2016.
From 2016 to 2018, she was a Postdoctoral Research Associate of Instructional Technology at the College of Education, Texas Tech University, Lubbock, TX, USA. She is currently an Assistant Professor with the School of Education, Hanyang University, Seoul, South Korea. Her expertise lies in the use of instructional technologies to enhance learning performances and establish learning communities. Her research interests include designing effective learning environments and fostering problem-solving processes in various learning contexts.View more
Samsung SDS, Seoul, South Korea
Myoung Jin Choi received the B.A. degree in economics from Underwood International College, Yonsei University, Seoul, South Korea, in 2014.
She was a Software Engineer with the Samsung SDS, Seoul, South Korea. From 2021 to 2022, she was a Manager of R&D for an EdTech corporate venture at the goal of assessing students’ attention/focus levels using vision artificial intelligence in the context of online learning. She is also a Technical Trainer for the Cloud Native training program with Samsung SDS.
Myoung Jin Choi received the B.A. degree in economics from Underwood International College, Yonsei University, Seoul, South Korea, in 2014.
She was a Software Engineer with the Samsung SDS, Seoul, South Korea. From 2021 to 2022, she was a Manager of R&D for an EdTech corporate venture at the goal of assessing students’ attention/focus levels using vision artificial intelligence in the context of online learning. She is also a Technical Trainer for the Cloud Native training program with Samsung SDS.View more
Center for Bionics, Biomedical Research Institute, Korea Institute of Science and Technology, Seoul, South Korea
Dong Hwan Kim (Member, IEEE) received the B.S. degree in electrical engineering and the M.S. and Ph.D. degrees in electrical engineering and computer science from Seoul National University, Seoul, South Korea, in 1999, 2001, and 2006, respectively.
From 2006 to 2007, he was a Senior Engineer at Samsung Electronics, Company Ltd., Suwon, South Korea. In 2007, he joined Korea Institute of Science and Technology (KIST), Seoul, Korea, and from 2007 to 2010, he was a visiting research associate at the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA. He is currently a principal researcher with the Center for Intelligent Robotics, KIST. His current research interests include computer vision, pattern recognition, machine learning, image processing, and their applications including object recognition, image segmentation, visual grasp affordance detection, and 3-D reconstruction.
Dong Hwan Kim (Member, IEEE) received the B.S. degree in electrical engineering and the M.S. and Ph.D. degrees in electrical engineering and computer science from Seoul National University, Seoul, South Korea, in 1999, 2001, and 2006, respectively.
From 2006 to 2007, he was a Senior Engineer at Samsung Electronics, Company Ltd., Suwon, South Korea. In 2007, he joined Korea Institute of Science and Technology (KIST), Seoul, Korea, and from 2007 to 2010, he was a visiting research associate at the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA. He is currently a principal researcher with the Center for Intelligent Robotics, KIST. His current research interests include computer vision, pattern recognition, machine learning, image processing, and their applications including object recognition, image segmentation, visual grasp affordance detection, and 3-D reconstruction.View more
Department of Biomedical Engineering, HY-KIST Bioconvergence, and the Department of Electronic Engineering, Hanyang University, Seoul, South Korea
Chang-Hwan Im (Member, IEEE) received the B.S. degree in electrical engineering from the School of Electrical Engineering, Seoul National University, Seoul, South Korea, in 1999, and the M.S. and Ph.D. degrees in electrical and computer engineering from Seoul National University, in 2001 and 2005, respectively.
From 2005 to 2006, he was a Postdoctoral Associate at the Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA. From 2006 to 2011, he was an Assistant/Associate Professor at the Department of Biomedical Engineering, Yonsei University, Seoul. Since 2011, he has been an Associate/Full Professor with the Department of Biomedical Engineering, Hanyang University, Seoul. He is currently the Director of the Computational Intelligence and Neural Engineering Laboratory, Hanyang University. He authored more than 200 articles in peer-reviewed international journals. His research interests include various fields of computational neuroengineering, especially brain–computer interfaces, diagnosis of neuropsychiatric diseases, noninvasive brain stimulation, and bioelectromagnetic source imaging.
Chang-Hwan Im (Member, IEEE) received the B.S. degree in electrical engineering from the School of Electrical Engineering, Seoul National University, Seoul, South Korea, in 1999, and the M.S. and Ph.D. degrees in electrical and computer engineering from Seoul National University, in 2001 and 2005, respectively.
From 2005 to 2006, he was a Postdoctoral Associate at the Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA. From 2006 to 2011, he was an Assistant/Associate Professor at the Department of Biomedical Engineering, Yonsei University, Seoul. Since 2011, he has been an Associate/Full Professor with the Department of Biomedical Engineering, Hanyang University, Seoul. He is currently the Director of the Computational Intelligence and Neural Engineering Laboratory, Hanyang University. He authored more than 200 articles in peer-reviewed international journals. His research interests include various fields of computational neuroengineering, especially brain–computer interfaces, diagnosis of neuropsychiatric diseases, noninvasive brain stimulation, and bioelectromagnetic source imaging.View more
