ERIC Number: EJ1366553
Record Type: Journal
Publication Date: 2023-Feb
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-1939-1382
Semantic Navigation of PowerPoint-Based Lecture Video for AutoNote Generation
IEEE Transactions on Learning Technologies, v16 n1 p1-17 Feb 2023
With the increasing popularity of open educational resources in the past few decades, more and more users watch online videos to gain knowledge. However, most educational videos only provide monotonous navigation tools and lack elaborating annotations. This makes the task of locating interesting contents time consuming. To address this limitation, in this article, we propose a slide-based video navigation tool that is able to extract the hierarchical structure and semantic relationship of visual entities in videos, by integrating multichannel information. Features of visual entities are first extracted from the presentation slides by a novel deep learning framework. Then, we propose a clustering approach to extract hierarchical relationships between visual entities (e.g., formulas, texts, or graphs appearing in educational slides). We use this information to associate visual entities with their corresponding audio speech text, by evaluating their semantic relationship. We present two cases where we use the structured data produced by this tool to generate a multilevel table of contents and notes to provide additional navigation materials for learning. The evaluation experiments demonstrate the effectiveness of our proposed solutions for visual entity extraction, hierarchical relationship extraction, as well as corresponding speech text matching. The user study also shows promising improvement in the autogenerated table of contents and notes for facilitating learning.
Descriptors: Open Educational Resources, Video Technology, Instructional Films, Navigation (Information Systems), Semantics, Visual Aids, Automation, Artificial Intelligence
Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://bibliotheek.ehb.be:2578/xpl/RecentIssue.jsp?punumber=4620076
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A