Video lectures are considered as one of the primary media to cater good-quality educational content to the learners. The video lectures illustrate the course-relevant concepts with necessary details. However, they sometimes fail to offer a basic understanding of off-topic concepts. Such off-topic concepts may spawn cognitive overload among the learners if those concepts are not familiar to them. To address this issue, we present a video lecture augmentation system that identifies the off-topic concepts and links them to relevant video lecture segments to furnish a basic understanding of the concerned concepts. Our augmentation system segregated the video lectures by identifying topical shifts in the lectures using a word embedding-based technique. The video segments were indexed on the basis of the underlying concepts. Identification of off-topic concepts was performed by modeling inter-concept relations in a semantic space. For each off-topic concept, appropriate video segments were fetched and re-ranked such that the top-ranked video segment offers the most basic understanding of the target off-topic concept. The proposed augmentation system was deployed as a web-based learning platform. Performance of the constituent modules was measured by using a manually curated dataset consisting of six video courses from the National Programme on Technology Enhanced Learning (NPTEL) archive. Feedback from 12 research scholars was considered to assess the quality of augmentations and usability of the learning platform. Both system and human-based evaluation indicated that the recommended augmentations were able to offer a basic understanding of the concerned off-topic concepts.