Key frame extraction method for lecture videos based on spatio-temporal subtitles

• Published in: Multimedia tools and applications 2024, Vol.83 (2), p.5437-5450
• Main Authors: Zhang, Yunzuo, Li, Yi, Cai, Zhaoquan, Wang, Xuejun, Zhang, Jiayu, Lam, Shui
• Format: Article
• Language: English
• Subjects: Algorithms; Computer Communication Networks; Computer Science; Coronaviruses; Data Structures and Information Theory; Edge detection; Frames (data processing); Multimedia Information Systems; Special Purpose and Application-Based Systems; Subtitles & subtitling; Video
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-023-15829-5

Affected by the Corona Virus Disease 2019 (COVID-19), online lecture videos have witnessed an explosive growth. In the face of massive videos, this paper proposes a method for extracting key frames of lecture videos based on spatio-temporal subtitles, which can efficiently and quickly obtain effective information. Firstly, the spatio-temporal slices of subtitle area of the video sequence are extracted and spliced along the time axis to construct the video spatio-temporal subtitle. Then, the video spatio-temporal subtitle is processed in binarization, and the projection method is used to construct the SSPA curve of the video spatio-temporal subtitle. Finally, a selection method for steady-state key frame is designed, that is, the key frame extraction is realized by combining curve edge detection and subtitle existence threshold, which ensures the robustness of the proposed method. The test results of 8 videos show that the average value of the comprehensive index F 1 -score of the key frame extracted by the algorithm can reach 0.97, the average precision is 0.97, and the average recall rate is 0.98. It can effectively extract the key frames in lecture videos, and compared with other algorithms, the average running time is reduced to 0.072 of the original, which is helpful to extract video information quickly and accurately.