Interp-SUM: Unsupervised Video Summarization with Piecewise Linear Interpolation

• Published in: Sensors (Basel, Switzerland) Switzerland), 2021-07, Vol.21 (13), p.4562
• Main Authors: Yoon, Ui-Nyoung, Hong, Myung-Duk, Jo, Geun-Sik
• Format: Article
• Language: English
• Subjects: piecewise linear interpolation; reinforcement learning; unsupervised learning; video summarization
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s21134562

This paper addresses the problem of unsupervised video summarization. Video summarization helps people browse large-scale videos easily with a summary from the selected frames of the video. In this paper, we propose an unsupervised video summarization method with piecewise linear interpolation (Interp-SUM). Our method aims to improve summarization performance and generate a natural sequence of keyframes with predicting importance scores of each frame utilizing the interpolation method. To train the video summarization network, we exploit a reinforcement learning-based framework with an explicit reward function. We employ the objective function of the exploring under-appreciated reward method for training efficiently. In addition, we present a modified reconstruction loss to promote the representativeness of the summary. We evaluate the proposed method on two datasets, SumMe and TVSum. The experimental result showed that Interp-SUM generates the most natural sequence of summary frames than any other the state-of-the-art methods. In addition, Interp-SUM still showed comparable performance with the state-of-art research on unsupervised video summarization methods, which is shown and analyzed in the experiments of this paper.