A Novel Shot Detection Approach Based on ORB Fused With Structural Similarity

Shots are the basic units for analyzing and retrieving video, and also the essential elements in creating video datasets. The traditional methods of shot detection exhibit unsatisfactory performance for being too sensitive to motion or too much time-consuming. This paper proposes an automatic shot d...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2020, Vol.8, p.2472-2481
Main Authors: Liu, Huibin, Tan, Tan-Hsu, Kuo, Tien-Ying
Format: Article
Language:English
Subjects:
cut transition
Datasets
Feature extraction
Frames (data processing)
gradual transition
ORB descriptor
Segments
Shot detection
Similarity
SSIM
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Shots are the basic units for analyzing and retrieving video, and also the essential elements in creating video datasets. The traditional methods of shot detection exhibit unsatisfactory performance for being too sensitive to motion or too much time-consuming. This paper proposes an automatic shot detection method, by employing the fast feature descriptor of Oriented FAST and Rotated BRIEF (ORB) fused with Structural Similarity (SSIM). Firstly, ORB descriptor is used to preselect candidate segments with a high tolerance for rapidly extracting the features of twenty-frame intervals in video sequences. Then, the cut transition is detected by comparing ORB features, fused with SSIM, of consecutive frames in the candidate segment. Finally, the gradual transition is detected by determining the maximum amount of the continuous increasing/decreasing interframe differences in the candidate segment without cut transition. Experimental result indicates that the proposed method can achieve an F1-Score of 92.5% and five times of real-time speed with one CPU on 106049 test frames from the Open-video project, YouTube, and YOUKU. In addition, the proposed method can outperform the existing shot detection methods, including the rule-based and learning-based methods, by testing on the video sequences from the Open-video project and RAI dataset.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2019.2962328