What Is Happening in the Video? -Annotate Video by Sentence

Due to the popularity of online video sharing Web sites such as YouTube, millions of users have treated online video as a source of information and entertainment. Therefore, video annotation has evoked great interest in the past few years. In this paper, we propose a four-step approach to automatica...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on circuits and systems for video technology 2016-09, Vol.26 (9), p.1746-1757
Main Authors: Qian, Xueming, Liu, Xiaoxiao, Ma, Xiang, Lu, Dan, Xu, Chenyang
Format: Article
Language:English
Subjects:
Algorithm
Algorithms
Artificial intelligence
Correlation
Correlation analysis
Datasets
Feature extraction
Graph algorithms
Image annotation
image data set
Semantics
sentence element
Sentences
Silicon
video annotation
Video data
Visualization
Websites
YouTube
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Due to the popularity of online video sharing Web sites such as YouTube, millions of users have treated online video as a source of information and entertainment. Therefore, video annotation has evoked great interest in the past few years. In this paper, we propose a four-step approach to automatically annotate video shots with sentences. The first step is video preprocessing, converting video shot into a sequence of frame images. The second step is to find related candidate elements of the sentence about the video contents. The main elements in the sentence are objects, events, scenes, and modifiers. These candidate elements are gained by searching for similar images with the video frames in our collected image data sets instead of video data sets. The third step is to select the best elements among these candidate ones by a weighted scoring algorithm. The final step is to construct a sentence with the help of a correlation graph algorithm to analyze the relationships among the best elements. The experimental results indicate that our method is effective to annotate videos with sentences. What is more, the weighted scoring algorithm and the correlation graph algorithm that we propose are efficient in developing the experimental performance.
ISSN:1051-8215
1558-2205
DOI:10.1109/TCSVT.2015.2475815