Moving foreground object detection via robust SIFT trajectories

• Published in: Journal of visual communication and image representation 2013-04, Vol.24 (3), p.232-243
• Main Authors: Sun, Shih-Wei, Wang, Yu-Chiang Frank, Huang, Fay, Liao, Hong-Yuan Mark
• Format: Article
• Language: English
• Subjects: Applied sciences; Artificial intelligence; Background subtraction; Cameras; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Construction; Exact sciences and technology; Foreground segmentation; Frames; Mathematical analysis; Object tracking; Pattern recognition. Digital image processing. Computational geometry; Software; Tasks; Template matching; Trajectories; Tuning; Video object segmentation
• ISSN: 1047-3203; 1095-9076
• DOI: 10.1016/j.jvcir.2012.12.003

► An automatic video foreground object detection approach is proposed. ► Can deal with videos captured by freely-moving cameras. ► A consensus foreground object template (CFOT) is proposed for object detection. ► We do not require prior knowledge (i.e., training data) on the objects of interest. ► No user interaction (e.g., annotation of the first few frames) is needed. In this paper, we present an automatic foreground object detection method for videos captured by freely moving cameras. While we focus on extracting a single foreground object of interest throughout a video sequence, our approach does not require any training data nor the interaction by the users. Based on the SIFT correspondence across video frames, we construct robust SIFT trajectories in terms of the calculated foreground feature point probability. Our foreground feature point probability is able to determine candidate foreground feature points in each frame, without the need of user interaction such as parameter or threshold tuning. Furthermore, we propose a probabilistic consensus foreground object template (CFOT), which is directly applied to the input video for moving object detection via template matching. Our CFOT can be used to detect the foreground object in videos captured by a fast moving camera, even if the contrast between the foreground and background regions is low. Moreover, our proposed method can be generalized to foreground object detection in dynamic backgrounds, and is robust to viewpoint changes across video frames. The contribution of this paper is trifold: (1) we provide a robust decision process to detect the foreground object of interest in videos with contrast and viewpoint variations; (2) our proposed method builds longer SIFT trajectories, and this is shown to be robust and effective for object detection tasks; and (3) the construction of our CFOT is not sensitive to the initial estimation of the foreground region of interest, while its use can achieve excellent foreground object detection results on real-world video data.