Motion clustering with hybrid-sample-based foreground segmentation for moving cameras

• Main Authors: Yi-Chan Wu, Ching-Te Chiu
• Format: Conference Proceeding
• Language: English
• Subjects: Adaptation models; Adaptive feedback; Adaptive optics; Cameras; Foreground segmentation; Mathematical model; Motion Clustering; Motion segmentation; Moving camera; Optical distortion; Optical flow; Sensitivity
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP.2017.7952346

Foreground segmentation/background subtraction is a vital step in many high-level video analysis applications. While many methods have been proposed for foreground segmentation, most assume the cameras to be stationary. With this assumption, they are unable to handle the movements caused by camera rotation. In this paper, we propose a robust hybrid-sample-based foreground segmentation method for moving cameras, and especially for pan-tilt-zoom cameras. First, we propose the use of motion clustering registration to reduce the impact of registration errors. Next, we propose a frame-level reinitialization scheme to solve the problem of sudden large movement between consecutive frames. Third, we adopt a hybrid-sample-based background modeling technique to easily detect camouflaged foreground objects. Lastly, in order to deal with dynamic backgrounds, we propose moving scene pixel-level feedback schemes to dynamically and locally control the sensitivity and adaptation speed of the background model. We evaluate the proposed method using the ChangeDetection.NET 2014 dataset. Experimental results show that our proposed motion clustering registration can eliminate most of the noise caused by registration errors. The proposed reinitialization scheme can handle the noises caused by sudden large movements. The proposed method performs at least 8% better than other state-of-the-art algorithms in terms of the F-score in the pan-tilt-zoom camera scenario, and it also achieves the highest F-score in camera jitter scenarios.