Statistical modeling of complex backgrounds for foreground object detection

• Published in: IEEE transactions on image processing 2004-11, Vol.13 (11), p.1459-1472
• Main Authors: Li, Liyuan, Huang, Weimin, Gu, Irene Yu-Hua, Tian, Qi
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Bayesian analysis; Bayesian methods; Change detection; Classification; Cluster Analysis; Computer Graphics; Computer Simulation; Computer vision; Detection, estimation, filtering, equalization, prediction; Exact sciences and technology; Feature extraction; Humans; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image processing; Image sequences; Information Storage and Retrieval - methods; Information, signal and communications theory; Layout; Learning; Learning systems; Mathematical models; Models, Biological; Models, Statistical; Motion analysis; Numerical Analysis, Computer-Assisted; Object detection; Pattern recognition; Pattern Recognition, Automated - methods; Pixel; Pixels; Reproducibility of Results; Sensitivity and Specificity; Signal and communications theory; Signal processing; Signal Processing, Computer-Assisted; Signal, noise; Statistics; Studies; Subtraction Technique; Telecommunications and information theory
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2004.836169

This paper addresses the problem of background modeling for foreground object detection in complex environments. A Bayesian framework that incorporates spectral, spatial, and temporal features to characterize the background appearance is proposed. Under this framework, the background is represented by the most significant and frequent features, i.e., the principal features , at each pixel. A Bayes decision rule is derived for background and foreground classification based on the statistics of principal features. Principal feature representation for both the static and dynamic background pixels is investigated. A novel learning method is proposed to adapt to both gradual and sudden "once-off" background changes. The convergence of the learning process is analyzed and a formula to select a proper learning rate is derived. Under the proposed framework, a novel algorithm for detecting foreground objects from complex environments is then established. It consists of change detection, change classification, foreground segmentation, and background maintenance. Experiments were conducted on image sequences containing targets of interest in a variety of environments, e.g., offices, public buildings, subway stations, campuses, parking lots, airports, and sidewalks. Good results of foreground detection were obtained. Quantitative evaluation and comparison with the existing method show that the proposed method provides much improved results.