Anomaly detection with a moving camera using spatio-temporal codebooks

• Published in: Multidimensional systems and signal processing 2018-07, Vol.29 (3), p.1025-1054
• Main Authors: Nakahata, Mateus T., Thomaz, Lucas A., da Silva, Allan F., da Silva, Eduardo A. B., Netto, Sergio L.
• Format: Article
• Language: English
• Subjects: Anomalies; Artificial Intelligence; Cameras; Circuits and Systems; Electrical Engineering; Engineering; Feature extraction; Filtration; Inspection; Methods; Motion simulation; Regularization; Signal,Image and Speech Processing; Subtraction; Surveillance
• ISSN: 0923-6082; 1573-0824
• DOI: 10.1007/s11045-017-0486-8

This paper proposes a method to detect anomalies in videos acquired by a camera mounted on a moving inspection robot. The proposed method is based on a spatio-temporal composition (STC) method, where a dense sampling is used to break the video into small 3D volumes that are used to calculate the probability of the spatio-temporal arrangements. This class of methods has been successfully used for surveillance videos obtained by static cameras. However, when applied to videos recorded by cameras on moving platforms, the STC gives a large number of false detections. In this work, we propose improvements to the present STC method that will alleviate this problem in two ways. First, a two-stage dictionary learning process is performed to allow a more reliable anomaly detection. Second, improved spatio-temporal features are employed. These modified features are extracted after an enhanced temporal filtering that performs a temporal regularization of the video sequence. The proposed approach gives very good results in the identification of anomalies without the need of background subtraction, motion estimation or tracking. The results are shown to be comparable or even superior to those of other state-of-the-art methods using bag-of-video words or other moving-camera surveillance methods. The system is accurate even with no prior knowledge of the type of event to be observed, being robust to cluttered environments, as illustrated by several practical examples. These results are obtained without compromising the performance of the algorithm in the static cameras case.