An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions

• Published in: Computer vision and image understanding 2013-10, Vol.117 (10), p.1436-1452
• Main Authors: Javan Roshtkhari, Mehrsan, Levine, Martin D.
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Data processing. List processing. Character string processing; Estimates; Exact sciences and technology; Illumination; Inference; Memory organisation. Data processing; On-line anomaly detection; Pattern recognition. Digital image processing. Computational geometry; Real time; Software; Spatio-temporal compositions; Spatio-temporal video volumes; Suspicious event detection; Temporal logic; Tracking; Video surveillance
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2013.06.007

•A fast and accurate algorithm for anomalous event detection in videos.•Densely sampling video to construct spatio-temporal video volumes.•Handling uncertainties in codebook construction.•Coding spatio-temporal composition of volumes by a probabilistic framework.•Unsupervised and continuously learning new events. This paper presents an approach for detecting suspicious events in videos by using only the video itself as the training samples for valid behaviors. These salient events are obtained in real-time by detecting anomalous spatio-temporal regions in a densely sampled video. The method codes a video as a compact set of spatio-temporal volumes, while considering the uncertainty in the codebook construction. The spatio-temporal compositions of video volumes are modeled using a probabilistic framework, which calculates their likelihood of being normal in the video. This approach can be considered as an extension of the Bag of Video words (BOV) approaches, which represent a video as an order-less distribution of video volumes. The proposed method imposes spatial and temporal constraints on the video volumes so that an inference mechanism can estimate the probability density functions of their arrangements. Anomalous events are assumed to be video arrangements with very low frequency of occurrence. The algorithm is very fast and does not employ background subtraction, motion estimation or tracking. It is also robust to spatial and temporal scale changes, as well as some deformations. Experiments were performed on four video datasets of abnormal activities in both crowded and non-crowded scenes and under difficult illumination conditions. The proposed method outperformed all other approaches based on BOV that do not account for contextual information.