RIMOC, a feature to discriminate unstructured motions: Application to violence detection for video-surveillance

• Published in: Computer vision and image understanding 2016-03, Vol.144, p.121-143
• Main Authors: Ribeiro, Pedro Canotilho, Audigier, Romaric, Pham, Quoc Cuong
• Format: Article
• Language: English
• Subjects: Abnormality detection; Aggression detection; Computer Science; Signal and Image Processing; Unstructured motion; Video-surveillance; Violence detection; Violent event detection
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2015.11.001

•A novel and compact feature discriminating structuredness of observed motions.•A feature embedded in a weakly supervised learning framework.•An efficient method for real-time violence detection in on-board video-surveillance.•Ability of the learned model to generalize training data for varied contexts.•A new dataset representative of the targeted application for extensive evaluation. In video-surveillance, violent event detection is of utmost interest. Although action recognition has been well studied in computer vision, literature for violence detection in video is far sparser, and even more for surveillance applications. As aggressive events are difficult to define due to their variability and often need high-level interpretation, we decided to first try to characterize what is frequently present in video with violent human behaviors, at a low level: jerky and unstructured motion. Thus, a novel problem-specific Rotation-Invariant feature modeling MOtion Coherence (RIMOC) was proposed, in order to capture its structure and discriminate the unstructured motions. It is based on the eigenvalues obtained from the second-order statistics of the Histograms of Optical Flow vectors from consecutive temporal instants, locally and densely computed, and further embedded into a spheric Riemannian manifold. The proposed RIMOC feature is used to learn statistical models of normal coherent motions in a weakly supervised manner. A multi-scale scheme applied on an inference-based method allows the events with erratic motion to be detected in space and time, as good candidates of aggressive events. We experimentally show that the proposed method produces results comparable to a state-of-the-art supervised approach, with added simplicity in training and computation. Thanks to the compactness of the feature, real-time computation is achieved in learning as well as in detection phase. Extensive experimental tests on more than 18 h of video are provided in different in-lab and real contexts, such as railway cars equipped with on-board cameras.