Instantaneous threat detection based on a semantic representation of activities, zones and trajectories

• Published in: Signal, image and video processing image and video processing, 2014-12, Vol.8 (Suppl 1), p.191-200
• Main Authors: Burghouts, G. J., Schutte, K., ten Hove, R. J.-M., van den Broek, S. P., Baan, J., Rajadell, O., van Huis, J. R., van Rest, J., Hanckmann, P., Bouma, H., Sanroma, G., Evans, M., Ferryman, J.
• Format: Article
• Language: English
• Subjects: Computer Imaging; Computer Science; Image Processing and Computer Vision; Multimedia Information Systems; Original Paper; Pattern Recognition and Graphics; Signal,Image and Speech Processing; Vision
• ISSN: 1863-1703; 1863-1711
• DOI: 10.1007/s11760-014-0672-1

Threat detection is a challenging problem, because threats appear in many variations and differences to normal behaviour can be very subtle. In this paper, we consider threats on a parking lot, where theft of a truck’s cargo occurs. The threats range from explicit, e.g. a person attacking the truck driver, to implicit, e.g. somebody loitering and then fiddling with the exterior of the truck in order to open it. Our goal is a system that is able to recognize a threat instantaneously as they develop. Typical observables of the threats are a person’s activity, presence in a particular zone and the trajectory. The novelty of this paper is an encoding of these threat observables in a semantic, intermediate-level representation, based on low-level visual features that have no intrinsic semantic meaning themselves. The aim of this representation was to bridge the semantic gap between the low-level tracks and motion and the higher-level notion of threats. In our experiments, we demonstrate that our semantic representation is more descriptive for threat detection than directly using low-level features. We find that a person’s activities are the most important elements of this semantic representation, followed by the person’s trajectory. The proposed threat detection system is very accurate: 96.6 % of the tracks are correctly interpreted, when considering the temporal context.