Monocular Visual Scene Understanding: Understanding Multi-Object Traffic Scenes

Following recent advances in detection, context modeling, and tracking, scene understanding has been the focus of renewed interest in computer vision research. This paper presents a novel probabilistic 3D scene model that integrates state-of-the-art multiclass object detection, object tracking and s...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2013-04, Vol.35 (4), p.882-897
Main Authors: Wojek, C., Walk, S., Roth, S., Schindler, K., Schiele, B.
Format: Article
Language:English
Subjects:
Algorithms
Applied sciences
Artificial intelligence
Automobiles
Cameras
Cluster Analysis
Cognition
Computational modeling
Computer science
control theory
systems
Databases, Factual
Detectors
Exact sciences and technology
Hidden Markov models
Human Activities
Humans
Image Processing, Computer-Assisted - methods
MCMC
Models, Theoretical
Object detection
Pattern Recognition, Automated - methods
Pattern recognition. Digital image processing. Computational geometry
scene tracklets
Scene understanding
Solid modeling
tracking
tracking-by-detection
Walking
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Following recent advances in detection, context modeling, and tracking, scene understanding has been the focus of renewed interest in computer vision research. This paper presents a novel probabilistic 3D scene model that integrates state-of-the-art multiclass object detection, object tracking and scene labeling together with geometric 3D reasoning. Our model is able to represent complex object interactions such as inter-object occlusion, physical exclusion between objects, and geometric context. Inference in this model allows us to jointly recover the 3D scene context and perform 3D multi-object tracking from a mobile observer, for objects of multiple categories, using only monocular video as input. Contrary to many other approaches, our system performs explicit occlusion reasoning and is therefore capable of tracking objects that are partially occluded for extended periods of time, or objects that have never been observed to their full extent. In addition, we show that a joint scene tracklet model for the evidence collected over multiple frames substantially improves performance. The approach is evaluated for different types of challenging onboard sequences. We first show a substantial improvement to the state of the art in 3D multipeople tracking. Moreover, a similar performance gain is achieved for multiclass 3D tracking of cars and trucks on a challenging dataset.
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2012.174