Consistent Depth Maps Recovery from a Video Sequence

This paper presents a novel method for recovering consistent depth maps from a video sequence. We propose a bundle optimization framework to address the major difficulties in stereo reconstruction, such as dealing with image noise, occlusions, and outliers. Different from the typical multi-view ster...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2009-06, Vol.31 (6), p.974-988
Main Authors: GUOFENG ZHANG, JIAYA JIA, WONG, Tien-Tsin, HUJUN BAO
Format: Article
Language:English
Subjects:
Algorithms
Application software
Applied sciences
Artificial Intelligence
Bundling
Coherence
Computer science
control theory
systems
Computer vision
Depth cues
Exact sciences and technology
Image Enhancement - methods
Image Interpretation, Computer-Assisted - methods
Image reconstruction
Image segmentation
Imaging, Three-Dimensional - methods
Information resources
Mathematical models
Motion
Noise
Noise reduction
Optimization
Pattern Recognition, Automated - methods
Pattern recognition. Digital image processing. Computational geometry
Photography - methods
Reconstruction
Reproducibility of Results
Sensitivity and Specificity
Stereo
Stereo image processing
Stereo vision
Studies
Video Recording - methods
Video sequences
Visibility
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper presents a novel method for recovering consistent depth maps from a video sequence. We propose a bundle optimization framework to address the major difficulties in stereo reconstruction, such as dealing with image noise, occlusions, and outliers. Different from the typical multi-view stereo methods, our approach not only imposes the photo-consistency constraint, but also explicitly associates the geometric coherence with multiple frames in a statistical way. It thus can naturally maintain the temporal coherence of the recovered dense depth maps without over-smoothing. To make the inference tractable, we introduce an iterative optimization scheme by first initializing the disparity maps using a segmentation prior and then refining the disparities by means of bundle optimization. Instead of defining the visibility parameters, our method implicitly models the reconstruction noise as well as the probabilistic visibility. After bundle optimization, we introduce an efficient space-time fusion algorithm to further reduce the reconstruction noise. Our automatic depth recovery is evaluated using a variety of challenging video examples.
ISSN:0162-8828
1939-3539
DOI:10.1109/TPAMI.2009.52