Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera

Extracting high-quality dynamic foreground layers from a video sequence is a challenging problem due to the coupling of color, motion, and occlusion. Many approaches assume that the background scene is static or undergoes the planar perspective transformation. In this paper, we relax these restricti...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2011-03, Vol.33 (3), p.603-617
Main Authors: Zhang, Guofeng, Jia, Jiaya, Hua, Wei, Bao, Hujun
Format: Article
Language:English
Subjects:
Algorithms
Applied sciences
Artificial Intelligence
Bilayer segmentation
Cameras
Color
Computer science
control theory
systems
Computer Simulation
Data mining
depth recovery
Dynamical systems
Dynamics
Exact sciences and technology
Fractals
Image Enhancement - methods
Image Interpretation, Computer-Assisted - instrumentation
Image motion analysis
Image segmentation
Information Storage and Retrieval - methods
Intelligence
Layout
Materials handling
Motion
Motion estimation
Movement
Optical computing
Pattern analysis
Pattern Recognition, Automated - methods
Pattern recognition. Digital image processing. Computational geometry
Phantoms, Imaging
Reproducibility of Results
Robustness
Segmentation
Signal Processing, Computer-Assisted - instrumentation
State estimation
Subtraction Technique - instrumentation
video editing
Video Recording - instrumentation
Video sequences
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Extracting high-quality dynamic foreground layers from a video sequence is a challenging problem due to the coupling of color, motion, and occlusion. Many approaches assume that the background scene is static or undergoes the planar perspective transformation. In this paper, we relax these restrictions and present a comprehensive system for accurately computing object motion, layer, and depth information. A novel algorithm that combines different clues to extract the foreground layer is proposed, where a voting-like scheme robust to outliers is employed in optimization. The system is capable of handling difficult examples in which the background is nonplanar and the camera freely moves during video capturing. Our work finds several applications, such as high-quality view interpolation and video editing.
ISSN:0162-8828
1939-3539
DOI:10.1109/TPAMI.2010.115