Markerless motion capture of interacting characters using multi-view image segmentation

We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of two closely interacting people from multi-view video. Due to ambiguities in feature-to-person assignments and frequent occlusions, it is not feasible to directly apply...

Full description

Saved in:
Bibliographic Details
Main Authors: Yebin Liu, Stoll, C., Gall, J., Seidel, H-P, Theobalt, C.
Format: Conference Proceeding
Language:English
Subjects:
Image color analysis
Image segmentation
Motion segmentation
Shape
Skeleton
Three dimensional displays
Tracking
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of two closely interacting people from multi-view video. Due to ambiguities in feature-to-person assignments and frequent occlusions, it is not feasible to directly apply single-person capture approaches to the multi-person case. We therefore propose a combined image segmentation and tracking approach to overcome these difficulties. A new probabilistic shape and appearance model is employed to segment the input images and to assign each pixel uniquely to one person. Thereafter, a single-person markerless motion and surface capture approach can be applied to each individual, either one-by-one or in parallel, even under strong occlusions. We demonstrate the performance of our approach on several challenging multi-person motions, including dance and martial arts, and also provide a reference dataset for multi-person motion capture with ground truth.
ISSN:1063-6919
DOI:10.1109/CVPR.2011.5995424