Panoptic Studio: A Massively Multiview System for Social Motion Capture

We present an approach to capture the 3D structure and motion of a group of people engaged in a social interaction. The core challenges in capturing social interactions are: (1) occlusion is functional and frequent, (2) subtle motion needs to be measured over a space large enough to host a social gr...

Full description

Saved in:
Bibliographic Details
Main Authors: Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Nabbe, Bart, Matthews, Iain, Kanade, Takeo, Nobuhara, Shohei, Sheikh, Yaser
Format: Conference Proceeding
Language:English
Subjects:
Algorithms
Cameras
Computer vision
Face
Proposals
Reconstruction
Sensors
Social aspects
Studios
Synchronization
Three dimensional
Three-dimensional displays
Trajectories
Trajectory
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present an approach to capture the 3D structure and motion of a group of people engaged in a social interaction. The core challenges in capturing social interactions are: (1) occlusion is functional and frequent, (2) subtle motion needs to be measured over a space large enough to host a social group, and (3) human appearance and configuration variation is immense. The Panoptic Studio is a system organized around the thesis that social interactions should be measured through the perceptual integration of a large variety of view points. We present a modularized system designed around this principle, consisting of integrated structural, hardware, and software innovations. The system takes, as input, 480 synchronized video streams of multiple people engaged in social activities, and produces, as output, the labeled time-varying 3D structure of anatomical landmarks on individuals in the space. The algorithmic contributions include a hierarchical approach for generating skeletal trajectory proposals, and an optimization framework for skeletal reconstruction with trajectory re-association.
ISSN:2380-7504
DOI:10.1109/ICCV.2015.381