Monocular 3D Pose Estimation via Pose Grammar and Data Augmentation

In this paper, we propose a pose grammar to tackle the problem of 3D human pose estimation from a monocular RGB image. Our model takes estimated 2D pose as the input and learns a generalized 2D-3D mapping function to leverage into 3D pose. The proposed model consists of a base network which efficien...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2022-10, Vol.44 (10), p.6327-6344
Main Authors: Xu, Yuanlu, Wang, Wenguan, Liu, Tengyu, Liu, Xiaobai, Xie, Jianwen, Zhu, Song-Chun
Format: Article
Language:English
Subjects:
3D pose estimation
Algorithms
Cameras
Constraint modelling
Data augmentation
deep neural network
dependency grammar
evaluation protocol
Grammar
Kinematics
learning-by-synthesis
Machine learning
Pose estimation
Protocols
recurrent neural network
Solid modeling
Three-dimensional displays
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we propose a pose grammar to tackle the problem of 3D human pose estimation from a monocular RGB image. Our model takes estimated 2D pose as the input and learns a generalized 2D-3D mapping function to leverage into 3D pose. The proposed model consists of a base network which efficiently captures pose-aligned features and a hierarchy of Bi-directional RNNs (BRNNs) on the top to explicitly incorporate a set of knowledge regarding human body configuration (i.e., kinematics, symmetry, motor coordination). The proposed model thus enforces high-level constraints over human poses. In learning, we develop a data augmentation algorithm to further improve model robustness against appearance variations and cross-view generalization ability. We validate our method on public 3D human pose benchmarks and propose a new evaluation protocol working on cross-view setting to verify the generalization capability of different methods. We empirically observe that most state-of-the-art methods encounter difficulty under such setting while our method can well handle such challenges.
ISSN:0162-8828
2160-9292
1939-3539
DOI:10.1109/TPAMI.2021.3087695