Towards Single 2D Image-Level Self-Supervision for 3D Human Pose and Shape Estimation

Three-dimensional human pose and shape estimation is an important problem in the computer vision community, with numerous applications such as augmented reality, virtual reality, human computer interaction, and so on. However, training accurate 3D human pose and shape estimators based on deep learni...

Full description

Saved in:
Bibliographic Details
Published in:Applied sciences 2021-10, Vol.11 (20), p.9724
Main Authors: Cha, Junuk, Saqlain, Muhammad, Lee, Changhwa, Lee, Seongyeong, Lee, Seungeun, Kim, Donguk, Park, Won-Hee, Baek, Seungryul
Format: Article
Language:English
Subjects:
Augmented reality
Computer applications
Computer vision
Datasets
Deep learning
Human body
human body mesh estimation
human body pose estimation
Image manipulation
Methods
neural rendering
Optimization
self-supervised learning
Semantics
Supervision
Virtual reality
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Three-dimensional human pose and shape estimation is an important problem in the computer vision community, with numerous applications such as augmented reality, virtual reality, human computer interaction, and so on. However, training accurate 3D human pose and shape estimators based on deep learning approaches requires a large number of images and corresponding 3D ground-truth pose pairs, which are costly to collect. To relieve this constraint, various types of weakly or self-supervised pose estimation approaches have been proposed. Nevertheless, these methods still involve supervision signals, which require effort to collect, such as unpaired large-scale 3D ground truth data, a small subset of 3D labeled data, video priors, and so on. Often, they require installing equipment such as a calibrated multi-camera system to acquire strong multi-view priors. In this paper, we propose a self-supervised learning framework for 3D human pose and shape estimation that does not require other forms of supervision signals while using only single 2D images. Our framework inputs single 2D images, estimates human 3D meshes in the intermediate layers, and is trained to solve four types of self-supervision tasks (i.e., three image manipulation tasks and one neural rendering task) whose ground-truths are all based on the single 2D images themselves. Through experiments, we demonstrate the effectiveness of our approach on 3D human pose benchmark datasets (i.e., Human3.6M, 3DPW, and LSP), where we present the new state-of-the-art among weakly/self-supervised methods.
ISSN:2076-3417
2076-3417
DOI:10.3390/app11209724