Image-Guided Human Reconstruction via Multi-Scale Graph Transformation Networks

3D human reconstruction from a single image is a challenging problem. Existing methods have difficulties to infer 3D clothed human models with consistent topologies for various poses. In this paper, we propose an efficient and effective method using a hierarchical graph transformation network. To de...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on image processing 2021-01, Vol.30, p.5239-5251
Main Authors: Li, Kun, Wen, Hao, Feng, Qiao, Zhang, Yuxiang, Li, Xiongzheng, Huang, Jing, Yuan, Cunkuan, Lai, Yu-Kun, Liu, Yebin
Format: Article
Language:English
Subjects:
3D human reconstruction
Color imagery
dataset
Datasets
Deformation
Deformation effects
deformation representation
Euclidean geometry
Finite element method
Geometry
Graph neural networks
Image reconstruction
Network topologies
Representations
Shape
single image
Solid modeling
Strain
Three dimensional models
Three-dimensional displays
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:3D human reconstruction from a single image is a challenging problem. Existing methods have difficulties to infer 3D clothed human models with consistent topologies for various poses. In this paper, we propose an efficient and effective method using a hierarchical graph transformation network. To deal with large deformations and avoid distorted geometries, rather than using Euclidean coordinates directly, 3D human shapes are represented by a vertex-based deformation representation that effectively encodes the deformation and copes well with large deformations. To infer a 3D human mesh consistent with the input real image, we also use a perspective projection layer to incorporate perceptual image features into the deformation representation. Our model is easy to train and fast to converge with short test time. Besides, we present the D^{2}Human (Dynamic Detailed Human) dataset, including variously posed 3D human meshes with consistent topologies and rich geometry details, together with the captured color images and SMPL models, which is useful for training and evaluation of deep frameworks, particularly for graph neural networks. Experimental results demonstrate that our method achieves more plausible and complete 3D human reconstruction from a single image, compared with several state-of-the-art methods. The code and dataset are available for research purposes at http://cic.tju.edu.cn/faculty/likun/projects/MGTnet .
ISSN:1057-7149
1941-0042
DOI:10.1109/TIP.2021.3080177