MeshLifter: Weakly Supervised Approach for 3D Human Mesh Reconstruction from a Single 2D Pose Based on Loop Structure

• Published in: Sensors (Basel, Switzerland) Switzerland), 2020-07, Vol.20 (15), p.4257
• Main Authors: Jeong, Sunwon, Chang, Ju Yong
• Format: Article
• Language: English
• Subjects: 3D human mesh reconstruction; 3D human pose estimation; deep neural network; Humans; Imaging, Three-Dimensional; Letter; Motion; Surgical Mesh; weakly supervised learning
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s20154257

In this paper, we address the problem of 3D human mesh reconstruction from a single 2D human pose based on deep learning. We propose MeshLifter, a network that estimates a 3D human mesh from an input 2D human pose. Unlike most existing 3D human mesh reconstruction studies that train models using paired 2D and 3D data, we propose a weakly supervised learning method based on a loop structure to train the MeshLifter. The proposed method alleviates the difficulty of obtaining ground-truth 3D data to ensure that the MeshLifter can be trained successfully from a 2D human pose dataset and an unpaired 3D motion capture dataset. We compare the proposed method with recent state-of-the-art studies through various experiments and show that the proposed method achieves effective 3D human mesh reconstruction performance. Notably, our proposed method achieves a reconstruction error of 59.1 mm without using the 3D ground-truth data of Human3.6M, the standard dataset for 3D human mesh reconstruction.