3D human pose and shape estimation via de-occlusion multi-task learning

• Published in: Neurocomputing (Amsterdam) 2023-09, Vol.548, p.126284, Article 126284
• Main Authors: Ran, Hang, Ning, Xin, Li, Weijun, Hao, Meilan, Tiwari, Prayag
• Format: Article
• Language: English
• Subjects: De-occlusion; Human mesh recovery; Multi-task learning; Occlusion-aware
• ISSN: 0925-2312; 1872-8286; 1872-8286
• DOI: 10.1016/j.neucom.2023.126284

Three-dimensional human pose and shape estimation is to compute a full human 3D mesh given a single image. The contamination of features caused by occlusion usually degrades its performance significantly. Recent progress in this field typically addressed the occlusion problem implicitly. By contrast, in this paper, we address it explicitly using a simple yet effective de-occlusion multi-task learning network. Our key insight is that feature for mesh parameter regression should be noiseless. Thus, in the feature space, our method disentangles the occludee that represents the noiseless human feature from the occluder. Specifically, a spatial regularization and an attention mechanism are imposed in the backbone of our network to disentangle the features into different channels. Furthermore, two segmentation tasks are proposed to supervise the de-occlusion process. The final mesh model is regressed by the disentangled occlusion-aware features. Experiments on both occlusion and non-occlusion datasets are conducted, and the results prove that our method is superior to the state-of-the-art methods on two occlusion datasets, while achieving competitive performance on a non-occlusion dataset. We also demonstrate that the proposed de-occlusion strategy is the main factor to improve the robustness against occlusion. The code is available athttps://github.com/qihangran/De-occlusion_MTL_HMR.