Monocular, One-stage, Regression of Multiple 3D People

• Main Authors: Sun, Yu, Bao, Qian, Liu, Wu, Fu, Yili, Black, Michael J., Mei, Tao
• Format: Conference Proceeding
• Language: English
• Subjects: Benchmark testing; Estimation; Gestures and body pose; Heating systems; Pipelines; Real-time systems; Solid modeling; Three-dimensional displays
• ISSN: 2380-7504
• DOI: 10.1109/ICCV48922.2021.01099

This paper focuses on the regression of multiple 3D people from a single RGB image. Existing approaches predominantly follow a multi-stage pipeline that first detects people in bounding boxes and then independently regresses their 3D body meshes. In contrast, we propose to Regress all meshes in a One-stage fashion for Multiple 3D People (termed ROMP). The approach is conceptually simple, bounding box-free, and able to learn a per-pixel representation in an end-to-end manner. Our method simultaneously predicts a Body Center heatmap and a Mesh Parameter map, which can jointly describe the 3D body mesh on the pixel level. Through a body-center-guided sampling process, the body mesh parameters of all people in the image are easily extracted from the Mesh Parameter map. Equipped with such a fine-grained representation, our one-stage framework is free of the complex multi-stage process and more robust to occlusion. Compared with state-of-the-art methods, ROMP achieves superior performance on the challenging multi-person benchmarks, including 3DPW and CMU Panoptic. Experiments on crowded/occluded datasets demonstrate the robustness under various types of occlusion. The code, released at https://github.com/Arthur151/ROMP, is the first real-time implementation of monocular multi-person 3D mesh regression.