Markerless Motion Capture for Humans Through a Multi-UAV System

• Published in: IEEE transactions on instrumentation and measurement 2024, Vol.73, p.1-12
• Main Authors: Zhuge, Sheng, He, Yuwei, Xu, Xiangpeng, Gan, Shuwei, Li, Chunjun, Lin, Bin, Yang, Xia, Zhang, Xiaohu
• Format: Article
• Language: English
• Subjects: 3σ rule based on reprojection error; airborne cameras pose estimation; Algorithms; Autonomous aerial vehicles; Cameras; Error analysis; markerless motion capture; Motion capture; multiunmanned aerial vehicle (UAV) system; Pose estimation; sparse point cloud reconstruction; Unmanned aerial vehicles
• ISSN: 0018-9456; 1557-9662
• DOI: 10.1109/TIM.2023.3338658

Capturing the motion of humans without any markers in a moving scene is a challenging issue. Typical methods, such as those based on fixed cameras or wearable sensors, have limits on the measurement range and the number of targets. Therefore, we propose a markerless motion-capture technique to estimate the 3-D pose of humans through a multiunmanned aerial vehicle (UAV) system. By utilizing the advantage of the large observation baselines between UAVs in the system, an algorithm to estimate the pose of airborne cameras and establish the sparse points cloud of background in overlapping view simultaneously was presented. A 3\sigma rule based on reprojection error is employed to eliminate the incorrect conjugate keypoint pairs, enhancing the precision of pose estimation for airborne cameras. To verify the accuracy of the developed technique, an indoor test was conducted to compare it with a commercial capture system, the results show that the average measurement error of the 3-D joints of humans is less than 30 mm. Furthermore, we demonstrate the feasibility of our approach in outdoor scenes, including multiperson scenarios and moving scenarios.