Fusing Skeleton Recognition With Face-TLD for Human Following of Mobile Service Robots

• Published in: IEEE transactions on systems, man, and cybernetics. Systems man, and cybernetics. Systems, 2021-05, Vol.51 (5), p.2963-2979
• Main Authors: Yuan, Jing, Cai, Jingxin, Zhang, Xuebo, Sun, Qinxuan, Sun, Fengchi, Zhu, Wenbin
• Format: Article
• Language: English
• Subjects: Control systems design; Controllers; Face recognition; Feature extraction; Head movement; Human following; Human motion; Human performance; improved Face-TLD algorithm; mobile robots; Model accuracy; Robustness; Service robots; Skeleton; skeleton recognition; Target recognition; Target tracking; Task analysis; Tracking; Uncertainty
• ISSN: 2168-2216; 2168-2232
• DOI: 10.1109/TSMC.2019.2921974

Target recognition is a challenging task for human following of mobile service robots. In this paper, we combine the principal-component-analysis (PCA)-based face recognition with the tracking-learning-detection applied to the human face (Face-TLD) to obtain an improvement, named as IFace-TLD. The proposed IFace-TLD can significantly improve the discrimination ability of the Face-TLD for ambiguous facial appearances. To further deal with motion uncertainties of the human head, especially the sudden motion change, which makes face-based target recognition methods unstable or even loses the target, a skeleton-based model is introduced to improve the accuracy and robustness of the target recognition. Specifically, within a walk half-cycle, the skeleton features are extracted from the upper-body three-dimensional skeleton coordinates. Then, the extracted skeleton features are fed into the support vector data description (SVDD) to identify the target person when the IFace-TLD becomes invalid. The seamless fusion of the skeleton recognition and the IFace-TLD, named as the SIFace-TLD, significantly enhances the robustness in complex scenarios, especially for people tracking from both front and behind. To achieve a complete human following system, the particle filter (PF) is adopted for estimating the state of the human motion. And then, a controller is designed to maintain the relative position between the robot and the target. Experimental results demonstrate that the proposed IFace-TLD is more accurate and flexible than the original Face-TLD. And the SIFace-TLD shows a robust performance to human motion uncertainties. Moreover, the developed controller can achieve a satisfactory human following performance.