Robust Data Association Against Detection Deficiency for Semantic SLAM

• Published in: IEEE transactions on automation science and engineering 2024-01, Vol.21 (1), p.868-880
• Main Authors: Lin, Xubin, Ruan, Jiahao, Yang, Yirui, He, Li, Guan, Yisheng, Zhang, Hong
• Format: Article
• Language: English
• Subjects: Algorithms; Boxes; Cameras; Collision avoidance; Data association; False alarms; Feature extraction; Image manipulation; Image reconstruction; Inference; local homography; multiple object tracking; Object motion; Object recognition; Object tracking; Occlusion; Robustness; semantic SLAM; Semantics; Simultaneous localization and mapping; Target tracking; Task space; Three-dimensional displays; Trajectories; Trajectory
• ISSN: 1545-5955; 1558-3783
• DOI: 10.1109/TASE.2022.3233662

Robust and accurate object association is essential for precise 3D object landmark inference in semantic Simultaneous Localization and Mapping (SLAM), and yet remains challenging due to the detection deficiency caused by high miss detection rate, false alarm, occlusion and limited field-of-view, etc. The 2D location of an object is a crucial complementary cue to the appearance feature, especially in the case of associating objects across frames under large viewpoint changes. However, motion model or trajectory pattern based methods struggle to infer object motion reliably with a moving camera. In this paper, by exploiting the local projective warping consistency, a local homography based 2D motion inference method is proposed to sequentially estimate the object location along with uncertainty. By integrating the deep appearance feature and semantic information, an object association method, named HOA, which is robust to detection deficiency is proposed. Experimental evaluations suggest that the proposed motion prediction method is capable of maintaining a low cumulative error over a long duration, which enhances the object association performance in both accuracy and robustness. Note to Practitioners-This work aims to consistently associate 2D detection boxes corresponding to the same 3D object across images. In tasks of landmark-based navigation, collision avoidance, grasping and manipulation, objects in the task space are commonly simplified into 3D enveloping surfaces (e.g. cuboid or ellipsoid) by using 2D object detection boxes from multiple image views, and accurate data association is a prerequisite for precise enveloping surface reconstruction. This problem remains challenging considering the imperfect object detections, the appearance similarity of objects and the unpredictable trajectory of the moving camera. This work proposes a long-term reliable 2D location prediction algorithm that is capable of handling the complex motion of the target. Along with the appearance feature extracted by a retrain-free deep learning based model, this work proposes an object association method that can simultaneously deal with multiple objects with unknown object categories under the moving camera scenario.