Line-Based 2-D-3-D Registration and Camera Localization in Structured Environments

• Published in: IEEE transactions on instrumentation and measurement 2020-11, Vol.69 (11), p.8962-8972
• Main Authors: Yu, Huai, Zhen, Weikun, Yang, Wen, Scherer, Sebastian
• Format: Article
• Language: English
• Subjects: 2-D line detection; 2-D-3-D registration; 2-D–3-D line correspondences; camera localization; camera-Light Detection and Ranging (LiDAR) fusion; Cameras; Cloud computing; Feature extraction; Geometric constraints; Hypothesis testing; Image segmentation; Laser radar; Lidar; Localization; Mathematical analysis; Matrix methods; Optimization; Registration; Three dimensional models; Three-dimensional displays; Two dimensional displays; vanishing point matching; Visualization
• ISSN: 0018-9456; 1557-9662
• DOI: 10.1109/TIM.2020.2999137

Accurate registration of 2-D imagery with point clouds is a key technology for image-Light Detection and Ranging (LiDAR) point cloud fusion, camera to laser scanner calibration, and camera localization. Despite continuous improvements, automatic registration of 2-D and 3-D data without using additional textured information still faces great challenges. In this article, we propose a new 2-D-3-D registration method to estimate 2-D-3-D line feature correspondences and the camera pose in untextured point clouds of structured environments. Specifically, we first use geometric constraints between vanishing points and 3-D parallel lines to compute all feasible camera rotations. Then, we utilize a hypothesis testing strategy to estimate the 2-D-3-D line correspondences and the translation vector. By checking the consistency with computed correspondences, the best rotation matrix can be found. Finally, the camera pose is further refined using nonlinear optimization with all the 2-D-3-D line correspondences. The experimental results demonstrate the effectiveness of the proposed method on the synthetic and real data set (outdoors and indoors) with repeated structures and rapid depth changes.