SE-Calib: Semantic Edge-Based LiDAR–Camera Boresight Online Calibration in Urban Scenes

• Published in: IEEE transactions on geoscience and remote sensing 2023, Vol.61, p.1-13
• Main Authors: Liao, Youqi, Li, Jianping, Kang, Shuhao, Li, Qiang, Zhu, Guifang, Yuan, Shenghai, Dong, Zhen, Yang, Bisheng
• Format: Article
• Language: English
• Subjects: Boresight error; Boresights; Calibration; Cameras; Data collection; Data integration; Detection; Edge detection; Image segmentation; Lidar; Methods; Optimization; Parameters; Robotics; Semantic segmentation; Semantics; Three dimensional models
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2023.3278024

Rigorous boresight calibration between light detection and ranging (LiDAR) and the camera is crucial for geometry and optical information fusion in Earth observation and robotic applications. Although boresight parameters can be obtained through precalibration with artificial targets, unforeseen movement of sensors during data collection can lead to significant errors in the boresight parameters. To address this issue, we propose SE-Calib, an automatic and target-free online boresight calibration method for LiDAR–camera systems. SE-Calib first extracts the semantic edge features from both point clouds and images simultaneously using the 3-D semantic segmentation (3-D-SS) and 2-D semantic edge detection (2-D-SED) methods. The boresight parameters are then optimized with an adaptive solver and maximizing the soft semantic response consistency metric (SSRCM) scores iteratively. The SSRCM is designed to evaluate the coherence of cross-modular semantic edge features, and a confidence function is proposed to filter out unreliable optimization results. Experiments conducted on challenging urban datasets show an average boresight error of 0.206° (2.47 pixels in reprojection error), demonstrating the effectiveness and robustness of the proposed method.