Camera-Radar Fusion with Radar Channel Extension and Dual-CBAM-FPN for Object Detection

• Published in: Sensors (Basel, Switzerland) Switzerland), 2024-08, Vol.24 (16), p.5317
• Main Authors: Sun, Xiyan, Jiang, Yaoyu, Qin, Hongmei, Li, Jingjing, Ji, Yuanfa
• Format: Article
• Language: English
• Subjects: camera-radar fusion; Cameras; CBAM; Clouds; FPN; object detection; radar channel extension; Sensors; Velocity
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s24165317

When it comes to road environment perception, millimeter-wave radar with a camera facilitates more reliable detection than a single sensor. However, the limited utilization of radar features and insufficient extraction of important features remain pertinent issues, especially with regard to the detection of small and occluded objects. To address these concerns, we propose a camera-radar fusion with radar channel extension and a dual-CBAM-FPN (CRFRD), which incorporates a radar channel extension (RCE) module and a dual-CBAM-FPN (DCF) module into the camera-radar fusion net (CRF-Net). In the RCE module, we design an azimuth-weighted RCS parameter and extend three radar channels, which leverage the secondary redundant information to achieve richer feature representation. In the DCF module, we present the dual-CBAM-FPN, which enables the model to focus on important features by inserting CBAM at the input and the fusion process of FPN simultaneously. Comparative experiments conducted on the NuScenes dataset and real data demonstrate the superior performance of the CRFRD compared to CRF-Net, as its weighted mean average precision (wmAP) increases from 43.89% to 45.03%. Furthermore, ablation studies verify the indispensability of the RCE and DCF modules and the effectiveness of azimuth-weighted RCS.