Joint Image and Feature Enhancement for Object Detection under Adverse Weather Conditions

• Main Authors: Yin, Mengyu, Ling, Mingyang, Chang, Kan, Yuan, Zijian, Qin, Qingpao, Chen, Boning
• Format: Conference Proceeding
• Language: English
• Subjects: Adverse Weather Conditions; Detectors; Feature Enhancement; Feature extraction; Head; Image Enhancement; Neck; Neural networks; Object detection; Reliability
• ISSN: 2161-4407
• DOI: 10.1109/IJCNN60899.2024.10650989

Object detection under adverse weather conditions remains a challenging problem to date. To address this problem, a joint image and feature enhancement method called JE-YOLO is proposed. Firstly, a lightweight image enhancement network is used to enhance the low-quality image captured under adverse weather conditions. Secondly, to provide rich information for detection, two detection backbones are applied in parallel to extract features from both the low-quality image and its enhanced result. Afterwards, the extracted features are further enhanced by a foreground-guided feature refinement module (FFRM), which introduces a task-driven attention mechanism and explores inter-layer correlation. Finally, the enhanced features from different branches are fused by the adaptive multi-branch weighting (AMW) strategy, and then fed to the neck and head of detector. Experiments are carried out on both the low-light and foggy conditions, and the results demonstrate that compared with state-of-the-art (SOTA) methods, the proposed JE-YOLO is able to achieve the highest accuracy of detection in all cases. Code will be available at https://github.com/Murray-Yin/JE-YOLO.