Adaptive Composite Feature Generation for Object Detection in Remote Sensing Images

• Published in: IEEE transactions on geoscience and remote sensing 2024, Vol.62, p.1-16
• Main Authors: Zhang, Ziye, Mei, Shaohui, Ma, Mingyang, Han, Zonghao
• Format: Article
• Language: English
• Subjects: Context modeling; Detectors; Feature extraction; Knowledge distillation (KD); Knowledge transfer; Object detection; Remote sensing; remote sensing image; self-attention; Task analysis
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2024.3424295

Object detection in remote sensing images identifies and extracts the acquired Earth surface information, providing data support and research basis for multiple fields. Remote sensing image object detection based on knowledge distillation (KD) can transfer the knowledge of a large teacher model to a smaller student model, achieving the effect of low parameter volume and high accuracy. Mainstream methods directly imitate teacher features to improve student performance, ignoring the generation of high-ranking features through teacher features instructing student feature maps in this knowledge transfer process. In this article, an adaptive composite feature generation (ACFG) strategy is proposed to achieve end-to-end trainable KD for object detection in remote sensing images, in which the robustness of feature points under composite masks is improved through adaptive feature mapping. In particular, a composite mask generator (CMG) module is proposed to select student instance-related features and point background features. Furthermore, a global and local projection layer (GLPL) module is proposed to connect the local information and global information of the feature map under the mask generator to adaptively realize the global recovery mapping of the feature map with partial feature points. Finally, balanced decoupling loss (BDL) is improved to handle foreground and background loss separately, so that the two decoupled features can better enable the student model to learn instance-related information. Note that the proposed ACFG is capable of conducting KD for both single-stage and two-stage object detectors. Experimental results using both anchor-based and anchor-free detectors on the DIOR dataset and DOTA dataset demonstrate that the proposed ACFG clearly achieved better performance than several state-of-the-art (SOTA) algorithms for KD.