Instance-Aware Distillation for Efficient Object Detection in Remote Sensing Images

Practical applications ask for object detection models that achieve high performance at low overhead. Knowledge distillation demonstrates favorable potential in this case by transferring knowledge from a cumbersome teacher model to a lightweight student model. However, previous distillation methods...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing 2023-01, Vol.61, p.1-1
Main Authors: Li, Cong, Cheng, Gong, Wang, Guangxing, Zhou, Peicheng, Han, Junwei
Format: Article
Language:English
Subjects:
Behavioral sciences
Detection
Detectors
Distillation
Distilling
Feature extraction
knowledge distillation
Knowledge management
Methods
Object detection
Object recognition
Remote sensing
remote sensing images
Sensors
Source code
Task analysis
Teachers
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Practical applications ask for object detection models that achieve high performance at low overhead. Knowledge distillation demonstrates favorable potential in this case by transferring knowledge from a cumbersome teacher model to a lightweight student model. However, previous distillation methods are plagued with massive misleading background information in remote sensing images and ignore investigating the relations between different instances. In this article, we propose an instance-aware distillation method (InsDist for short) to derive efficient remote sensing object detectors. Our InsDist combines feature-based and relation-based knowledge distillation to make the most of instance-related information in the knowledge transfer from the teacher to the student. On the one hand, we propose a parameter-free masking module to decouple instance-related foreground from instance-irrelevant background in multi-scale features. On the other hand, we construct the relations between different instances to enhance the learning of intraclass compactness and interclass dispersion. The student comprehensively imitates both features and relations from the teacher, yielding considerable effectiveness in dealing with complex remote sensing images. In addition, our InsDist can be easily built upon mainstream object detectors with negligible extra cost. Extensive experiments on two large-scale remote sensing object detection datasets, namely DIOR and DOTA, show that our InsDist obtains noticeable gains over other distillation methods for both one-stage and two-stage, as well as both anchor-based and anchor-free detectors. The source code will be publicly available at https://github.com/swift1988/InsDist..
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2023.3238801