Dual selective knowledge transfer for few-shot classification

• Published in: Applied intelligence (Dordrecht, Netherlands) Netherlands), 2023-11, Vol.53 (22), p.27779-27789
• Main Authors: He, Kai, Pu, Nan, Lao, Mingrui, Bakker, Erwin M., Lew, Michael S.
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Classification; Computer Science; Datasets; Deep learning; Distillation; Feature extraction; Knowledge; Knowledge management; Learning; Machines; Manufacturing; Mechanical Engineering; Optimization; Processes; Representations; Teachers
• ISSN: 0924-669X; 1573-7497
• DOI: 10.1007/s10489-023-04994-7

Few-shot learning aims at recognizing novel visual categories from very few labelled examples. Different from the existing few-shot classification methods that are mainly based on metric learning or meta-learning, in this work we focus on improving the representation capacity of feature extractors. For this purpose, we propose a new two-stage dual selective knowledge transfer (DSKT) framework, to guide models towards better optimization. Specifically, we first exploit an improved multi-task learning approach to train a feature extractor with robust representation capability as a teacher model. Then, we design an effective dual selective knowledge distillation method, which enables the student model to selectively learn knowledge from the teacher model and current samples, thereby improving the student model’s ability to generalize on unseen classes. Extensive experimental results show that our DSKT achieves competitive performances on four well-known few-shot classification benchmarks.