Sequential Knowledge Transfer in Teacher-Student Framework Using Densely Distilled Flow-Based Information

This paper proposes an iterative sequential knowledge transfer (KT) technique suitable for teacher-student framework (TSF)-based image classification when a state-of-the-art residual network (ResNet) is used for the TSF. To this end, we first extracted densely distilled knowledge in terms of the flo...

Full description

Saved in:
Bibliographic Details
Main Authors: Yeo, Doyeob, Bae, Ji-Roon, Kim, Nae-Soo, Pyo, Cheol-Sig, Yim, Junho, Kim, Junmo
Format: Conference Proceeding
Language:English
Subjects:
Computational modeling
Data mining
deep-learning
Image classification
Knowledge transfer
Optimization
Reliability
teacher-student framework
Training
transfer learning
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper proposes an iterative sequential knowledge transfer (KT) technique suitable for teacher-student framework (TSF)-based image classification when a state-of-the-art residual network (ResNet) is used for the TSF. To this end, we first extracted densely distilled knowledge in terms of the flow between the layers in the teacher ResNet. Subsequently, we iteratively and sequentially trained the student ResNet using the densely extracted flow-based teacher information. When using the proposed method, the trained student ResNet exhibited better performance in terms of classification accuracy and fast learning than the existing TSF-based KT methods considered in this study.
ISSN:2381-8549
DOI:10.1109/ICIP.2018.8451121