Deep attention branch networks for skin lesion classification

• Published in: Computer methods and programs in biomedicine 2021-11, Vol.212, p.106447-106447, Article 106447
• Main Authors: Ding, Saisai, Wu, Zhongyi, Zheng, Yanyan, Liu, Zhaobang, Yang, Xiaodong, Yang, Xiaokai, Yuan, Gang, Xie, Jing
• Format: Article
• Language: English
• Subjects: Attention branch; Class activation mapping; Class imbalance; Dermoscopy image; Loss weighting; Skin lesion classification
• ISSN: 0169-2607; 1872-7565
• DOI: 10.1016/j.cmpb.2021.106447

•In order to obtain CAM in the forward propagation processing, we propose a Deep Attention Branch Network (DABN) which introduces two attention branches with an attention mechanism to expand the DCNN model.•DABN takes the CAM as an attention map during the training process to make network focus on the lesion areas, leading to the performance improvement of DCNN, and the attention maps can be visualized in the forward propagation.•Αttention branch has a simple structure and few parameters, which can be applied in various DCNN structures and trained in an end-to-end manner.•A novel entropy-guided loss weighting strategy is introduced to counter the class imbalance. The results show that incorporating the entropy information into the loss function can significantly increase the accuracy of easily misclassification samples.•Without using any extra data and ensemble learning, we obtains better classification results on ISIC-2016 and 2017 dataset compared to state-of-the-art methods. Moreover, our method can also be applied in multi-class skin dataset and substantially improves classification performances. The skin lesion usually covers a small region of the dermoscopy image, and the lesions of different categories might own high similarities. Therefore, it is essential to design an elaborate network for accurate skin lesion classification, which can focus on semantically meaningful lesion parts. Although the Class Activation Mapping (CAM) shows good localization capability of highlighting the discriminative parts, it cannot be obtained in the forward propagation process. We propose a Deep Attention Branch Network (DABN) model, which introduces the attention branches to expand the conventional Deep Convolutional Neural Networks (DCNN). The attention branch is designed to obtain the CAM in the training stage, which is then utilized as an attention map to make the network focus on discriminative parts of skin lesions. DABN is applicable to multiple DCNN structures and can be trained in an end-to-end manner. Moreover, a novel Entropy-guided Loss Weighting (ELW) strategy is designed to counter class imbalance influence in the skin lesion datasets. The proposed method achieves an Average Precision (AP) of 0.719 on the ISIC-2016 dataset and an average area under the ROC curve (AUC) of 0.922 on the ISIC-2017 dataset. Compared with other state-of-the-art methods, our method obtains better performance without external data and ensemble learning. Moreover, extensive exper