Coarse-to-fine feature representation based on deformable partition attention for melanoma identification

• Published in: Pattern recognition 2023-04, Vol.136, p.109247, Article 109247
• Main Authors: Zhang, Dong, Yang, Jing, Du, Shaoyi, Han, Hongcheng, Ge, Yuyan, Zhu, Longfei, Li, Ce, Xu, Meifeng, Zheng, Nanning
• Format: Article
• Language: English
• Subjects: Attention mechanism; Deep learning; Deformable convolution; Feature representation; Histopathological image; Melanoma identification
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2022.109247

•We propose a novel and efficient coarse-to-fine neural network for melanomas and nevi identification, which adopt a divide and conquer idea to overcome unbalanced inter-class spacing, and then leverages a fine sub-network to tackle the challenges of arduous training and unfavorable classification accuracy.•The proposed deformable partition attention module gradually refine the channel and spatial features, which enriches feature representation by flexibly combining global and local features to produce fine-grained attention features.•A joint loss function is proposed by organically integrating the inter-class cross-entropy with the intra-class similarity to increase the inter-class differences and narrow the intra-class disparities, compensating for the shortcomings of the single-loss function. In the histopathological melanoma image diagnosis system, manual identification of super-scale slides with dense cells is tedious, time-consuming, and subjective. To deal with this problem, we propose an automatic identification network based on the deformable partition attention to identify lots of dense slides as an assistant. A coarse-to-fine strategy is adopted in feature representation and qualitative identification to improve the identification accuracy of melanomas and nevi. First of all, because it is difficult to extract features in the lesion area with blurred boundaries and uneven distribution, we develop a deformable partition attention module, which integrates the advantage of the attention mechanism and deformable convolution. The module overcomes the limitation of rectangular convolution and gradually refines the channel and spatial features, which enriches feature representation by combining global and local features. Secondly, to address the problem of difficult convergence and poor recognition rate caused by the excessive non-aligned distance between benign-malignant and benign subcategories, we propose a progressive architecture via a coarse sub-network closely followed by a fine sub-network. Moreover, to further increase the inter-class differences and reduce the intra-class disparities, we propose a joint loss function to mine hard samples, which effectively improves the identification performance. Experimental results on the clinical dataset show that the proposed algorithm has higher sensitivity and specificity and outperforms state-of-the-art deep neural networks.