SegMind: Semisupervised Remote Sensing Image Semantic Segmentation With Masked Image Modeling and Contrastive Learning Method

• Published in: IEEE transactions on geoscience and remote sensing 2023, Vol.61, p.1-17
• Main Authors: Li, Zhenghong, Chen, Hao, Wu, Jiangjiang, Li, Jun, Jing, Ning
• Format: Article
• Language: English
• Subjects: Annotations; Contrastive learning (CL); Deep learning; Image enhancement; Image processing; Image segmentation; Learning systems; masked image modeling (MIM); Modelling; Remote sensing; remote sensing (RS) image semantic segmentation; Roads; Semantic segmentation; Semantics; semisupervised learning; Training; Transformers
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2023.3321041

Remote sensing (RS) image semantic segmentation has attracted much attention due to its wide applications. However, deep learning-based RS image semantic segmentation methods usually require substantial manual pixelwise annotations, which are expensive and hard to obtain in practice. Although the existing semisupervised RS semantic segmentation methods effectively reduce dependence on labeled data, they generally focus on information consistency between labeled and unlabeled images, but ignore the potential context information between different areas of the RS image. In fact, the objects contained in an RS image usually have some long-range dependence between each other, since trees are usually on both sides of a road, and the middle of two rows of houses is commonly a road. Therefore, we believe that the potential dependencies between different areas of the RS image should be beneficial to reduce the label dependence of RS semantic segmentation. Based on this point, we propose a novel semisupervised RS image semantic segmentation network named SegMind, which is based on mean-teacher (MT) architecture and adopts masked image modeling (MIM) to enhance information interactions of different areas. Moreover, contrastive learning (CL) and entropy loss are introduced to SegMind framework to further improve the linear separability and prediction confidence of the proposed model. Experiments on three datasets have demonstrated the superiority of the proposed method over the state-of-the-art methods. The code is available at https://github.com/lzh-ggs-ddu/SegMind .