Multilevel Superpixel Structured Graph U-Nets for Hyperspectral Image Classification

• Published in: IEEE transactions on geoscience and remote sensing 2022, Vol.60, p.1-15
• Main Authors: Liu, Qichao, Xiao, Liang, Yang, Jingxiang, Wei, Zhihui
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Classification; Computation; Convolutional neural network (CNN); Convolutional neural networks; Feature extraction; Fuses; graph convolutional network (GCN); graph U-Net; Graphs; hierarchical superpixel segmentation; hyperspectral image (HSI) classification; Hyperspectral imaging; Image classification; Image segmentation; Kernel; Land use; Machine learning; Merging; Multilevel; Neural networks; Shape; Structured data; Topology; Vegetation
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2021.3112586

Limited by the shape-fixed kernels, convolutional neural networks (CNNs) are usually difficult to model difform land covers in hyperspectral images (HSIs), leading to inadequate land use. Recently, benefiting from the ability to conduct shape-adaptive convolutions and model complex patterns in graph-structured data, graph convolutional networks (GCNs) have been applied to HSI classification. However, due to the massive computation in GCNs, HSI is usually pretreated into a graph based on a specific superpixel segmentation, which limits the modeling of spatial topologies to the same scale. To break this limitation, we propose a multilevel superpixel structured graph U-Net (MSSGU) to learn multiscale features on multilevel graphs. Specifically, we construct several hierarchical segmentations from fine to coarse by progressively merging adjacent superpixels and then convert them into multilevel graphs. Meanwhile, based on the merging relations between hierarchical superpixels, we establish the pooling and unpooling functions to transfer features from one graph to another, thereby enabling different-level graphs to collaborate in a single network. Different from concatenating different-scale features straightforwardly in the feature fusion stage, MSSGU fuses them in a coarse-to-fine progressive manner, which can generate subtler fusion features adaptive to the pixelwise classification task. Moreover, we use a CNN instead of GCN to extract and fuse the pixel-level features, which greatly reduces the computation. Such a hybrid U-Net can exploit features of HSIs from a multiscale hierarchical perspective, and its performance has been proven competitive with other deep-learning-based methods by extensive experiments on three benchmark datasets.