MSLAN: A Two-Branch Multidirectional Spectral-Spatial LSTM Attention Network for Hyperspectral Image Classification

• Published in: IEEE transactions on geoscience and remote sensing 2022, Vol.60, p.1-14
• Main Authors: Song, Tiecheng, Wang, Yuanlin, Gao, Chenqiang, Chen, Haonan, Li, Jun
• Format: Article
• Language: English
• Subjects: Attention; Classification; Convolutional neural networks; Deep learning; Entropy; Feature extraction; hyperspectral image (HSI) classification; Hyperspectral imaging; Image classification; Logic gates; Long short-term memory; long short-term memory (LSTM); Machine learning; Modelling; Neural networks; Principal component analysis; recurrent neural network (RNN); Recurrent neural networks; Spatial dependencies; Spatial distribution; Spatial memory; spectral–spatial feature; Training
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2022.3176216

Recurrent neural networks (RNNs) have been widely used for hyperspectral image (HSI) classification via sequence modeling. However, most of the RNN methods focus on modeling long-range dependencies along the spectral direction without fully exploring multidirectional dependencies in the joint spectral-spatial domain. To tackle this issue, we propose MSLAN, a two-branch multidirectional spectral-spatial long short-term memory (LSTM) attention network, for HSI classification. In particular, we employ LSTMs to extract six-directional spatial-spectral features that simultaneously capture the spectral-spatial dependencies along with different directions. We then design an attention-based feature fuse module to integrate these directional features, followed by a fully connected layer with cross-entropy loss for classification. In addition, we incorporate an auxiliary branch into our model to enhance the generalization capability. In this branch, random spatial shuffle and a cosine loss are explored for feature consistency learning by taking into account the varying spatial distributions. The resulting two branch networks, sharing the same network structure and weights, are incorporated into a unified deep learning architecture for training. Experiments show the superiority of MSLAN to the state-of-the-art methods for HSI classification with limited training samples.