Hyperspectral Image Transformer Classification Networks

Hyperspectral image (HSI) classification is an important task in earth observation missions. Convolution neural networks (CNNs) with the powerful ability of feature extraction have shown prominence in HSI classification tasks. However, existing CNN-based approaches cannot sufficiently mine the seque...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing 2022, Vol.60, p.1-15
Main Authors: Yang, Xiaofei, Cao, Weijia, Lu, Yao, Zhou, Yicong
Format: Article
Language:English
Subjects:
3-D convolution projection
Artificial neural networks
Classification
Convolution
convolution neural network (CNN)
Data mining
Datasets
Dimensions
Embedding
Feature extraction
hyperspectral image (HSI) classification
Hyperspectral imaging
Image classification
Modules
Neural networks
Projection
Task analysis
Three-dimensional displays
Transformers
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Hyperspectral image (HSI) classification is an important task in earth observation missions. Convolution neural networks (CNNs) with the powerful ability of feature extraction have shown prominence in HSI classification tasks. However, existing CNN-based approaches cannot sufficiently mine the sequence attributes of spectral features, hindering the further performance promotion of HSI classification. This article presents a hyperspectral image transformer (HiT) classification network by embedding convolution operations into the transformer structure to capture the subtle spectral discrepancies and convey the local spatial context information. HiT consists of two key modules, i.e., spectral-adaptive 3-D convolution projection module and convolution permutator (ConV-Permutator) to retrieve the subtle spatial-spectral discrepancies. The spectral-adaptive 3-D convolution projection module produces the local spatial-spectral information from HSIs using two spectral-adaptive 3-D convolution layers instead of the linear projection layer. In addition, the Conv-Permutator module utilizes the depthwise convolution operations to separately encode the spatial-spectral representations along the height, width, and spectral dimensions, respectively. Extensive experiments on four benchmark HSI datasets, including Indian Pines, Pavia University, Houston2013, and Xiongan (XA) datasets, show the superiority of the proposed HiT over existing transformers and the state-of-the-art CNN-based methods. Our codes of this work are available at https://github.com/xiachangxue/DeepHyperX for the sake of reproducibility.
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2022.3171551