Joint spatial-spectral hyperspectral image classification based on convolutional neural network

• Published in: Pattern recognition letters 2020-02, Vol.130, p.38-45
• Main Authors: Han, Mengxin, Cong, Runmin, Li, Xinyu, Fu, Huazhu, Lei, Jianjun
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Classification; CNN; Hyperspectral image classification; Hyperspectral imaging; Image classification; Image contrast; Image detection; Image enhancement; Joint spatial-spectral; Machine learning; Neural networks; Object recognition; Pixels; Spatial enhancement; Spectra; Training
• ISSN: 0167-8655; 1872-7344
• DOI: 10.1016/j.patrec.2018.10.003

•A joint spatial-spectral hyperspectral image classification framework based on convolutional neural network is proposed.•A two-stream convolutional network is designed to learn the spatial-spectral features at different scales.•A spatial enhancement strategy is proposed to obtain more comprehensive spatial features with limited training samples. Hyperspectral image (HSI) classification technology has been widely used in many earth observation tasks, such as detection, recognition, and surveillance. The traditional hyperspectral image classification methods mainly utilize hand-crafted features, such as edge and texture descriptors, which are not robust for different input data. By contrast, deep learning based methods exploit high-level features for hyperspectral image classification, but they usually degenerate the spatial-spectral structure, depend on a large number of training samples, and ignore a large amount of implicitly useful information. To address these problems, a new joint spatial-spectral hyperspectral image classification method based on different-scale two-stream convolutional network and spatial enhancement strategy is proposed in this paper. First, the pixel blocks at different scales around the center pixel are selected as the basic units to be processed. Then, a spatial enhancement strategy is designed to obtain various spatial location information under the limited training samples by the spatial rotation and row-column transformation. Finally, the spatial-spectral feature is learned by a different-scale two-stream convolutional network, and the classification result of the center pixel is obtained by a softmax layer. Experimental results on two datasets demonstrate that the proposed method outperforms other state-of-the-art methods qualitatively and quantitatively.