A Biologically Interpretable Two-Stage Deep Neural Network (BIT-DNN) for Vegetation Recognition From Hyperspectral Imagery

• Published in: IEEE transactions on geoscience and remote sensing 2022, Vol.60, p.1-20
• Main Authors: Shi, Yue, Han, Liangxiu, Huang, Wenjiang, Chang, Sheng, Dong, Yingying, Dancey, Darren, Han, Lianghao
• Format: Article
• Language: English
• Subjects: Accuracy; Artificial neural networks; Biochemistry; Biological system modeling; Biology; Classification; Computer applications; Crop diseases; Data models; Deep learning; Feature extraction; hyper-spectral images (HSIs); Hyperspectral imaging; Image classification; Imagery; interpretability; Land cover; Machine learning; Model accuracy; Monitoring; Neural networks; Plant diseases; Plant species; Recognition; Spatial discrimination learning; Species classification; Spectra; Structural hierarchy; Vegetation; Vegetation mapping
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2021.3058782

Spectral-spatial-based deep learning models have recently proven to be effective in hyper-spectral image (HSI) classification for various earth monitoring applications such as land cover classification and agricultural monitoring. However, due to the nature of "black-box" model representation, how to explain and interpret the learning process and the model decision, especially for vegetation classification, remains an open challenge. This study proposes a novel interpretable deep learning model-a biologically interpretable two-stage deep neural network (BIT-DNN), by incorporating the prior-knowledge (i.e., biophysical and biochemical attributes and their hierarchical structures of target entities)-based spectral-spatial feature transformation into the proposed framework, capable of achieving both high accuracy and interpretability on HSI-based classification tasks. The proposed model introduces a two-stage feature learning process: in the first stage, an enhanced interpretable feature block extracts the low-level spectral features associated with the biophysical and biochemical attributes of target entities; and in the second stage, an interpretable capsule block extracts and encapsulates the high-level joint spectral-spatial features representing the hierarchical structure of biophysical and biochemical attributes of these target entities, which provides the model an improved performance on classification and intrinsic interpretability with reduced computational complexity. We have tested and evaluated the model using four real HSI data sets for four separate tasks (i.e., plant species classification, land cover classification, urban scene recognition, and crop disease recognition tasks). The proposed model has been compared with five state-of-the-art deep learning models. The results demonstrate that the proposed model has competitive advantages in terms of both classification accuracy and model interpretability, especially for vegetation classification.