Multisource Hyperspectral and LiDAR Data Fusion for Urban Land-Use Mapping based on a Modified Two-Branch Convolutional Neural Network

• Published in: ISPRS international journal of geo-information 2019-01, Vol.8 (1), p.28
• Main Authors: Feng, Quanlong, Zhu, Dehai, Yang, Jianyu, Li, Baoguo
• Format: Article
• Language: English
• Subjects: Accuracy; Artificial neural networks; Classification; convolutional neural networks; Data; Data integration; Deep learning; Detection; Feature extraction; feature fusion; Hyperspectral imaging; Image classification; Imagery; Land surveys; Land use; Land use classification; Lidar; Machine learning; Mapping; Model accuracy; multisource data; Network design; Neural networks; Object recognition; Remote sensing; Remote sensors; Teaching methods; urban land-use mapping
• ISSN: 2220-9964; 2220-9964
• DOI: 10.3390/ijgi8010028

Accurate urban land-use mapping is a challenging task in the remote-sensing field. With the availability of diverse remote sensors, synthetic use and integration of multisource data provides an opportunity for improving urban land-use classification accuracy. Neural networks for Deep Learning have achieved very promising results in computer-vision tasks, such as image classification and object detection. However, the problem of designing an effective deep-learning model for the fusion of multisource remote-sensing data still remains. To tackle this issue, this paper proposes a modified two-branch convolutional neural network for the adaptive fusion of hyperspectral imagery (HSI) and Light Detection and Ranging (LiDAR) data. Specifically, the proposed model consists of a HSI branch and a LiDAR branch, sharing the same network structure to reduce the time cost of network design. A residual block is utilized in each branch to extract hierarchical, parallel, and multiscale features. An adaptive-feature fusion module is proposed to integrate HSI and LiDAR features in a more reasonable and natural way (based on "Squeeze-and-Excitation Networks"). Experiments indicate that the proposed two-branch network shows good performance, with an overall accuracy of almost 92%. Compared with single-source data, the introduction of multisource data improves accuracy by at least 8%. The adaptive fusion model can also increase classification accuracy by more than 3% when compared with the feature-stacking method (simple concatenation). The results demonstrate that the proposed network can effectively extract and fuse features for a better urban land-use mapping accuracy.