A Feature Level Multi-Sensor Information Fusion Strategy for Land Region Classification

• Main Authors: Schierl, Jonathan, Asari, Vijayan, Singer, Nina, Aspiras, Theus, Stokes, Andrew, Keaffaber, Brett, Van Rynbach, Andre, Decker, Kevin, Rabb, David
• Format: Conference Proceeding
• Language: English
• Subjects: 3D processing; aerial imagery; automatic target recognition; classification; computer vision; Deep learning; fusion; geospa-tial intelligence; GIS; Image recognition; Laser radar; lidar; machine learning; mapping; point clouds; remote sensing; scene understanding; Surveillance; Target recognition; Three-dimensional displays; transfer learning; Urban planning; wide area surveillance
• ISSN: 2332-5615
• DOI: 10.1109/AIPR52630.2021.9762086

Deep learning has seen success in image recognition and classification in remote sensing research areas. Specifically, it is capable of classifying land coverage by its usage, such as determining if a plot of land is a residential area, industrial area, forest, etc. This is useful for quickly parsing datasets for surveillance applications and also for autonomously categorizing municipal land for urban planning. However, current methods learn and classify based on a single data stream (EO) and consequentially a single set of features. Our approach for land classification, LandNet, uses a feature-level fusion approach of both EO and LiDAR data in a deep learning classification system. In our approach, we use an aerial dataset of geo-registered scenes captured by both EO and LiDAR sensors. We propose a deep-learning based feature-level fusion framework for integrating information extracted from 2D and 3D data. In our preliminary testing, we observed that this proposed network is a more effective approach as it employs feature sets of multiple sensors, and it outperforms each of the individual modalities.