DroneSegNet: Robust Aerial Semantic Segmentation for UAV-Based IoT Applications

• Published in: IEEE transactions on vehicular technology 2022-04, Vol.71 (4), p.4277-4286
• Main Authors: Chakravarthy, Anirudh S., Sinha, Soumendu, Narang, Pratik, Mandal, Murari, Chamola, Vinay, Yu, F. Richard
• Format: Article
• Language: English
• Subjects: aerial scene analysis; Cloud computing; Coders; Computer architecture; Deep learning; Drone aircraft; Drones; Feature extraction; Feature maps; Image segmentation; Internet of Things; IoT; Machine learning; Modules; Remote sensing; Remote sensors; Rescue operations; Robustness; Search and rescue missions; Semantic segmentation; Semantics; UAVs; Unmanned aerial vehicles
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2022.3144358

Unmanned Aerial Vehicles (UAVs) are the promising "Flying IoT" devices of the future, which can be equipped with various sensors and cognitive capabilities to perform numerous tasks related to remote sensing, search and rescue operations, object tracking, segmentation of roads and buildings, surveillance, etc. However, these AI-driven tasks require heavy computation and may lead to suboptimal performance with embedded processors on a power-constrained battery-operated drone. This work proposes a novel deep learning approach for performing robust semantic segmentation of aerial scenes captured by UAVs. In our setup, the power-constrained drone is used only for data collection, while the computationally intensive tasks are offloaded to a GPU cloud server. Our architecture performs robust semantic segmentation by learning the segmentation maps from jointly utilizing of aerial scenes along with the respective "elevation maps" in a semi-supervised approach. We propose a three-tier deep learning architecture, wherein the first module aims at preliminary feature extraction from aerial scenes using a backbone feature extractor. The second module captures the spatial dependency between the aerial scenes and their respective elevation maps to obtain better semantic information, which is achieved by a bi-directional LSTM. The third module is aimed at enhancing the performance of semantic segmentation through a semi-supervised approach with an encoder to generate segmentation maps and a decoder to reconstruct feature maps. This semi-supervised feature learning ensures robust extraction along with scalability. The proposed architecture was validated on real-world aerial datasets and achieves state-of-the-art results for aerial image segmentation.