Hybrid CNN and Transformer Network for Semantic Segmentation of UAV Remote Sensing Images

Semantic segmentation of unmanned aerial vehicle (UAV) remote sensing images is a recent research hotspot, offering technical support for diverse types of UAV remote sensing missions. However, unlike general scene images, UAV remote sensing images present inherent challenges. These challenges includ...

Full description

Saved in:
Bibliographic Details
Published in:IEEE journal on miniaturization for air and space systems 2024-03, Vol.5 (1), p.33-41
Main Authors: Zhou, Xuanyu, Zhou, Lifan, Gong, Shengrong, Zhang, Haizhen, Zhong, Shan, Xia, Yu, Huang, Yizhou
Format: Article
Language:English
Subjects:
Artificial neural networks
Autonomous aerial vehicles
Coders
Complexity
Convolutional neural networks
Feature extraction
Image segmentation
Modules
Remote sensing
Semantic segmentation
Semantics
Swin transformer
Technical services
Transformers
unmanned aerial vehicle (UAV)
Unmanned aerial vehicles
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Semantic segmentation of unmanned aerial vehicle (UAV) remote sensing images is a recent research hotspot, offering technical support for diverse types of UAV remote sensing missions. However, unlike general scene images, UAV remote sensing images present inherent challenges. These challenges include the complexity of backgrounds, substantial variations in target scales, and dense arrangements of small targets, which severely hinder the accuracy of semantic segmentation. To address these issues, we propose a convolutional neural network (CNN) and transformer hybrid network for semantic segmentation of UAV remote sensing images. The proposed network follows an encoder-decoder architecture that merges a transformer-based encoder with a CNN-based decoder. First, we incorporate the Swin transformer as the encoder to address the limitations of CNN in global modeling, mitigating the interference caused by complex background information. Second, to effectively handle the significant changes in target scales, we design the multiscale feature integration module (MFIM) that enhances the multiscale feature representation capability of the network. Finally, the semantic feature fusion module (SFFM) is designed to filter the redundant noise during the feature fusion process, which improves the recognition of small targets and edges. Experimental results demonstrate that the proposed method outperforms other popular methods on the UAVid and Aeroscapes datasets.
ISSN:2576-3164
2576-3164
DOI:10.1109/JMASS.2023.3332948