Remote Sensing Image Rectangling With Iterative Warping Kernel Self-Correction Transformer

Stitched remote sensing images often exhibit irregular boundaries, which can be frustrating for general users and detrimental to downstream tasks such as object detection and segmentation. However, this issue has received insufficient attention and remains unexplored within the remote sensing domain...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing 2024, Vol.62, p.1-17
Main Authors: Qiu, Linwei, Xie, Fengying, Liu, Chang, Wang, Ke, Song, Xuedong, Shi, Zhenwei
Format: Article
Language:English
Subjects:
Algorithms
Boundaries
Deformation
Distortion
Feature extraction
Image processing
Image segmentation
Indexes
Kernel
Modules
Object recognition
Rectangling
Remote sensing
remote sensing images
Stitching
Transformers
Warping
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Stitched remote sensing images often exhibit irregular boundaries, which can be frustrating for general users and detrimental to downstream tasks such as object detection and segmentation. However, this issue has received insufficient attention and remains unexplored within the remote sensing domain. In this study, we investigate mesh-based rectangling techniques for remote sensing images, aiming to produce rectangular outputs while preserving the original field-of-view (FoV) and avoiding the introduction of unreliable content. Observing that prior rectangling algorithms tend to generate unsatisfactory boundaries or discernible distortions, that is, under-rectangling or over-rectangling, we propose the concept of a warping kernel associated with mesh deformations to account for these phenomena. Consequently, we introduce the iterative warping kernel self-correction transformer (IWKFormer), designed to enhance warping kernel estimation and generate superior rectangular outcomes. It primarily comprises two components: a mesh feature extractor built upon the partial swin transformer block (PSTB) and a corrector module using the swin transformer block (STB). These modules collaborate to derive warping kernels implicitly. The extractor extracts latent features pertinent to mesh deformation, whereas the corrector iteratively refines the warping kernel estimation to improve the ultimate prediction. Furthermore, to bolster further research, we have constructed an aerial imagery stitching rectangling dataset (AIRD), featuring a wide array of stitching scenes. Extensive experimentation on the AIRD demonstrates that our method yields visually appealing and naturally rectangled images, achieving state-of-the-art performance. The code and data will be available at https://github.com/yyywxk/IWKFormer .
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2024.3441246