DR-AVIT: Toward Diverse and Realistic Aerial Visible-to-Infrared Image Translation

Image-to-image (I2I) translation methods based on generative adversarial networks (GANs) have shown general solutions for aerial visible-to-infrared image translation (AVIT) task. Though the existing approaches have made impressive results, they still struggle to produce diverse or high-realism tran...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing 2024, Vol.62, p.1-13
Main Authors: Han, Zonghao, Zhang, Shun, Su, Yuru, Chen, Xiaoning, Mei, Shaohui
Format: Article
Language:English
Subjects:
Aerial visible-to-infrared image translation (AVIT)
Benchmarks
Consistency
Datasets
deep learning
Generative adversarial networks
generative adversarial networks (GANs)
Generators
Image representation
image-to-image (I2I) translation
Imaging
Infrared imagery
Realism
Representations
Semantics
Silver
Task analysis
Training
Translation
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Image-to-image (I2I) translation methods based on generative adversarial networks (GANs) have shown general solutions for aerial visible-to-infrared image translation (AVIT) task. Though the existing approaches have made impressive results, they still struggle to produce diverse or high-realism translated aerial infrared images (AIIs). In this article, a novel model is proposed to achieve both diverse and realistic AVIT, named DR-AVIT. Specifically, we introduce disentangled representation learning to disentangle the image representation of aerial visible images (AVIs) and AIIs into a domain-invariant semantic structure space and two domain-specific imaging style spaces. By leveraging this disentanglement, our model can perform the translation process conditioned on semantic structure information derived from the input AVI and randomly sampled imaging style features from the AII domain to obtain diverse outputs. Furthermore, a new constraint is present to encourage GANs to learn efficient mappings between AVI and AII domains by integrating geometry-consistency constraint and a dual learning framework, named dual geometry-consistency constraint. Coping with these two designs, our method exhibits superiority in both realism and diversity of the translation results over several state-of-the-art I2I translation methods on AVIID dataset and two new benchmark datasets for AVIT, which are obtained by extracting data from publicly available datasets. Codes of DR-AVIT and proposed benchmark datasets are available at https://github.com/silver-hzh/DR-AVIT .
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2024.3405989