Weakly Alignment-free RGBT Salient Object Detection with Deep Correlation Network

RGBT Salient Object Detection (SOD) focuses on common salient regions of a pair of visible and thermal infrared images. Existing methods perform on the well-aligned RGBT image pairs, but the captured image pairs are always unaligned and aligning them requires much labor cost. To handle this problem,...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on image processing 2022-01, Vol.PP, p.1-1
Main Authors: Tu, Zhengzheng, Li, Zhun, Li, Chenglong, Tang, Jin
Format: Article
Language:English
Subjects:
Affine transformations
Alignment
Convolution
Correlation
Decoding
Feature extraction
Infrared imagery
Kernel
Modules
Object recognition
Salience
Semantics
Task analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:RGBT Salient Object Detection (SOD) focuses on common salient regions of a pair of visible and thermal infrared images. Existing methods perform on the well-aligned RGBT image pairs, but the captured image pairs are always unaligned and aligning them requires much labor cost. To handle this problem, we propose a novel deep correlation network (DCNet), which explores the correlations across RGB and thermal modalities, for weakly alignment-free RGBT SOD. In particular, DCNet includes a modality alignment module based on the spatial affine transformation, the feature-wise affine transformation and the dynamic convolution to model the strong correlation of two modalities. Moreover, we propose a novel bi-directional decoder model, which combines the coarse-to-fine and fine-to-coarse processes for better feature enhancement. In particular, we design a modality correlation ConvLSTM by adding the first two components of modality alignment module and a global context reinforcement module into ConvLSTM, which is used to decode hierarchical features in both top-down and button-up manners. Extensive experiments on three public benchmark datasets show the remarkable performance of our method against state-of-the-art methods.
ISSN:1057-7149
1941-0042
DOI:10.1109/TIP.2022.3176540