Deep Domain Adaptation Based Multi-Spectral Salient Object Detection

Salient Object Detection (SOD) plays an important role in many image-related multimedia applications. Although there are many existing research works about the salient object detection in traditional RGB (visible-light spectrum) images, there are still many complex situations that regular RGB images...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2022, Vol.24, p.128-140
Main Authors: Song, Shaoyue, Miao, Zhenjiang, Yu, Hongkai, Fang, Jianwu, Zheng, Kang, Ma, Cong, Wang, Song
Format: Article
Language:English
Subjects:
Adaptation
Adaptation models
Color imagery
Computer vision
Datasets
Deep learning
Domain adaptation
Domains
Infrared imagery
Infrared spectra
Lighting
Meteorology
multi-spectral
Multimedia
Near infrared radiation
Object detection
Object recognition
Salience
salient object detection
Synchronization
Task analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Salient Object Detection (SOD) plays an important role in many image-related multimedia applications. Although there are many existing research works about the salient object detection in traditional RGB (visible-light spectrum) images, there are still many complex situations that regular RGB images cannot provide enough cues for the accurate SOD, such as the shadow effect, similar appearance between background and foreground, strong or insufficient illumination, etc. Because of the success of near-infrared spectrum in many computer vision tasks, we explore the multi-spectral SOD in the synchronized RGB images and near-infrared (NIR) images for the both simple and complex situations. We assume that the RGB SOD in the existing RGB image datasets could provide references for the multi-spectral SOD problem. In this paper, we mainly model this research problem as a deep learning based domain adaptation from the traditional RGB image data (source domain) to the multi-spectral data (target domain), and an adversarial deep domain adaptation model is proposed. We first collect and will publicize a large multi-spectral dataset, RGBN-SOD dataset, including 780 synchronized RGB and NIR image pairs for the multi-spectral SOD problem in the simple and complex situations. Intensive experimental results show the effectiveness and accuracy of the proposed deep domain adaptation for the multi-spectral SOD. Besides, due to the absence of research on the field of multi-spectral co-saliency detection, we also collect 200 synchronized RGB and NIR image pairs in addition to explore the multi-spectral co-saliency detection.
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2020.3046868