Salient Object Detection in the Deep Learning Era: An In-Depth Survey

As an essential problem in computer vision, salient object detection (SOD) has attracted an increasing amount of research attention over the years. Recent advances in SOD are predominantly led by deep learning-based solutions (named deep SOD). To enable in-depth understanding of deep SOD, in this pa...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2022-06, Vol.44 (6), p.3239-3259
Main Authors: Wang, Wenguan, Lai, Qiuxia, Fu, Huazhu, Shen, Jianbing, Ling, Haibin, Yang, Ruigang
Format: Article
Language:English
Subjects:
Algorithms
Analytical models
Annotations
Attention
benchmark
Benchmark testing
Benchmarking
Computational modeling
Computer architecture
Computer vision
Datasets
Deep Learning
image saliency
Machine learning
Object detection
Object recognition
Perturbation
Predictive models
Salience
Salient object detection
Superoxide Dismutase
Taxonomy
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As an essential problem in computer vision, salient object detection (SOD) has attracted an increasing amount of research attention over the years. Recent advances in SOD are predominantly led by deep learning-based solutions (named deep SOD). To enable in-depth understanding of deep SOD, in this paper, we provide a comprehensive survey covering various aspects, ranging from algorithm taxonomy to unsolved issues. In particular, we first review deep SOD algorithms from different perspectives, including network architecture, level of supervision, learning paradigm, and object-/instance-level detection. Following that, we summarize and analyze existing SOD datasets and evaluation metrics. Then, we benchmark a large group of representative SOD models, and provide detailed analyses of the comparison results. Moreover, we study the performance of SOD algorithms under different attribute settings, which has not been thoroughly explored previously, by constructing a novel SOD dataset with rich attribute annotations covering various salient object types, challenging factors, and scene categories. We further analyze, for the first time in the field, the robustness of SOD models to random input perturbations and adversarial attacks. We also look into the generalization and difficulty of existing SOD datasets. Finally, we discuss several open issues of SOD and outline future research directions. All the saliency prediction maps, our constructed dataset with annotations, and codes for evaluation are publicly available at https://github.com/wenguanwang/SODsurvey .
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2021.3051099