Visual attention prediction for Autism Spectrum Disorder with hierarchical semantic fusion

Visual attention for the diagnosis of Autism Spectrum Disorder (ASD) which is a kind of mental disorder has attracted the interests of increasing number of researchers. Although multiple visual attention prediction models have been proposed, this problem is still open. In this paper, considering the...

Full description

Saved in:
Bibliographic Details
Published in:Signal processing. Image communication 2021-04, Vol.93, p.116186, Article 116186
Main Authors: Fang, Yuming, Zhang, Haiyan, Zuo, Yifan, Jiang, Wenhui, Huang, Hanqin, Yan, Jiebin
Format: Article
Language:English
Subjects:
Autism
Autism Spectrum Disorder
Deep learning
Feature extraction
Hierarchical semantic fusion
Mental disorders
Modules
Prediction models
Salience
Semantics
Visual attention
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Visual attention for the diagnosis of Autism Spectrum Disorder (ASD) which is a kind of mental disorder has attracted the interests of increasing number of researchers. Although multiple visual attention prediction models have been proposed, this problem is still open. In this paper, considering the shift of visual attention, we propose that an image can be viewed as a pseudo sequence. Besides, we propose a novel visual attention prediction method for ASD with hierarchical semantic fusion (ASD-HSF). Specifically, the proposed model mainly contains a Spatial Feature Module (SFM) and a Pseudo Sequential Feature Module (PSFM). SFM is designed to extract spatial semantic features with a fully convolutional network, while PSFM implemented by two Convolutional Long Short-Term Memory networks (ConvLSTMs) is applied to learn pseudo sequential features. And the outputs of these two modules are fused to extract the final saliency map which simultaneously includes spatial semantic information and pseudo sequential information. Experimental results show that the proposed model not only outperforms ten state-of-the-art general saliency prediction counterparts, but also reaches the first and the second ranks under four metrics and the rest ones of ASD saliency prediction respectively. •People with ASD have different understanding of natural scenes.•Considering visual attention shift, an image can be viewed as a pseudo sequence.•We present a novel visual attention prediction method for ASD.
ISSN:0923-5965
1879-2677
DOI:10.1016/j.image.2021.116186