A new representation of scene layout improves saliency detection in traffic scenes

• Published in: Expert systems with applications 2022-05, Vol.193, p.116425, Article 116425
• Main Authors: He, De-Huai, Yang, Kai-Fu, Wan, Xue-Mei, Xiao, Fen, Yan, Hong-Mei, Li, Yong-Jie
• Format: Article
• Language: English
• Subjects: Algorithms; Datasets; Deep learning; Eye movements; Fixation; Layouts; Machine learning; Nighttime traffic scenes; Representations; Roadsides; Robustness; Salience; Saliency detection; Scene layout; Vanishing point; Visual tasks
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2021.116425

It has been well established that scene context plays an important role in directing visual attention. A robust representation of scene layout is expected to facilitate further analysis of traffic scenes, especially under challenging visual conditions like nighttime and/or on a small-scale dataset. In this work, a new layout representation for traffic scenes is proposed and applied to the popular visual task of saliency detection. First, a general layout representation for traffic scenes is defined as a combination of an original point (Vanishing Point, VP) and two axes along roadsides. Then, a simple algorithm is proposed to build a robust layout representation for traffic scenes, along with an improved VP detection method. Finally, to verify the contribution of the proposed layout representation, a layout-guided saliency detection framework is proposed to improve existing methods by integrating layout-guided prior learned from human fixations collected with an eye-tracking recorder. Experimental results show that the proposed layout representation can significantly improve the performance of various saliency detection methods including classical bottom-up methods and deep-learning-based methods. Moreover, compared with the deep-learning methods, the layout-guided method has an obvious advantage in terms of robustness when only a small-scale dataset is available and under varying visual scenes. •A new layout representation for traffic scene is defined.•A layout extraction method for traffic scenes is proposed.•An improved vanishing point detection method for nighttime traffic scenes.•The proposed layout representation improves saliency detection for traffic scenes.