Top-Down Visual Saliency via Joint CRF and Dictionary Learning

• Published in: IEEE transactions on pattern analysis and machine intelligence 2017-03, Vol.39 (3), p.576-588
• Main Authors: Yang, Jimei, Yang, Ming-Hsuan
• Format: Article
• Language: English
• Subjects: Computational modeling; Context; Context modeling; Dictionaries; dictionary learning and conditional random fields; fixation prediction; Image coding; Machine learning; Performance enhancement; Prediction algorithms; Predictive models; Stochastic processes; top-down visual saliency; Visual fields; Visual saliency; Visualization
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2016.2547384

Top-down visual saliency is an important module of visual attention. In this work, we propose a novel top-down saliency model that jointly learns a Conditional Random Field (CRF) and a visual dictionary. The proposed model incorporates a layered structure from top to bottom: CRF, sparse coding and image patches. With sparse coding as an intermediate layer, CRF is learned in a feature-adaptive manner; meanwhile with CRF as the output layer, the dictionary is learned under structured supervision. For efficient and effective joint learning, we develop a max-margin approach via a stochastic gradient descent algorithm. Experimental results on the Graz-02 and PASCAL VOC datasets show that our model performs favorably against state-of-the-art top-down saliency methods for target object localization. In addition, the dictionary update significantly improves the performance of our model. We demonstrate the merits of the proposed top-down saliency model by applying it to prioritizing object proposals for detection and predicting human fixations.