Computational versus Psychophysical Bottom-Up Image Saliency: A Comparative Evaluation Study

• Published in: IEEE transactions on pattern analysis and machine intelligence 2011-11, Vol.33 (11), p.2131-2146
• Main Author: Toet, A.
• Format: Article
• Language: English
• Subjects: Adaptation model; Algorithms; Analytical models; Anatomical correlates of behavior; Applied sciences; Artificial intelligence; Assessments; Behavioral psychophysiology; Biological and medical sciences; Computation; Computational modeling; Computer science; control theory; systems; Computer Simulation; Conspicuity; Correlation; Engineering; Exact sciences and technology; Fixation, Ocular - physiology; Fundamental and applied biological sciences. Psychology; Human; Humans; image analysis; Mathematical models; Measurement; Models, Neurological; Pattern Recognition, Automated - statistics & numerical data; Pattern Recognition, Visual - physiology; Pattern recognition. Digital image processing. Computational geometry; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Psychophysics; Saliency; Sun; Visual; visual search; Visualization
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2011.53

The predictions of 13 computational bottom-up saliency models and a newly introduced Multiscale Contrast Conspicuity (MCC) metric are compared with human visual conspicuity measurements. The agreement between human visual conspicuity estimates and model saliency predictions is quantified through their rank order correlation. The maximum of the computational saliency value over the target support area correlates most strongly with visual conspicuity for 12 of the 13 models. A simple multiscale contrast model and the MCC metric both yield the largest correlation with human visual target conspicuity (>;0.84). Local image saliency largely determines human visual inspection and interpretation of static and dynamic scenes. Computational saliency models therefore have a wide range of important applications, like adaptive content delivery, region-of-interest-based image compression, video summarization, progressive image transmission, image segmentation, image quality assessment, object recognition, and content-aware image scaling. However, current bottom-up saliency models do not incorporate important visual effects like crowding and lateral interaction. Additional knowledge about the exact nature of the interactions between the mechanisms mediating human visual saliency is required to develop these models further. The MCC metric and its associated psychophysical saliency measurement procedure are useful tools to systematically investigate the relative contribution of different feature dimensions to overall visual target saliency.