Context-aware saliency detection for image retargeting using convolutional neural networks

• Published in: Multimedia tools and applications 2021-03, Vol.80 (8), p.11917-11941
• Main Authors: Ahmadi, Mahdi, Karimi, Nader, Samavi, Shadrokh
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Computer Communication Networks; Computer Science; Context; Data Structures and Information Theory; Datasets; Image processing; Image segmentation; Multimedia Information Systems; Neural networks; Quality assessment; Salience; Semantic segmentation; Semantics; Special Purpose and Application-Based Systems
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-020-10185-0

Image retargeting is the task of making images capable of being displayed on screens with different sizes. This work should be done so that high-level visual information and low-level features such as texture remain as intact as possible to the human visual system. At the same time, the output image may have different dimensions. Thus, simple methods such as scaling and cropping are not adequate for this purpose. In recent years, researchers have tried to improve the existing retargeting methods, and they have introduced new ones. However, a specific method cannot be utilized to retarget all types of images. In other words, different images require different retargeting methods. Image retargeting has a close relationship to image saliency detection, which is a relatively new image processing task. Earlier saliency detection methods were based on local and global but low-level image information. These methods are called bottom-up processes. On the other hand, newer approaches are top-down and mixed methods that consider the high level and semantic knowledge of the image too. In this paper, we introduce the proposed methods in both saliency detection and retargeting. For the saliency detection, the use of image context and semantic segmentation are examined, and a novel mixed bottom-up and top-down saliency detection method is introduced. After saliency detection, a modified version of an existing retargeting technique is utilized for retargeting the images. The results suggest that the proposed image retargeting pipeline has excellent performance compared to other tested methods. Also, the subjective evaluations on the Pascal dataset can be used as a retargeting quality assessment dataset for further research.