Blind image quality assessment with semantic information

• Published in: Journal of visual communication and image representation 2019-01, Vol.58, p.195-204
• Main Authors: Ji, Weiping, Wu, Jinjian, Shi, Guangming, Wan, Wenfei, Xie, Xuemei
• Format: Article
• Language: English
• Subjects: Human perception; No-reference image quality assessment; Semantic network; Spatial semantics; Structural semantics
• ISSN: 1047-3203; 1095-9076
• DOI: 10.1016/j.jvcir.2018.11.038

•A novel blind image quality assessment is proposed.•The proposed method evaluates distorted image from a completely new perspective of human subjective perception.•The experiment results demonstrate the superiority of the proposed method. No-reference (NR) image quality assessment (IQA) aims to evaluate the quality of an image without reference image, which is greatly desired in the automatic visual signal processing system. Distortions degrade the visual contents and affect the semantics acquisition during the process of human perception. Although the existing methods evaluate the quality of images based on the structure, texture, or statistical characteristics, and deliver high quality prediction accuracy, they do not take the spatial semantics into account. From the perspective of human perception, distortions decrease the structural semantics that represent the structural information, and disturb the spatial semantics that describe the contents of images. Therefore, we attempt to measure the image quality by its degradation of semantics in an image. To extract the semantics of an image, a semantic network is proposed. The network contains convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) that correspond to structural semantics and spatial semantics, respectively. CNN can be regarded as a coarse imitation of human visual mechanism to obtain the structural information, and LSTM can express the contents of an image. Then, by measuring the degradations of different semantics on images, a novel NR IQA is introduced. The proposed approach is evaluated on the databases of LIVE, CSIQ, TID2013, and LIVE multiply distorted database as well as LIVE in the wild image quality challenge database, and the results show superior performance to other state-of-the-art NR IQA methods. Furthermore, we explore the generalization capability of the proposed approach, and the experimental results indicate the proposed approach has a high robustness.