A Novel Rank Learning Based No-Reference Image Quality Assessment Method

• Published in: IEEE transactions on multimedia 2022-01, Vol.24, p.4197-4211
• Main Authors: Ou, Fu-Zhao, Wang, Yuan-Gen, Li, Jin, Zhu, Guopu, Kwong, Sam
• Format: Article
• Language: English
• Subjects: Artificial neural networks; authentic distortion; convolutional neural network; Convolutional neural networks; Datasets; Deep learning; Distortion; Feature extraction; Heuristic methods; Image quality; Lower bounds; No-reference image quality assessment; Prediction models; Predictive models; Quality assessment; rank learning; Ranking; Source code; synthetic distortion; Training; Transform coding; Weber-Fechner law
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2021.3114551

Recently, applying deep learning to no-reference image quality assessment (NR-IQA) has received significant attention. Especially in the last five years, an increasing interest has been drawn to the studies of rank learning since it can help mitigate the problem of small IQA datasets. However, on one hand, existing rank learning is not suitable for the authentically distorted images due to the lack of generated rank samples. On the other hand, the output of existing rank loss functions is uncontrollable, resulting in reduced performance. Motivated by these two limitations, we propose a novel rank learning based NR-IQA method, termed controllable list-wise ranking IQA (CLRIQA) in this paper. To be specific, we first present an imaging-heuristic approach, in which the over- and under-exposure is formulated as an inverse of the Weber-Fechner law, and fusion strategy and compression are adopted, to simulate the authentic distortion and generate the rank image samples. These samples are label-free yet associated with quality ranking information. Then we design a controllable list-wise ranking (CLR) loss function by setting an upper and lower bound of rank range and introducing an adaptive margin to tune rank interval. Finally, both the generated rank samples and proposed CLR are used to pre-train a convolutional neural network. Moreover, to obtain a more accurate prediction model, we take advantage of the IQA datasets to fine-tune the pre-trained network further. Various experiments are conducted on the IQA benchmark datasets, and experimental results demonstrate the effectiveness of the proposed CLRIQA method. The source code and network model can be downloaded at the following web address: https://github.com / GZHU-DVL / CLRIQA .