Toward Top-Down Just Noticeable Difference Estimation of Natural Images

• Published in: IEEE transactions on image processing 2022, Vol.31, p.3697-3712
• Main Authors: Jiang, Qiuping, Liu, Zhentao, Wang, Shiqi, Shao, Feng, Lin, Weisi
• Format: Article
• Language: English
• Subjects: Adaptation models; Algorithms; critical perceptual lossless; Critical point; Data Compression - methods; Differential Threshold; distortion visibility; Distribution functions; Energy value; Estimation; Frequency-domain analysis; Humans; Image coding; Image compression; Just noticeable difference; Karhunen-Loéve transform; Masking; masking effect; Philosophical considerations; Predictive models; Visualization; Weibull distribution
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2022.3174398

Just noticeable difference (JND) of natural images refers to the maximum pixel intensity change magnitude that typical human visual system (HVS) cannot perceive. Existing efforts on JND estimation mainly dedicate to modeling the diverse masking effects in either/both spatial or/and frequency domains, and then fusing them into an overall JND estimate. In this work, we turn to a dramatically different way to address this problem with a top-down design philosophy. Instead of explicitly formulating and fusing different masking effects in a bottom-up way, the proposed JND estimation model dedicates to first predicting a critical perceptual lossless (CPL) counterpart of the original image and then calculating the difference map between the original image and the predicted CPL image as the JND map. We conduct subjective experiments to determine the critical points of 500 images and find that the distribution of cumulative normalized KLT coefficient energy values over all 500 images at these critical points can be well characterized by a Weibull distribution. Given a testing image, its corresponding critical point is determined by a simple weighted average scheme where the weights are determined by a fitted Weibull distribution function. The performance of the proposed JND model is evaluated explicitly with direct JND prediction and implicitly with two applications including JND-guided noise injection and JND-guided image compression. Experimental results have demonstrated that our proposed JND model can achieve better performance than several latest JND models. In addition, we also compare the proposed JND model with existing visual difference predicator (VDP) metrics in terms of the capability in distortion detection and discrimination. The results indicate that our JND model also has a good performance in this task. The code of this work are available at https://github.com/Zhentao-Liu/KLT-JND .