Structural similarity-based Bi-representation through true noise level for noise-robust face super-resolution

• Published in: Multimedia tools and applications 2023-07, Vol.82 (17), p.26255-26288
• Main Authors: Nagar, Surendra, Jain, Ankush, Singh, Pramod Kumar, Kumar, Ajay
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Computer Communication Networks; Computer Science; Computer vision; Data Structures and Information Theory; Datasets; Dictionaries; Hallucinations; Image enhancement; Image reconstruction; Image resolution; Image restoration; Methods; Multimedia; Multimedia Information Systems; Neural networks; Noise levels; Pixels; Representations; Robustness; Similarity; Sparsity; Special Purpose and Application-Based Systems; Training
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-022-14325-6

In today’s real-world scenarios’ of computer vision applications, enhancing low-resolution (LR) facial images corrupted with unwanted noise effects is very challenging as the uneven noise distribution severely distorts these images’ local structure. This paper proposes a novel noise-robust face super-resolution (SR) method, namely structural similarity-based Bi-representation SR (SS-BRSR), to tackle this problem. It firstly estimates the true noise level in the corrupted LR face through the novel noise-level estimation algorithm. Afterward, it employs a robust deep-convolutional neural network, namely DnCNN, to separate the pixel-wise noise from the noisy LR face image. This network produces two outputs: (i) a residual image and (ii) a smooth LR face image. We utilize the first output for pixel-wise updating the entire LR training images, making the structural similarity between the test and the training LR images. Further, for SR reconstruction, the SS-BRSR consists of two patch representation components that individually reconstruct the HR faces corresponding to the initial noisy LR and smooth LR face images. Besides, in both the components, the Gradient and Laplacian features-based learning scheme is incorporated to preserve the discriminative facial features in the SR reconstruction. Here, the first component substantially minimizes the reconstruction error due to noise, and the second component compensates for the lost detail in the LR face image. The target HR face image is restored by taking the appropriate proportions of obtained HR face images from each component. The experimental results on different face datasets justify the SS-BRSR method’s superiority over the state-of-the-art face SR methods. For instance, the quantitative performance (in terms of PNSR and SSIM) of the proposed method over the state-of-the-art RLENR and DFDNet methods gained an improvement of [1%, 1.5%, 2.5%, 2.5%] under [10, 15, 20, 30] noise-level densities, and [1%, 1.5%, 2%, 1.5%] under [10, 15, 20, 30] noise-level densities, respectively, for the standard CelebA and FEI datasets.