Frequency-Domain Deep Guided Image Denoising

• Published in: IEEE transactions on multimedia 2023-01, Vol.25, p.1-15
• Main Authors: Sheng, Zehua, Liu, Xiongwei, Cao, Si-Yuan, Shen, Hui-Liang, Zhang, Huaqi
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; convolutional neural network; Decomposition; Filtering; Frequency decomposition; Frequency domain analysis; guided image denoising; Image denoising; Image edge detection; Image restoration; Noise levels; Noise measurement; Noise reduction; Noise sensitivity; Optimization; Smoothness; Visualization
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2022.3214375

Despite the tremendous advances in image denoising techniques, there still remains a challenge to restore a clean image with salient structures based only on one single noisy observation, especially at high noise levels. In this work, to address this issue, we propose a novel frequency-domain guided denoising algorithm to conduct denoising with the help of an additional well-aligned guidance image. Thanks to the structural correlation between the input paired images, the frequency characteristics of the guidance image can indicate whether the frequency coefficients of the noisy target image are contributed by noise or textures. Therefore, the explicit frequency decomposition enables our model to avoid over-smoothing those detailed contents when removing noise. However, as two input images are usually captured in different fields, their structures are not always consistent with each other. Therefore, we model guided denoising with an optimization problem which takes into account both the representation model with respect to the guidance image and the structural fidelity to the noisy target image. Based on this, we further design a convolutional neural network, called as FGDNet, to explore the optimal solution. Due to the visual masking phenomenon, the human visual system is sensitive to noise in the flat areas, but may not perceive noise around edges or textures. Therefore, to handle a denoising task, we expect to remove as much noise as possible to guarantee the spatial smoothness of the flat contents, while preserving the high-frequency structures as well. Through frequency decomposition, our model can process the low-frequency and high-frequency contents separately with different denoising schemes. In the training stage, we also adopt a frequency-relevant loss function to optimize the network. Experimental results show that, compared with the state-of-the-art guided and non-guided denoising algorithms, our FGDNet can achieve higher denoising accuracy and also better visual quality in both flat and texture-rich regions. Source codes are available at https://github.com/lustrouselixir/FGDNet .