Locally-guided neural denoising

• Published in: Graphics & visual computing 2022-12, Vol.7, p.200058, Article 200058
• Main Authors: Bode, Lukas, Merzbach, Sebastian, Kaltheuner, Julian, Weinmann, Michael, Klein, Reinhard
• Format: Article
• Language: English
• Subjects: Computer graphics; Denoising; Image processing; SVBRDF restoration
• ISSN: 2666-6294; 2666-6294
• DOI: 10.1016/j.gvc.2022.200058

Noise-like artifacts are common in measured or fitted data across various domains, e.g. photography, geometric reconstructions in terms of point clouds or meshes, as well as reflectance measurements and the respective fitting of commonly used reflectance models to them. State-of-the-art denoising approaches focus on specific noise characteristics usually observed in photography. However, these approaches do not perform well if data is corrupted with location-dependent noise. A typical example is the acquisition of heterogeneous materials, which leads to different noise levels due to different behavior of the components either during acquisition or during reconstruction. We address this problem by first automatically determining location-dependent noise levels in the input data and demonstrate that state-of-the-art denoising algorithms can usually benefit from this guidance with only minor modifications to their loss function or employed regularization mechanisms. To generate this information for guidance, we analyze patchwise variances and subsequently derive per-pixel importance values. We demonstrate the benefits of such locally-guided denoising at the examples of the Deep Image Prior method and the Self2Self method. [Display omitted] •Current denoising algorithms do not work well with spatially concentrated noise.•We propose to generate guidance images used to guide the denoising process.•Many denoising algorithms can benefit from guidance with minor modifications only.•The work is evaluated on two exemplary state-of-the-art denoising approaches.