Joint demosaicking and denoising benefits from a two-stage training strategy

• Published in: Journal of computational and applied mathematics 2023-12, Vol.434, p.115330, Article 115330
• Main Authors: Guo, Yu, Jin, Qiyu, Morel, Jean-Michel, Zeng, Tieyong, Facciolo, Gabriele
• Format: Article
• Language: English
• Subjects: Computer Science; Convolutional neural networks; Demosaicking; Denoising; Pipeline; Residual; Signal and Image Processing
• ISSN: 0377-0427; 1879-1778
• DOI: 10.1016/j.cam.2023.115330

Image demosaicking and denoising are the first two key steps of the color image production pipeline. The classical processing sequence has for a long time consisted of applying denoising first, and then demosaicking. Applying the operations in this order leads to oversmoothing and checkerboard effects. Yet, it was difficult to change this order, because once the image is demosaicked, the statistical properties of the noise are dramatically changed and hard to handle by traditional denoising models. In this paper, we address this problem by a hybrid machine learning method. We invert the traditional color filter array (CFA) processing pipeline by first demosaicking and then denoising. Our demosaicking algorithm, trained on noiseless images, combines a traditional method and a residual convolutional neural network (CNN). This first stage retains all known information, which is the key point to obtain faithful final results. The noisy demosaicked image is then passed through a second CNN restoring a noiseless full-color image. This pipeline order completely avoids checkerboard effects and restores fine image detail. Although CNNs can be trained to solve jointly demosaicking–denoising end-to-end, we find that this two-stage training performs better and is less prone to failure. It is shown experimentally to improve on the state of the art, both quantitatively and in terms of visual quality.