Sparse spectral signal reconstruction for one proposed nine-band multispectral imaging system

• Published in: Mechanical systems and signal processing 2020-07, Vol.141, p.106627, Article 106627
• Main Authors: Sun, Bangyong, Zhao, Zhe, Xie, Dehong, Yuan, Nianzeng, Yu, Zhe, Chen, Fuwei, Cao, Congjun, de Dravo, Vincent Whannou
• Format: Article
• Language: English
• Subjects: Algorithms; Filter array; Guide filter; Image acquisition; Image demosaicking; Image quality; Image reconstruction; Interpolation; Mechanical systems; Multispectral imaging system; Object recognition; Performance evaluation; Sensor arrays; Signal reconstruction; Spectra; Spectral reflectance
• ISSN: 0888-3270; 1096-1216
• DOI: 10.1016/j.ymssp.2020.106627

•We proposed a new spectral signal reconstruction method for a multispectral imaging capturing system with nine spectral bands.•We design a new filter array in a 4×4 MSFA pattern and propose a multispectral demosaicking algorithm for recovering the sparse spectral data.•The experimental results demonstrate that our proposed demosaicking algorithm outperforms the other methods in PSNR, SSIM, and subjective evaluation. Multispectral filter array (MSFA) imaging with one single sensor is a portable and inexpensive means of acquiring spectral image which is widely used for object detection, material analysis and mechanical system diagnosis. The most challenging task for MSFA imaging is the multispectral demosaicking with the aim of reconstructing the captured raw/mosaic image, especially for the systems with many bands which result in higher sparseness of the raw data. In this paper, we present a 9-band MSFA imaging system in a repetitive 4 × 4 filter array on a single sensor, and propose a demosaicking algorithm for reconstructing the raw spectral image. Within the 4 × 4 MSFA pattern, the fifth spectral band takes up half of the total spatial position while the remaining eight bands occupy 1/16 respectively. To reconstruct the sparse raw data, we first recover the fifth band by propagating the neighboring sampled pixels to the unsampled position using the image gradients, and then employ the reconstructed fifth band as a guided image to demosaick the other bands with the guided filter and residual interpolation. Finally, we estimate the spectral reflectance values from the multispectral image and the characterization matrix. In the experiment, we evaluate the performance of the 9-band imaging system with the binary tree-based edge-sensing (BTES) algorithm, compressed sensing (CS) algorithm, and our proposed demosaicking algorithm. The experiment results demonstrate that our demosaicking algorithm not only outperforms BTES and CS algorithms in terms of objective image quality, e.g., PSNR values and spectral errors, but also reduces the demosaicking artifacts in terms of subjective evaluations.