Hyperspectral Recovery from RGB Images using Gaussian Processes

• Published in: IEEE transactions on pattern analysis and machine intelligence 2020-01, Vol.42 (1), p.100-113
• Main Authors: Akhtar, Naveed, Mian, Ajmal
• Format: Article
• Language: English
• Subjects: Cameras; Color; Color imagery; Gaussian process; Gaussian processes; Hyperspectral imaging; Measurement; Patches (structures); Reflectance; Signal processing; Smoothness; Spectra; spectral recovery; Training
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2018.2873729

We propose to recover spectral details from RGB images of known spectral quantization by modeling natural spectra under Gaussian Processes and combining them with the RGB images. Our technique exploits Process Kernels to model the relative smoothness of reflectance spectra, and encourages non-negativity in the resulting signals for better estimation of the reflectance values. The Gaussian Processes are inferred in sets using clusters of spatio-spectrally correlated hyperspectral training patches. Each set is transformed to match the spectral quantization of the test RGB image. We extract overlapping patches from the RGB image and match them to the hyperspectral training patches by spectrally transforming the latter. The RGB patches are encoded over the transformed Gaussian Processes related to those hyperspectral patches and the resulting image is constructed by combining the codes with the original processes. Our approach infers the desired Gaussian Processes under a fully Bayesian model inspired by Beta-Bernoulli Process, for which we also present the inference procedure. A thorough evaluation using three hyperspectral datasets demonstrates the effective extraction of spectral details from RGB images by the proposed technique.