Deep Coupled-Representation Learning for Sparse Linear Inverse Problems with Side Information

• Published in: arXiv.org 2019-07
• Main Authors: Tsiligianni, Evaggelia, Deligiannis, Nikos
• Format: Article
• Language: English
• Subjects: Algorithms; Color imagery; Complexity; Data recovery; Deep learning; Image reconstruction; Infrared imagery; Inverse problems; Machine learning; Mathematical models; Neural networks; Numerical analysis; Numerical methods; Optimization; Representations
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.1907.02511

In linear inverse problems, the goal is to recover a target signal from undersampled, incomplete or noisy linear measurements. Typically, the recovery relies on complex numerical optimization methods; recent approaches perform an unfolding of a numerical algorithm into a neural network form, resulting in a substantial reduction of the computational complexity. In this paper, we consider the recovery of a target signal with the aid of a correlated signal, the so-called side information (SI), and propose a deep unfolding model that incorporates SI. The proposed model is used to learn coupled representations of correlated signals from different modalities, enabling the recovery of multimodal data at a low computational cost. As such, our work introduces the first deep unfolding method with SI, which actually comes from a different modality. We apply our model to reconstruct near-infrared images from undersampled measurements given RGB images as SI. Experimental results demonstrate the superior performance of the proposed framework against single-modal deep learning methods that do not use SI, multimodal deep learning designs, and optimization algorithms.