Deep Gaussian Scale Mixture Prior for Image Reconstruction

Image reconstruction from partial observations has attracted increasing attention. Conventional image reconstruction methods with hand-crafted priors often fail to recover fine image details due to the poor representation capability of the hand-crafted priors. Deep learning methods attack this probl...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2023-09, Vol.45 (9), p.1-17
Main Authors: Huang, Tao, Yuan, Xin, Dong, Weisheng, Wu, Jinjian, Shi, Guangming
Format: Article
Language:English
Subjects:
Cellular communication
Deep learning
Gaussian scale mixture prior
GSM
Image coding
Image enhancement
Image reconstruction
Image resolution
Imaging
maximum a posterior
Mixtures
Reconstruction algorithms
single image super-resolution
spectral compressive imaging
Superresolution
swin transformer
Transformers
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Image reconstruction from partial observations has attracted increasing attention. Conventional image reconstruction methods with hand-crafted priors often fail to recover fine image details due to the poor representation capability of the hand-crafted priors. Deep learning methods attack this problem by directly learning mapping functions between the observations and the targeted images can achieve much better results. However, most powerful deep networks lack transparency and are nontrivial to design heuristically. This paper proposes a novel image reconstruction method based on the Maximum a Posterior (MAP) estimation framework using learned Gaussian Scale Mixture (GSM) prior. Unlike existing unfolding methods that only estimate the image means (i.e., the denoising prior) but neglected the variances, we propose characterizing images by the GSM models with learned means and variances through a deep network. Furthermore, to learn the long-range dependencies of images, we develop an enhanced variant based on the Swin Transformer for learning GSM models. All parameters of the MAP estimator and the deep network are jointly optimized through end-to-end training. Extensive simulation and real data experimental results on spectral compressive imaging and image super-resolution demonstrate that the proposed method outperforms existing state-of-the-art methods.
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2023.3265103