Geometry-assisted multi-representation view reconstruction network for Light Field image angular super-resolution

• Published in: Knowledge-based systems 2023-05, Vol.267, p.110390, Article 110390
• Main Authors: Liu, Deyang, Tong, Zaidong, Huang, Yan, Chen, Yilei, Zuo, Yifan, Fang, Yuming
• Format: Article
• Language: English
• Subjects: Angular reconstruction; Angular super-resolution; Light field image; Multi-representation; Reconstruction network
• ISSN: 0950-7051; 1872-7409
• DOI: 10.1016/j.knosys.2023.110390

Light Field (LF) imaging enables many attractive applications since scene angular and spatial information can be captured simultaneously. However, the limited angular resolution hinders its development. To alleviate this problem, in this paper, we put forward a geometry-assisted multi-representation view reconstruction network for LF angular super-resolution. There are two stages in our method, namely multi-representation view reconstruction and geometry-assisted refinement. The former fully utilizes the LF lenslet image, sub-aperture image array, and pseudo video sequence representations to synthesize a high-angular-resolution dense LF image stack by adequately exploring the structural characteristics of such three representations. The latter aims to exploit the intra-LF spatial–angular information and inter-LF geometry information contained in the derived dense LF image stack to further promote the reconstruction performance. Specifically, a bidirectional view stack structure is constituted first, which is then fed into the proposed geometry-aware refinement network to allow adequate interactions between the intra-LF spatial–angular information and inter-LF geometry information. Comprehensive angular reconstruction experiments on both real-world and synthetic LF scenes validate the effectiveness of the proposed method. Furthermore, extensible applications on scene depth estimation also demonstrate that the proposed method can recover more texture details.