Hierarchical feature fusion network for light field spatial super-resolution

• Published in: The Visual computer 2023, Vol.39 (1), p.267-279
• Main Authors: Hua, Xiyao, Wang, Minghui, Su, Boni, Liu, Xia
• Format: Article
• Language: English
• Subjects: Ablation; Aperture imaging; Artificial Intelligence; Cameras; Computer Graphics; Computer Science; Deep learning; Image degradation; Image Processing and Computer Vision; Image resolution; Image restoration; Methods; Neural networks; Optimization; Original Article; Performance evaluation
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-021-02327-8

Light field (LF) spatial super-resolution (SR) aims to restore a high-resolution LF image from a degraded low-resolution one. However, due to the complexity of high-dimensional LF images, the existing LF spatial SR methods failed to fully incorporate the correlation between sub-aperture images of the LF. To mitigate this problem, we propose a hierarchical feature fusion network (LF-HFNet) for LF spatial SR with two novel components, namely feature interaction module and residual spatial and channel attention block. By cascading several residual spatial-angular separable convolution blocks with concatenation connections, the former can fully utilize the hierarchical and complementary information between SAIs. And the latter can adaptively rescale the feature responses for emphasizing informative features. Experimental results on both synthetic and real-world LF datasets demonstrate that the proposed method outperforms other state-of-the-art methods with higher PSNR/SSIM.