Fast Physically Correct Refocusing for Sparse Light Fields Using Block-Based Multi-Rate View Interpolation

• Published in: IEEE transactions on image processing 2017-02, Vol.26 (2), p.603-618
• Main Authors: Huang, Chao-Tsung, Wang, Yu-Wen, Huang, Li-Ren, Chin, Jui, Chen, Liang-Gee
• Format: Article
• Language: English
• Subjects: Acceleration; Apertures; Cameras; Complexity theory; Digital refocusing; fast algorithm; Interpolation; Kernel; light field; Two dimensional displays; view interpolation
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2016.2631904

Digital refocusing has a tradeoff between complexity and quality when using sparsely sampled light fields for lowstorage applications. In this paper, we propose a fast physically correct refocusing algorithm to address this issue in a twofold way. First, view interpolation is adopted to provide photorealistic quality at infocus-defocus hybrid boundaries. Regarding its conventional high complexity, we devised a fast line-scan method specifically for refocusing, and its 1D kernel can be 30× faster than the benchmark View Synthesis Reference Software (VSRS)1D-Fast. Second, we propose a block-based multi-rate processing flow for accelerating purely infocused or defocused regions, and a further 3-34× speedup can be achieved for high-resolution images. All candidate blocks of variable sizes can interpolate different numbers of rendered views and perform refocusing in different subsampled layers. To avoid visible aliasing and block artifacts, we determine these parameters and the simulated aperture filter through a localized filter response analysis using defocus blur statistics. The final quadtree block partitions are then optimized in terms of computation time. Extensive experimental results are provided to show superior refocusing quality and fast computation speed. In particular, the run time is comparable with the conventional single-image blurring, which causes serious boundary artifacts.