Multiple-Layer Visibility Propagation-Based Synthetic Aperture Imaging through Occlusion

• Published in: Sensors (Basel, Switzerland) Switzerland), 2015-08, Vol.15 (8), p.18965-18984
• Main Authors: Yang, Tao, Li, Jing, Yu, Jingyi, Zhang, Yanning, Ma, Wenguang, Tong, Xiaomin, Yu, Rui, Ran, Lingyan
• Format: Article
• Language: English
• Subjects: all-in-focus synthetic aperture imaging; camera array; multiple-layer visibility propagation; occluded object imaging
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s150818965

Heavy occlusions in cluttered scenes impose significant challenges to many computer vision applications. Recent light field imaging systems provide new see-through capabilities through synthetic aperture imaging (SAI) to overcome the occlusion problem. Existing synthetic aperture imaging methods, however, emulate focusing at a specific depth layer, but are incapable of producing an all-in-focus see-through image. Alternative in-painting algorithms can generate visually-plausible results, but cannot guarantee the correctness of the results. In this paper, we present a novel depth-free all-in-focus SAI technique based on light field visibility analysis. Specifically, we partition the scene into multiple visibility layers to directly deal with layer-wise occlusion and apply an optimization framework to propagate the visibility information between multiple layers. On each layer, visibility and optimal focus depth estimation is formulated as a multiple-label energy minimization problem. The layer-wise energy integrates all of the visibility masks from its previous layers, multi-view intensity consistency and depth smoothness constraint together. We compare our method with state-of-the-art solutions, and extensive experimental results demonstrate the effectiveness and superiority of our approach.