Simultaneous estimation of super-resolved depth and all-in-focus images from a plenoptic camera

• Main Authors: Perez Nava, F., Luke, J.P.
• Format: Conference Proceeding
• Language: English
• Subjects: 3DTV; all-in-focus image; Cameras; depth; Field programmable gate arrays; focal stack; Focusing; Hardware; Image generation; Image resolution; Layout; Lenses; lightfield; Microoptics; Optical arrays; plenoptic camera; super-resolution
• ISSN: 2161-2021; 2161-203X
• DOI: 10.1109/3DTV.2009.5069675

This paper presents a new technique to simultaneously estimate the depth map and the all-in-focus image of a scene, both at super-resolution, from a plenoptic camera. A plenoptic camera uses a microlens array to measure the radiance and direction of all the light rays in a scene. It is composed of nxn microlenses and each of them generates a mxm image. Previous approaches to the depth and all-in- focus estimation problem processed the plenoptic image, generated a nxnxm focal stack, and were able to obtain a nxn depth map and all-in-focus image of the scene. This is a major drawback of the plenoptic camera approach to 3DTV since the total resolution of the camera n 2 m 2 is divided by m 2 to obtain a final resolution of n 2 pixels. In our approach we propose a new super-resolution focal stack that is combined with multiview depth estimation. This technique allows a theoretical resolution of approximately n 2 m 2 /4 pixels. This is an o(m 2 ) increment over previous approaches. From a practical point of view, in typical scenes we are able to increase 25 times the resolution of previous techniques. The time complexity of the algorithm makes possible to obtain real-time processing for 3DTV using appropriate hardware (GPU's or FPGA's) so it could be used in plenoptic video-cameras.