3D reconstruction under light ray distortion from parametric focal cameras

• Published in: Pattern recognition letters 2019-06, Vol.124, p.91-99
• Main Authors: Morinaka, Satoshi, Sakaue, Fumihiko, Sato, Jun, Ishimaru, Kazuhisa, Kawasaki, Naoki
• Format: Article
• Language: English
• Subjects: Cameras; Distortion; Mathematical models; Parameters; Raindrops; Reconstruction
• ISSN: 0167-8655; 1872-7344
• DOI: 10.1016/j.patrec.2018.11.007

•We can reconstruct 3D objects under unknown refractive media, such as curved glass.•We can recover not only the 3D shape, but also the refractive properties of the media.•We can recover more accurate 3D points by using sequential images. In this paper, we propose a new method for reconstructing 3D objects under light ray distortion caused by unknown refractive media. The proposed method can reconstruct 3D scene, even if light rays projected into the cameras are refracted by unknown refractive media, such as glasses and raindrops. For this objective, we represent light ray projection of multiple cameras by using a pair of planes shared by the multiple cameras in the scene. By using this model, intrinsic and extrinsic camera parameters as well as the refractive properties of the refractive media can be represented efficiently. By using the newly defined camera model, we propose a method for recovering 3D points and camera parameters with refractive properties simultaneously. The experimental results show the efficiency of the proposed camera model and reconstruction method.