DiffPoseNet: Direct Differentiable Camera Pose Estimation

Current deep neural network approaches for camera pose estimation rely on scene structure for 3D motion estimation, but this decreases the robustness and thereby makes cross-dataset generalization difficult. In contrast, classical approaches to structure from motion estimate 3D motion utilizing opti...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2022-03
Main Authors: Parameshwara, Chethan M, Gokul Hari, Fermüller, Cornelia, Sanket, Nitin J, Aloimonos, Yiannis
Format: Article
Language:English
Subjects:
Artificial neural networks
Cameras
Datasets
Machine learning
Motion simulation
Noise sensitivity
Optical flow (image analysis)
Optimization
Pose estimation
Three dimensional flow
Three dimensional motion
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Current deep neural network approaches for camera pose estimation rely on scene structure for 3D motion estimation, but this decreases the robustness and thereby makes cross-dataset generalization difficult. In contrast, classical approaches to structure from motion estimate 3D motion utilizing optical flow and then compute depth. Their accuracy, however, depends strongly on the quality of the optical flow. To avoid this issue, direct methods have been proposed, which separate 3D motion from depth estimation but compute 3D motion using only image gradients in the form of normal flow. In this paper, we introduce a network NFlowNet, for normal flow estimation which is used to enforce robust and direct constraints. In particular, normal flow is used to estimate relative camera pose based on the cheirality (depth positivity) constraint. We achieve this by formulating the optimization problem as a differentiable cheirality layer, which allows for end-to-end learning of camera pose. We perform extensive qualitative and quantitative evaluation of the proposed DiffPoseNet's sensitivity to noise and its generalization across datasets. We compare our approach to existing state-of-the-art methods on KITTI, TartanAir, and TUM-RGBD datasets.
ISSN:2331-8422