Reconstruct, Rasterize and Backprop: Dense shape and pose estimation from a single image

This paper presents a new system to obtain dense object reconstructions along with 6-DoF poses from a single image. Geared towards high fidelity reconstruction, several recent approaches leverage implicit surface representations and deep neural networks to estimate a 3D mesh of an object, given a si...

Full description

Saved in:
Bibliographic Details
Main Authors: Pokale, Aniket, Aggarwal, Aditya, Jatavallabhula, Krishna Murthy, Madhava Krishna, K.
Format: Conference Proceeding
Language:English
Subjects:
Cameras
Image reconstruction
Neural networks
Shape
Simultaneous localization and mapping
Task analysis
Three-dimensional displays
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper presents a new system to obtain dense object reconstructions along with 6-DoF poses from a single image. Geared towards high fidelity reconstruction, several recent approaches leverage implicit surface representations and deep neural networks to estimate a 3D mesh of an object, given a single image. However, all such approaches recover only the shape of an object; the reconstruction is often in a canonical frame, unsuitable for downstream robotics tasks. To this end, we leverage recent advances in differentiable rendering (in particular, rasterization) to close the loop with 3D reconstruction in camera frame.We demonstrate that our approach-dubbed reconstruct, rasterize and backprop (RRB)-achieves significantly lower pose estimation errors compared to prior art, and is able to recover dense object shapes and poses from imagery. We further extend our results to an (offline) setup, where we demonstrate a dense monocular object-centric egomotion estimation system.
ISSN:2160-7516
DOI:10.1109/CVPRW50498.2020.00028