Autolabeling 3D Objects With Differentiable Rendering of SDF Shape Priors

We present an automatic annotation pipeline to recover 9D cuboids and 3D shapes from pre-trained off-the-shelf 2D detectors and sparse LIDAR data. Our autolabeling method solves an ill-posed inverse problem by considering learned shape priors and optimizing geometric and physical parameters. To addr...

Full description

Saved in:
Bibliographic Details
Main Authors: Zakharov, Sergey, Kehl, Wadim, Bhargava, Arjun, Gaidon, Adrien
Format: Conference Proceeding
Language:English
Subjects:
Cascading style sheets
Pipelines
Rendering (computer graphics)
Shape
Solid modeling
Three-dimensional displays
Two dimensional displays
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present an automatic annotation pipeline to recover 9D cuboids and 3D shapes from pre-trained off-the-shelf 2D detectors and sparse LIDAR data. Our autolabeling method solves an ill-posed inverse problem by considering learned shape priors and optimizing geometric and physical parameters. To address this challenging problem, we apply a novel differentiable shape renderer to signed distance fields (SDF), leveraged together with normalized object coordinate spaces (NOCS). Initially trained on synthetic data to predict shape and coordinates, our method uses these predictions for projective and geometric alignment over real samples. Moreover, we also propose a curriculum learning strategy, iteratively retraining on samples of increasing difficulty in subsequent self-improving annotation rounds. Our experiments on the KITTI3D dataset show that we can recover a substantial amount of accurate cuboids, and that these autolabels can be used to train 3D vehicle detectors with state-of-the-art results.
ISSN:2575-7075
DOI:10.1109/CVPR42600.2020.01224