LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories

Semantic annotations are indispensable to train or evaluate perception models, yet very costly to acquire. This work introduces a fully automated 2D/3D labeling framework that, without any human intervention, can generate labels for RGB-D scans at equal (or better) level of accuracy than comparable...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-11
Main Authors: Weder, Silvan, Blum, Hermann, Engelmann, Francis, Pollefeys, Marc
Format: Article
Language:English
Subjects:
Annotations
Datasets
Labeling
Labels
Semantics
Three dimensional models
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Semantic annotations are indispensable to train or evaluate perception models, yet very costly to acquire. This work introduces a fully automated 2D/3D labeling framework that, without any human intervention, can generate labels for RGB-D scans at equal (or better) level of accuracy than comparable manually annotated datasets such as ScanNet. Our approach is based on an ensemble of state-of-the-art segmentation models and 3D lifting through neural rendering. We demonstrate the effectiveness of our LabelMaker pipeline by generating significantly better labels for the ScanNet datasets and automatically labelling the previously unlabeled ARKitScenes dataset. Code and models are available at https://labelmaker.org
ISSN:2331-8422