Exploring 3D-aware Latent Spaces for Efficiently Learning Numerous Scenes

We present a method enabling the scaling of NeRFs to learn a large number of semantically-similar scenes. We combine two techniques to improve the required training time and memory cost per scene. First, we learn a 3D-aware latent space in which we train Tri-Plane scene representations, hence reduci...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-05
Main Authors: Schnepf, Antoine, Kassab, Karim, Franceschi, Jean-Yves, Caraffa, Laurent, Vasile, Flavian, Jeremie, Mary, Comport, Andrew, Gouet-Brunet, Valérie
Format: Article
Language:English
Subjects:
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a method enabling the scaling of NeRFs to learn a large number of semantically-similar scenes. We combine two techniques to improve the required training time and memory cost per scene. First, we learn a 3D-aware latent space in which we train Tri-Plane scene representations, hence reducing the resolution at which scenes are learned. Moreover, we present a way to share common information across scenes, hence allowing for a reduction of model complexity to learn a particular scene. Our method reduces effective per-scene memory costs by 44% and per-scene time costs by 86% when training 1000 scenes. Our project page can be found at https://3da-ae.github.io .
ISSN:2331-8422