HybridNeRF: Efficient Neural Rendering via Adaptive Volumetric Surfaces

Neural radiance fields provide state-of-the-art view synthesis quality but tend to be slow to render. One reason is that they make use of volume rendering, thus requiring many samples (and model queries) per ray at render time. Although this representation is flexible and easy to optimize, most real...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-03
Main Authors: Turki, Haithem, Agrawal, Vasu, Samuel Rota Bulò, Porzi, Lorenzo, Kontschieder, Peter, Ramanan, Deva, Zollhöfer, Michael, Richardt, Christian
Format: Article
Language:English
Subjects:
Datasets
Rendering
Representations
Synthesis
Virtual reality
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Neural radiance fields provide state-of-the-art view synthesis quality but tend to be slow to render. One reason is that they make use of volume rendering, thus requiring many samples (and model queries) per ray at render time. Although this representation is flexible and easy to optimize, most real-world objects can be modeled more efficiently with surfaces instead of volumes, requiring far fewer samples per ray. This observation has spurred considerable progress in surface representations such as signed distance functions, but these may struggle to model semi-opaque and thin structures. We propose a method, HybridNeRF, that leverages the strengths of both representations by rendering most objects as surfaces while modeling the (typically) small fraction of challenging regions volumetrically. We evaluate HybridNeRF against the challenging Eyeful Tower dataset along with other commonly used view synthesis datasets. When comparing to state-of-the-art baselines, including recent rasterization-based approaches, we improve error rates by 15-30% while achieving real-time framerates (at least 36 FPS) for virtual-reality resolutions (2Kx2K).
ISSN:2331-8422