Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis

While surface-based view synthesis algorithms are appealing due to their low computational requirements, they often struggle to reproduce thin structures. In contrast, more expensive methods that model the scene's geometry as a volumetric density field (e.g. NeRF) excel at reconstructing fine g...

Full description

Saved in:
Bibliographic Details
Published in:ACM transactions on graphics 2024-07, Vol.43 (4), p.1-14, Article 149
Main Authors: Reiser, Christian, Garbin, Stephan, Srinivasan, Pratul, Verbin, Dor, Szeliski, Richard, Mildenhall, Ben, Barron, Jonathan, Hedman, Peter, Geiger, Andreas
Format: Article
Language:English
Subjects:
Appearance and texture representations
Artificial intelligence
Computer graphics
Computer vision
Computer vision problems
Computer vision representations
Computing methodologies
Rasterization
Reconstruction
Rendering
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:While surface-based view synthesis algorithms are appealing due to their low computational requirements, they often struggle to reproduce thin structures. In contrast, more expensive methods that model the scene's geometry as a volumetric density field (e.g. NeRF) excel at reconstructing fine geometric detail. However, density fields often represent geometry in a "fuzzy" manner, which hinders exact localization of the surface. In this work, we modify density fields to encourage them to converge towards surfaces, without compromising their ability to reconstruct thin structures. First, we employ a discrete opacity grid representation instead of a continuous density field, which allows opacity values to discontinuously transition from zero to one at the surface. Second, we anti-alias by casting multiple rays per pixel, which allows occlusion boundaries and subpixel structures to be modelled without using semi-transparent voxels. Third, we minimize the binary entropy of the opacity values, which facilitates the extraction of surface geometry by encouraging opacity values to binarize towards the end of training. Lastly, we develop a fusion-based meshing strategy followed by mesh simplification and appearance model fitting. The compact meshes produced by our model can be rendered in real-time on mobile devices and achieve significantly higher view synthesis quality compared to existing mesh-based approaches. Our interactive webdemo is available at https://binary-opacity-grid.github.io.
ISSN:0730-0301
1557-7368
DOI:10.1145/3658130