Free-form scanning of non-planar appearance with neural trace photography

We propose neural trace photography, a novel framework to automatically learn high-quality scanning of non-planar, complex anisotropic appearance. Our key insight is that free-form appearance scanning can be cast as a geometry learning problem on unstructured point clouds, each of which represents a...

Full description

Saved in:
Bibliographic Details
Published in:ACM transactions on graphics 2021-08, Vol.40 (4), p.1-13, Article 124
Main Authors: Ma, Xiaohe, Kang, Kaizhang, Zhu, Ruisheng, Wu, Hongzhi, Zhou, Kun
Format: Article
Language:English
Subjects:
Computer graphics
Computing methodologies
Reflectance modeling
Rendering
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose neural trace photography, a novel framework to automatically learn high-quality scanning of non-planar, complex anisotropic appearance. Our key insight is that free-form appearance scanning can be cast as a geometry learning problem on unstructured point clouds, each of which represents an image measurement and the corresponding acquisition condition. Based on this connection, we carefully design a neural network, to jointly optimize the lighting conditions to be used in acquisition, as well as the spatially independent reconstruction of reflectance from corresponding measurements. Our framework is not tied to a specific setup, and can adapt to various factors in a data-driven manner. We demonstrate the effectiveness of our framework on a number of physical objects with a wide variation in appearance. The objects are captured with a light-weight mobile device, consisting of a single camera and an RGB LED array. We also generalize the framework to other common types of light sources, including a point, a linear and an area light.
ISSN:0730-0301
1557-7368
DOI:10.1145/3450626.3459679