Generalizable Novel-View Synthesis Using a Stereo Camera

In this paper, we propose the first generalizable view synthesis approach that specifically targets multi-view stereocamera images. Since recent stereo matching has demonstrated accurate geometry prediction, we introduce stereo matching into novel-view synthesis for high-quality geometry reconstruct...

Full description

Saved in:
Bibliographic Details
Main Authors: Lee, Haechan, Jin, Wonjoon, Baek, Seung-Hwan, Cho, Sunghyun
Format: Conference Proceeding
Language:English
Subjects:
Accuracy
Cameras
Computer vision
Estimation
Feature extraction
Generalization
Geometry
Neural radiance fields
Novel view synthesis
Stereo camera
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we propose the first generalizable view synthesis approach that specifically targets multi-view stereocamera images. Since recent stereo matching has demonstrated accurate geometry prediction, we introduce stereo matching into novel-view synthesis for high-quality geometry reconstruction. To this end, this paper proposes a novel framework, dubbed StereoNeRF, which integrates stereo matching into a NeRF-based generalizable view synthesis approach. StereoNeRF is equipped with three key components to effectively exploit stereo matching in novel-view synthesis: a stereo feature extractor, a depth-guided plane-sweeping, and a stereo depth loss. Moreover, we propose the StereoNVS dataset, the first multi-view dataset of stereocamera images, encompassing a wide variety of both real and synthetic scenes. Our experimental results demonstrate that StereoNeRF surpasses previous approaches in generalizable view synthesis.
ISSN:2575-7075
DOI:10.1109/CVPR52733.2024.00472