Camera Relocalization in Shadow-free Neural Radiance Fields

Camera relocalization is a crucial problem in computer vision and robotics. Recent advancements in neural radiance fields (NeRFs) have shown promise in synthesizing photo-realistic images. Several works have utilized NeRFs for refining camera poses, but they do not account for lighting changes that...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-05
Main Authors: Xu, Shiyao, Liu, Caiyun, Chen, Yuantao, Zhu, Zhenxin, Yan, Zike, Shi, Yongliang, Zhao, Hao, Zhou, Guyue
Format: Article
Language:English
Subjects:
Cameras
Computer vision
Lighting
Localization
Low pass filters
Optimization
Radiance
Robotics
Shadows
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Camera relocalization is a crucial problem in computer vision and robotics. Recent advancements in neural radiance fields (NeRFs) have shown promise in synthesizing photo-realistic images. Several works have utilized NeRFs for refining camera poses, but they do not account for lighting changes that can affect scene appearance and shadow regions, causing a degraded pose optimization process. In this paper, we propose a two-staged pipeline that normalizes images with varying lighting and shadow conditions to improve camera relocalization. We implement our scene representation upon a hash-encoded NeRF which significantly boosts up the pose optimization process. To account for the noisy image gradient computing problem in grid-based NeRFs, we further propose a re-devised truncated dynamic low-pass filter (TDLF) and a numerical gradient averaging technique to smoothen the process. Experimental results on several datasets with varying lighting conditions demonstrate that our method achieves state-of-the-art results in camera relocalization under varying lighting conditions. Code and data will be made publicly available.
ISSN:2331-8422