DEMVSNet: Denoising and depth inference for unstructured multi‐view stereo on noised images

Most deep‐learning‐based multi‐view stereo series studies are concerned with improving the depth prediction accuracy of noise‐free images. However, it is difficult to obtain off‐the‐set clean images in practice and 3D convolutional neural networks require a lot of computing resources. To make full u...

Full description

Saved in:
Bibliographic Details
Published in:IET computer vision 2022-10, Vol.16 (7), p.570-580
Main Authors: Han, Jiawei, Chen, Xiaomei, Zhang, Yongtian, Hou, Weimin, Hu, Zibo
Format: Article
Language:English
Subjects:
Algorithms
Artificial neural networks
Computation
computer vision
Datasets
Feature maps
Inference
Lighting
Methods
neural net architecture
Neural networks
Noise prediction
Noise reduction
Pixels
random noise
Regularization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Most deep‐learning‐based multi‐view stereo series studies are concerned with improving the depth prediction accuracy of noise‐free images. However, it is difficult to obtain off‐the‐set clean images in practice and 3D convolutional neural networks require a lot of computing resources. To make full use of its computing power, different types of information can be processed simultaneously in the network. For these two issues, this paper proposes a novel multi‐stage network architecture to address depth inference and denoising simultaneously. Specifically, 2D feature maps are first converted into 3D cost volumes containing pixel information and depth information through differentiable homography and Gaussian probability mapping. Then, the cost volume is input into the regularisation module in each network stage to obtain the predicted probability volumes. Furthermore, simple static weights lead to training failure, and it is necessary to dynamically adjust the loss function by gradient normalisation. The proposed method can dispose of pixel information and depth information simultaneously and both reach an excellent level. Extensive experimental results show that the authors’ work surpasses the state‐of‐the‐art denoising on the DTU dataset (adding Gaussian–Poisson noise) and is more robust to noise images in depth inference.
ISSN:1751-9632
1751-9640
DOI:10.1049/cvi2.12102