HMS-Net: Hierarchical Multi-Scale Sparsity-Invariant Network for Sparse Depth Completion

Dense depth cues are important and have wide applications in various computer vision tasks. In autonomous driving, LIDAR sensors are adopted to acquire depth measurements around the vehicle to perceive the surrounding environments. However, depth maps obtained by LIDAR are generally sparse because o...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on image processing 2020-01, Vol.29, p.3429-3441
Main Authors: Huang, Zixuan, Fan, Junming, Cheng, Shenggan, Yi, Shuai, Wang, Xiaogang, Li, Hongsheng
Format: Article
Language:English
Subjects:
Benchmark testing
Benchmarks
Coders
Computer vision
Convolution
convolutional neural network
Depth completion
Encoders-Decoders
Fans
Feature maps
Invariants
Laser radar
Lidar
Neural networks
Sensors
Sparsity
sparsity-invariant operations
Task analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Dense depth cues are important and have wide applications in various computer vision tasks. In autonomous driving, LIDAR sensors are adopted to acquire depth measurements around the vehicle to perceive the surrounding environments. However, depth maps obtained by LIDAR are generally sparse because of its hardware limitation. The task of depth completion attracts increasing attention, which aims at generating a dense depth map from an input sparse depth map. To effectively utilize multi-scale features, we propose three novel sparsity-invariant operations, based on which, a sparsity-invariant multi-scale encoder-decoder network (HMS-Net) for handling sparse inputs and sparse feature maps is also proposed. Additional RGB features could be incorporated to further improve the depth completion performance. Our extensive experiments and component analysis on two public benchmarks, KITTI depth completion benchmark and NYU-depth-v2 dataset, demonstrate the effectiveness of the proposed approach. As of Aug. 12th, 2018, on KITTI depth completion leaderboard, our proposed model without RGB guidance ranks 1st among all peer-reviewed methods without using RGB information, and our model with RGB guidance ranks 2nd among all RGB-guided methods.
ISSN:1057-7149
1941-0042
DOI:10.1109/TIP.2019.2960589