Learning a confidence measure in the disparity domain from O(1) features

• Published in: Computer vision and image understanding 2020-04, Vol.193, p.102905, Article 102905
• Main Authors: Poggi, Matteo, Tosi, Fabio, Mattoccia, Stefano
• Format: Article
• Language: English
• Subjects: Confidence measure; Machine learning; Semi global matching; Stereo matching
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2020.102905

Depth sensing is of paramount importance for countless applications and stereo represents a popular, effective and cheap solution for this purpose. As highlighted by recent works concerned with stereo, uncertainty estimation can be a powerful cue to improve accuracy in stereo. Most confidence measures rely on features, mainly extracted from the cost volume, fed to a random forest or a convolutional neural network trained to estimate match uncertainty. In contrast, we propose a novel strategy for confidence estimation based on features computed in the disparity domain, making our proposal suited for any stereo system including COTS devices, and in constant time. We exhaustively assess the performance of our proposals, referred to as O1 and O2, on KITTI and Middlebury datasets with three popular and different stereo algorithms (CENSUS, MC-CNN and SGM), as well as a deep stereo network (PSM-Net). We also evaluate how well confidence measures generalize to different environments/datasets. •We estimate confidence relying only on the reference disparity map.•A random forest learns to estimate confidence on features from the disparity domain.•A random forest learns to estimate confidence on features from the disparity domain.•A smarter SGM variant is proposed leveraging on the learned confidence measure.