Real-Time Single Image Depth Perception in the Wild with Handheld Devices

• Published in: Sensors (Basel, Switzerland) Switzerland), 2020-12, Vol.21 (1), p.15
• Main Authors: Aleotti, Filippo, Zaccaroni, Giulio, Bartolomei, Luca, Poggi, Matteo, Tosi, Fabio, Mattoccia, Stefano
• Format: Article
• Language: English
• Subjects: deep learning; mobile systems; monocular depth estimation; smartphone
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s21010015

Depth perception is paramount for tackling real-world problems, ranging from autonomous driving to consumer applications. For the latter, depth estimation from a single image would represent the most versatile solution since a standard camera is available on almost any handheld device. Nonetheless, two main issues limit the practical deployment of monocular depth estimation methods on such devices: (i) the low reliability when deployed in the wild and (ii) the resources needed to achieve real-time performance, often not compatible with low-power embedded systems. Therefore, in this paper, we deeply investigate all these issues, showing how they are both addressable by adopting appropriate network design and training strategies. Moreover, we also outline how to map the resulting networks on handheld devices to achieve real-time performance. Our thorough evaluation highlights the ability of such fast networks to generalize well to new environments, a crucial feature required to tackle the extremely varied contexts faced in real applications. Indeed, to further support this evidence, we report experimental results concerning real-time, depth-aware augmented reality and image blurring with smartphones in the wild.