DTS-Depth: Real-Time Single-Image Depth Estimation Using Depth-to-Space Image Construction

• Published in: Sensors (Basel, Switzerland) Switzerland), 2022-03, Vol.22 (5), p.1914
• Main Authors: Ibrahem, Hatem, Salem, Ahmed, Kang, Hyun-Soo
• Format: Article
• Language: English
• Subjects: 3-D graphics; Accuracy; Algorithms; Autonomous vehicles; Computer architecture; Construction; convolutional neural networks; depth estimation; Embedded systems; Experiments; Frames per second; High resolution; Methods; Neural networks; Real time; real-time processing
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22051914

As most of the recent high-resolution depth-estimation algorithms are computationally so expensive that they cannot work in real time, the common solution is using a low-resolution input image to reduce the computational complexity. We propose a different approach, an efficient and real-time convolutional neural network-based depth-estimation algorithm using a single high-resolution image as the input. The proposed method efficiently constructs a high-resolution depth map using a small encoding architecture and eliminates the need for a decoder, which is typically used in the encoder-decoder architectures employed for depth estimation. The proposed algorithm adopts a modified MobileNetV2 architecture, which is a lightweight architecture, to estimate the depth information through the depth-to-space image construction, which is generally employed in image super-resolution. As a result, it realizes fast frame processing and can predict a high-accuracy depth in real time. We train and test our method on the challenging KITTI, Cityscapes, and NYUV2 depth datasets. The proposed method achieves low relative absolute error (0.028 for KITTI, 0.167 for CITYSCAPES, and 0.069 for NYUV2) while working at speed reaching 48 frames per second on a GPU and 20 frames per second on a CPU for high-resolution test images. We compare our method with the state-of-the-art methods on depth estimation, showing that our method outperforms those methods. However, the architecture is less complex and works in real time.