Real-time and accurate monocular 3D sensor using the reference plane calibration and an optimized SGM based on opencl acceleration

• Published in: Optics and lasers in engineering 2023-06, Vol.165, p.107536, Article 107536
• Main Authors: Yin, Wei, Cao, Lu, Zhao, Hang, Hu, Yan, Feng, Shijie, Zhang, Xiaolei, Shen, Detong, Wang, Huai, Chen, Qian, Zuo, Chao
• Format: Article
• Language: English
• Subjects: 3D Imaging; Opencl; Speckle projection profilometry; Stereo matching
• ISSN: 0143-8166; 1873-0302
• DOI: 10.1016/j.optlaseng.2023.107536

•A real-time and accurate 3D measurement method using a monocular 3D sensor based on the infrared speckle projection.•Based on the 3D imaging principle of monocular 3D sensors, a reference plane calibration method is proposed to obtain a high-quality reference speckle image for improving the monocular matching accuracy.•an optimized semi-global matching (SGM) algorithm using GPU is presented to achieve efficient and accurate depth reconstruction dynamically.•Within the measurement range of 0.8 m (length) × 0.5 m (width) × 1 m (depth), the proposed method can achieve real-time and single-shot 3D imaging with an accuracy of 1.277 mm at 75 FPS on GTX 1060 and 15 FPS on ARM Mail G52(mobile platform). Speckle projection profilometry (SPP), as a promising structured light projection technique, can achieve global unambiguous 3D measurement by projecting a single random speckle pattern. In addition, the projected speckle pattern is usually etched into the microstructure of highly integrated Vertical-Cavity Surface-Emitting Laser (VCSEL), which makes the hardware system compact enough to be mounted on mobile devices such as robots. However, since the stereo matching algorithm used in SPP involves high computational overhead, it usually runs in real-time on specially customized hardware platforms such as ASIC/FPGA, rather than general-purpose mobile platforms. In this paper, we propose a real-time and accurate 3D measurement method using a monocular 3D sensor based on the infrared speckle projection. Similar to Kinect v1, our sensor mainly consists of an IR dot projector and one IR camera for projecting and capturing speckle images synchronously. Low-cost and high-quality speckle projection is achieved by customizing the projection pattern of VCSEL and using the beam copy function of Diffractive Optical Elements (DOE). Based on the 3D imaging principle of monocular 3D sensors, a reference plane calibration method is proposed to obtain a high-quality reference speckle image for improving the monocular matching accuracy. Then, benefited from the local memory mechanism and multiple operating synchronizations on the OpenCL environment, an optimized semi-global matching (SGM) algorithm using GPU is presented to achieve efficient and accurate depth reconstruction dynamically. Within the measurement range of 0.8m (length) ×0.5m (width) ×1m (depth), the proposed method can achieve real-time and single-shot 3D imaging with an accuracy of 1.277 mm at 75 FPS on GTX 1060 and 15