A Calibration and Real-Time Object Matching Method for Heterogeneous Multi-Camera System

The heterogeneous multi-camera system composed of thermal-infrared camera and visible spectrum (VS) camera can better perceive the surrounding information in complex environment, so it is widely used in many intelligent unmanned devices, such as unmanned aerial vehicles (UAV) and patrol robots. In t...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on instrumentation and measurement 2023-01, Vol.72, p.1-1
Main Authors: Li, Chenming, Cai, Chengtao
Format: Article
Language:English
Subjects:
Calibration
Calibration optimization
Cameras
Heating systems
Image color analysis
Infrared cameras
Intelligent unmanned equipment
Machine vision
Matching
Photothermal effects
Real time
Real-time target matching
Robot vision systems
Sorting algorithms
Stereo vision system
Target detection
Unmanned aerial vehicles
Visible spectrum
Vision systems
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The heterogeneous multi-camera system composed of thermal-infrared camera and visible spectrum (VS) camera can better perceive the surrounding information in complex environment, so it is widely used in many intelligent unmanned devices, such as unmanned aerial vehicles (UAV) and patrol robots. In this paper, a stereo calibration strategy is proposed for this vision system, and a real-time target matching method for heterogeneous images is proposed based on double epipolar constraint. First, a new calibration board is designed, and a calibration points detection and sorting algorithm is proposed. Secondly, the thermal-infrared camera and VS camera are calibrated. Then, the relative position and posture between the two cameras and the projection relationship between the heterologous images are obtained by stereo calibration using the adjusted images, and applied to stereo vision. Finally, the epipolar constraint is applied to the target matching under the heterogenous image combined with the existing target detection technology. In experiments, the average relative error of calibration point positioning using heterogeneous vision system is 0.0209, which is only 0.0014 different from that using homogeneous vision system. When the target appears in the image as a whole and is detected, target matching in a pair of heterologous images only takes 0.068s and the accuracy is 96%, which meeting the real-time and accuracy requirements of general application environments.
ISSN:0018-9456
1557-9662
DOI:10.1109/TIM.2023.3260263