Applying image registration algorithm combined with CNN model to video image stitching

• Published in: The Journal of supercomputing 2021, Vol.77 (12), p.13879-13896
• Main Author: Cao, Weiran
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Compilers; Computer Science; Correlation coefficients; Deep Learning in IoT: Emerging Trends and Applications - 2019; Image processing; Image quality; Image registration; Interpreters; Machine learning; Matching; Processor Architectures; Programming Languages; Registration; Remote sensing; Stitching; Unmanned aerial vehicles
• ISSN: 0920-8542; 1573-0484
• DOI: 10.1007/s11227-021-03840-2

The purposes are to explore the video image stitching technique of Unmanned Aerial Vehicles (UAVs), expand the application of image registration algorithms and new sensing equipment in video image stitching, and improve the development of video remote sensing image processing. First, how to remotely stitch UAV video images is analyzed. Second, the Scale Invariant Feature Transform (SIFT) algorithm is improved by re-dividing the pixel region and incorporating the vector correlation coefficient. Then, the matching results of the improved SIFT algorithm are compared with those of the traditional SIFT algorithm and Speed Up Robust Feature algorithm. Second, the Random Sample Consensus algorithm is introduced to match the video images accurately, and the matching accuracy of the converted images is verified. Finally, the improved image registration algorithm, the homography matrix estimation model based on convolutional neural network, and the conformation equation of the video sensor are combined to complete the video image stitching of UAVs. Also, the stitching quality of the video image is analyzed. The results show that the improved SIFT algorithm is better than the traditional SIFT algorithm in terms of correct matching and matching time. The visually transformed finely-matched image has a higher matching accuracy rate, and the combination of the two registration algorithms eliminates mismatch points effectively. Compared with the traditional stitching method, the video image stitching method proposed in this study has a higher structural similarity index and edge difference spectrum index, which is feasible and effective. The combination of image registration algorithm with new sensors and deep learning has great application potential in video image stitching.