A hybrid approach for vision-based structural displacement measurement using transforming model prediction and KLT

• Published in: Mechanical systems and signal processing 2025-01, Vol.223, p.111866, Article 111866
• Main Authors: Nguyen, Xuan Tinh, Jeon, Geonyeol, Vy, Van, Lee, Geonhee, Lam, Phat Tai, Yoon, Hyungchul
• Format: Article
• Language: English
• Subjects: Deep learning; Object tracking; Structural health monitoring; Transformer-based model
• ISSN: 0888-3270
• DOI: 10.1016/j.ymssp.2024.111866

As civil infrastructures age, monitoring their health conditions has become increasingly critical. Dynamic displacement measurement is a prevalent method for assessing structural health. Traditional techniques, which often involve installing instruments and scaffolding, can interfere with the response of the structures. To address the challenge, non-contact measurement methods have been developed; however, these are typically costly and require expert operation. Advances in high-speed industrial cameras and image processing technology now enable vision-based displacement measurement. Despite their effectiveness, existing vision-based methods face significant limitations, including their inability to maintain tracking when line-of-sight is obstructed, sensitivity to lighting variations, and the need for manual intervention when feature points are lost. This study introduces a novel hybrid approach, termed ToMP-KLT, which combines the KLT tracker with a deep learning-based model. This method harnesses the precision of the KLT tracker under favorable conditions and the robustness of the deep learning-based tracker under adverse conditions. Its effectiveness is validated through simulation-based tests, lab-scale experiments, and field testing on Cheonsa Bridge, demonstrating substantial improvements in tracking robustness against occlusions and varying light conditions. [Display omitted] •A novel approach for structural displacement measurement combining the conventional tracker together with a Transformer-based model was proposed.•The proposed method improves the overall tracking robustness against occlusion and brightness conditions.•The proposed method can automatically redefine the region of interest after the failure.•The effectiveness of the proposed method is demonstrated through simulation-based experiments, lab-scale experiments, and an on-site test.