Hunt-inspired Transformer for visual object tracking

• Published in: Pattern recognition 2024-12, Vol.156, p.110703, Article 110703
• Main Authors: Zhang, Zhibin, Xue, Wanli, Zhou, Yuxi, Zhang, Kaihua, Chen, Shengyong
• Format: Article
• Language: English
• Subjects: Particle filter; Transformer; Uncertainty estimation; Visual object tracking
• ISSN: 0031-3203
• DOI: 10.1016/j.patcog.2024.110703

This paper presents a hunt-inspired Transformer for visual object tracking, dubbed as HuntFormer. The HuntFormer focuses on robust target detection and identification, simulating natural hunting processes. Specifically, the HuntFormer comprises two essential module designs including a predictor for detection and a verifier for identification. The predictor emulates the detection stage by designing a motion trajectory guided particle filter, which identifies potential target locations by predicting the motion state within a particle filtering framework. The predictor utilizes spatio-temporal correlation scores between dynamic target templates and the search region to guide the learning process to generate a set of reliable particles. This enables the base tracker to narrow its search range to focus on the target, and swiftly re-detect the target in case of model drift. Once the target is re-detected, the verifier assesses the detection result as a reliable tracked item. The verifier initially maintains a dynamic memory that stores reliable target templates and their corresponding locations in the motion trajectory. It then models the uncertainty of appearance information within this memory probabilistically. The output uncertainty score determines whether the memory gets updated or not. Ultimately, the predictor and the verifier collaborate, ensuring a robust tracking outcome. Extensive evaluations on six challenging benchmark datasets demonstrate HuntFormer’s favorable performance against various state-of-the-art trackers. Notably, in the VOT-LT2022 tracking challenge, the HuntFormer won the third place with an F-score of 0.598, closely competing for the second place with an F-score of 0.600. •The novel tracker is inspired by hunting and contains a predictor and a validator.•Predictor based on trajectory-guided particle filtering for searching the region.•Validator with uncertainty modelling to reduce overconfidence issues in tracking.