Joint visual template and natural language for robust visual tracking

At present, the target of interest in visual tracking is given in the form of a bounding box. Due to the randomness of the target shape, the bounding box may contain a lot of non‐target information. When encountering complex tracking scenarios, the performance of the tracker reduces severely. To add...

Full description

Saved in:
Bibliographic Details
Published in:Electronics letters 2022-10, Vol.58 (21), p.798-800
Main Authors: Wang, Jingchao, Zhang, Huanlong, Zhang, Jianwei
Format: Article
Language:English
Subjects:
Annotations
Boxes
Language
Modules
Natural language
Optical tracking
Query languages
Semantics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:At present, the target of interest in visual tracking is given in the form of a bounding box. Due to the randomness of the target shape, the bounding box may contain a lot of non‐target information. When encountering complex tracking scenarios, the performance of the tracker reduces severely. To address this problem, in this letter, the authors propose a novel tracking framework based on the joint of the visual template and natural language (VNTrack) to alleviate the impact of bounding box ambiguity. Specifically, the authors first use a pre‐trained language model to extract the features of the language description of the target. Then, a feature alignment module is designed to align and enhance the visual template feature and natural language feature. In addition, the authors design a multimodal query module to fuse the visual template, natural language, and search region information. Experimental results over tracking benchmarks with language annotations show that the proposed VNTrack is competitive among the state‐of‐the‐art trackers.
ISSN:0013-5194
1350-911X
DOI:10.1049/ell2.12610