MCTracker: Satellite video multi-object tracking considering inter-frame motion correlation and multi-scale cascaded feature enhancement

• Published in: ISPRS journal of photogrammetry and remote sensing 2024-08, Vol.214, p.82-103
• Main Authors: Wang, Bin, Sui, Haigang, Ma, Guorui, Zhou, Yuan
• Format: Article
• Language: English
• Subjects: Inter-frame motion correlation; Multi-object tracking; Multi-scale feature; Satellite video; Spatio-temporal fusion
• ISSN: 0924-2716; 1872-8235
• DOI: 10.1016/j.isprsjprs.2024.06.006

[Display omitted] •MCTracker integrates motion correlation and mix-scale feature enhancement to track small, dense, and blurry objects effectively.•CDFE, IFMCM, and MSEB structures enhance contrast encoding, motion correlation, and multi-scale feature integration.•STFM resolves temporal-spatial fusion inconsistencies, boosting fusion expressive capability for temporal-spatial features. The video data captured by satellites, colloquially referred to as “gazing,” holds significant value for analyzing object status and dynamically tracking movements. Such data and their associated applications play pivotal roles in disaster response, urban traffic monitoring, national defense, and security operations. The multi-object tracking task in satellite video surveillance is inherently more complex than single-object tracking, presenting challenges such as tracking dense small objects and limited capability in multi-object tracking in ambiguous environments. This study proposes a satellite video multi-object tracking (MCTracker) method that considers inter-frame motion correlation and multi-scale cascaded feature enhancement, achieving outstanding performance in multi-object tracking tasks. This study builds upon a differentially encoded backbone network incorporating both global and local information, proposing an Inter-frame Motion Correlation Module (IFMCM) to enhance inter-frame dynamic continuity. A Mixed-scale Enhancement Block (MSEB) is designed to address the challenge of detecting and tracking small objects under multiscale effects. Additionally, a Spatio-temporal Fusion Module (STFM) is introduced to improve the feature fusion representation capability among modules. Through experiments conducted on two open-source datasets, AIR-MOT and SAT-MTB, it has been demonstrated that the proposed MCTracker exhibits efficient synergistic consistency in detection and ReID tasks. On the AIR-MOT dataset, the object tracking accuracy, as measured by MOTA, reaches 59.4 %, achieving optimal performance. In the SAT-MTB dataset’s four-class multiple object tracking task, particularly in tracking small-scale objects such as cars and ships, the proposed method demonstrates comprehensive optimal performance, yielding robust tracking results. The satellite video-based multiple object tracking approach proposed in this study holds significant reference value for further exploration of object motion states and trajectory tracking.