SiamMask: A Framework for Fast Online Object Tracking and Segmentation

In this article, we introduce SiamMask, a framework to perform both visual object tracking and video object segmentation, in real-time, with the same simple method. We improve the offline training procedure of popular fully-convolutional Siamese approaches by augmenting their losses with a binary se...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2023-03, Vol.45 (3), p.3072-3089
Main Authors: Hu, Weiming, Wang, Qiang, Zhang, Li, Bertinetto, Luca, Torr, Philip H.S.
Format: Article
Language:English
Subjects:
Benchmark testing
Benchmarks
Frames per second
multiple object tracking
Multiple target tracking
Multitasking
Object segmentation
Object tracking
Optical tracking
Real time
Segmentation
siamese network
Streaming media
Target tracking
Training
video object segmentation
Visualization
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this article, we introduce SiamMask, a framework to perform both visual object tracking and video object segmentation, in real-time, with the same simple method. We improve the offline training procedure of popular fully-convolutional Siamese approaches by augmenting their losses with a binary segmentation task. Once the offline training is completed, SiamMask only requires a single bounding box for initialization and can simultaneously carry out visual object tracking and segmentation at high frame-rates. Moreover, we show that it is possible to extend the framework to handle multiple object tracking and segmentation by simply re-using the multi-task model in a cascaded fashion. Experimental results show that our approach has high processing efficiency, at around 55 frames per second. It yields real-time state-of-the art results on visual-object tracking benchmarks, while at the same time demonstrating competitive performance at a high speed for video object segmentation benchmarks.
ISSN:0162-8828
1939-3539
DOI:10.1109/TPAMI.2022.3172932