Robust High-Resolution Video Matting with Temporal Guidance

We introduce a robust, real-time, high-resolution human video matting method that achieves new state-of-the-art performance. Our method is much lighter than previous approaches and can process 4K at 76 FPS and HD at 104 FPS on an Nvidia GTX 1080Ti GPU. Unlike most existing methods that perform video...

Full description

Saved in:
Bibliographic Details
Main Authors: Lin, Shanchuan, Yang, Linjie, Saleemi, Imran, Sengupta, Soumyadip
Format: Conference Proceeding
Language:English
Subjects:
Computational modeling
Computational Photography
Image and Video Synthesis Segmentation
Grouping and Shape
Vision for Graphics
Computer architecture
Computer vision
Graphics processing units
Image segmentation
Semantics
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We introduce a robust, real-time, high-resolution human video matting method that achieves new state-of-the-art performance. Our method is much lighter than previous approaches and can process 4K at 76 FPS and HD at 104 FPS on an Nvidia GTX 1080Ti GPU. Unlike most existing methods that perform video matting frame-by-frame as independent images, our method uses a recurrent architecture to exploit temporal information in videos and achieves significant improvements in temporal coherence and matting quality. Furthermore, we propose a novel training strategy that enforces our network on both matting and segmentation objectives. This significantly improves our model's robustness. Our method does not require any auxiliary inputs such as a trimap or a pre-captured background image, so it can be widely applied to existing human matting applications. Our code is available at https://peterl1n.github.io/RobustVideoMatting/
ISSN:2642-9381
DOI:10.1109/WACV51458.2022.00319