Learning Correspondence From the Cycle-Consistency of Time

We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful f...

Full description

Saved in:
Bibliographic Details
Main Authors: Wang, Xiaolong, Jabri, Allan, Efros, Alexei A.
Format: Conference Proceeding
Language:English
Subjects:
Computer vision
Motion and Tracking
Object segmentation
Optical flow
Pattern recognition
Representation Learning
Task analysis
Training
Video Analytics
Visualization
Citations: Items that cite this one
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful for performing cycle-consistent tracking. At test time, we use the acquired representation to find nearest neighbors across space and time. We demonstrate the generalizability of the representation -- without finetuning -- across a range of visual correspondence tasks, including video object segmentation, keypoint tracking, and optical flow. Our approach outperforms previous self-supervised methods and performs competitively with strongly supervised methods.
ISSN:2575-7075
DOI:10.1109/CVPR.2019.00267