Learning Correspondence From the Cycle-Consistency of Time

We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful f...

Full description

Saved in:
Bibliographic Details
Main Authors: Wang, Xiaolong, Jabri, Allan, Efros, Alexei A.
Format: Conference Proceeding
Language:English
Subjects:
Computer vision
Motion and Tracking
Object segmentation
Optical flow
Pattern recognition
Representation Learning
Task analysis
Training
Video Analytics
Visualization
Citations: Items that cite this one
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c249t-cb5ce4cf019c4c04a39e291e21da9402b9638c9c23f268ddb38bda14192413ac3
cites
container_end_page 2571
container_issue
container_start_page 2561
container_title
container_volume
creator Wang, Xiaolong
Jabri, Allan
Efros, Alexei A.
description We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful for performing cycle-consistent tracking. At test time, we use the acquired representation to find nearest neighbors across space and time. We demonstrate the generalizability of the representation -- without finetuning -- across a range of visual correspondence tasks, including video object segmentation, keypoint tracking, and optical flow. Our approach outperforms previous self-supervised methods and performs competitively with strongly supervised methods.
doi_str_mv 10.1109/CVPR.2019.00267
format conference_proceeding
fullrecord <record><control><sourceid>ieee_CHZPO</sourceid><recordid>TN_cdi_ieee_primary_8954240</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>8954240</ieee_id><sourcerecordid>8954240</sourcerecordid><originalsourceid>FETCH-LOGICAL-c249t-cb5ce4cf019c4c04a39e291e21da9402b9638c9c23f268ddb38bda14192413ac3</originalsourceid><addsrcrecordid>eNotjsFKxDAUAKMguKw9e_CSH2jNe0mbPG8SXFcoKLJ6XdL0VSvbdkl72b93QU9zGBhGiFtQBYCie__59l6gAiqUwspeiIysA4sONJJ2l2KFpS1zq2x5LbJ5_lFKaQSoyK3EQ80hjf34Jf2UEs_HaWx5jCw3aRrk8s3Sn-KBcz-Ncz8vZ3WSUyd3_cA34qoLh5mzf67Fx-Zp57d5_fr84h_rPKKhJY9NGdnE7jwYTVQmaGIkYIQ2kFHYUKVdpIi6w8q1baNd0wYwQGhAh6jX4u6v2zPz_pj6IaTT3lFp0Cj9CxKfR2M</addsrcrecordid><sourcetype>Publisher</sourcetype><iscdi>true</iscdi><recordtype>conference_proceeding</recordtype></control><display><type>conference_proceeding</type><title>Learning Correspondence From the Cycle-Consistency of Time</title><source>IEEE Xplore All Conference Series</source><creator>Wang, Xiaolong ; Jabri, Allan ; Efros, Alexei A.</creator><creatorcontrib>Wang, Xiaolong ; Jabri, Allan ; Efros, Alexei A.</creatorcontrib><description>We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful for performing cycle-consistent tracking. At test time, we use the acquired representation to find nearest neighbors across space and time. We demonstrate the generalizability of the representation -- without finetuning -- across a range of visual correspondence tasks, including video object segmentation, keypoint tracking, and optical flow. Our approach outperforms previous self-supervised methods and performs competitively with strongly supervised methods.</description><identifier>EISSN: 2575-7075</identifier><identifier>EISBN: 9781728132938</identifier><identifier>EISBN: 1728132932</identifier><identifier>DOI: 10.1109/CVPR.2019.00267</identifier><language>eng</language><publisher>IEEE</publisher><subject>Computer vision ; Motion and Tracking ; Object segmentation ; Optical flow ; Pattern recognition ; Representation Learning ; Task analysis ; Training ; Video Analytics ; Visualization</subject><ispartof>2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, p.2561-2571</ispartof><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c249t-cb5ce4cf019c4c04a39e291e21da9402b9638c9c23f268ddb38bda14192413ac3</citedby></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/8954240$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>309,310,780,784,789,790,27925,54555,54932</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/8954240$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Wang, Xiaolong</creatorcontrib><creatorcontrib>Jabri, Allan</creatorcontrib><creatorcontrib>Efros, Alexei A.</creatorcontrib><title>Learning Correspondence From the Cycle-Consistency of Time</title><title>2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)</title><addtitle>CVPR</addtitle><description>We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful for performing cycle-consistent tracking. At test time, we use the acquired representation to find nearest neighbors across space and time. We demonstrate the generalizability of the representation -- without finetuning -- across a range of visual correspondence tasks, including video object segmentation, keypoint tracking, and optical flow. Our approach outperforms previous self-supervised methods and performs competitively with strongly supervised methods.</description><subject>Computer vision</subject><subject>Motion and Tracking</subject><subject>Object segmentation</subject><subject>Optical flow</subject><subject>Pattern recognition</subject><subject>Representation Learning</subject><subject>Task analysis</subject><subject>Training</subject><subject>Video Analytics</subject><subject>Visualization</subject><issn>2575-7075</issn><isbn>9781728132938</isbn><isbn>1728132932</isbn><fulltext>true</fulltext><rsrctype>conference_proceeding</rsrctype><creationdate>2019</creationdate><recordtype>conference_proceeding</recordtype><sourceid>6IE</sourceid><recordid>eNotjsFKxDAUAKMguKw9e_CSH2jNe0mbPG8SXFcoKLJ6XdL0VSvbdkl72b93QU9zGBhGiFtQBYCie__59l6gAiqUwspeiIysA4sONJJ2l2KFpS1zq2x5LbJ5_lFKaQSoyK3EQ80hjf34Jf2UEs_HaWx5jCw3aRrk8s3Sn-KBcz-Ncz8vZ3WSUyd3_cA34qoLh5mzf67Fx-Zp57d5_fr84h_rPKKhJY9NGdnE7jwYTVQmaGIkYIQ2kFHYUKVdpIi6w8q1baNd0wYwQGhAh6jX4u6v2zPz_pj6IaTT3lFp0Cj9CxKfR2M</recordid><startdate>201906</startdate><enddate>201906</enddate><creator>Wang, Xiaolong</creator><creator>Jabri, Allan</creator><creator>Efros, Alexei A.</creator><general>IEEE</general><scope>6IE</scope><scope>6IH</scope><scope>CBEJK</scope><scope>RIE</scope><scope>RIO</scope></search><sort><creationdate>201906</creationdate><title>Learning Correspondence From the Cycle-Consistency of Time</title><author>Wang, Xiaolong ; Jabri, Allan ; Efros, Alexei A.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c249t-cb5ce4cf019c4c04a39e291e21da9402b9638c9c23f268ddb38bda14192413ac3</frbrgroupid><rsrctype>conference_proceedings</rsrctype><prefilter>conference_proceedings</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Computer vision</topic><topic>Motion and Tracking</topic><topic>Object segmentation</topic><topic>Optical flow</topic><topic>Pattern recognition</topic><topic>Representation Learning</topic><topic>Task analysis</topic><topic>Training</topic><topic>Video Analytics</topic><topic>Visualization</topic><toplevel>online_resources</toplevel><creatorcontrib>Wang, Xiaolong</creatorcontrib><creatorcontrib>Jabri, Allan</creatorcontrib><creatorcontrib>Efros, Alexei A.</creatorcontrib><collection>IEEE Electronic Library (IEL) Conference Proceedings</collection><collection>IEEE Proceedings Order Plan (POP) 1998-present by volume</collection><collection>IEEE Xplore All Conference Proceedings</collection><collection>IEEE/IET Electronic Library (IEL)</collection><collection>IEEE Proceedings Order Plans (POP) 1998-present</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Wang, Xiaolong</au><au>Jabri, Allan</au><au>Efros, Alexei A.</au><format>book</format><genre>proceeding</genre><ristype>CONF</ristype><atitle>Learning Correspondence From the Cycle-Consistency of Time</atitle><btitle>2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)</btitle><stitle>CVPR</stitle><date>2019-06</date><risdate>2019</risdate><spage>2561</spage><epage>2571</epage><pages>2561-2571</pages><eissn>2575-7075</eissn><eisbn>9781728132938</eisbn><eisbn>1728132932</eisbn><abstract>We introduce a self-supervised method for learning visual correspondence from unlabeled video. The main idea is to use cycle-consistency in time as free supervisory signal for learning visual representations from scratch. At training time, our model learns a feature map representation to be useful for performing cycle-consistent tracking. At test time, we use the acquired representation to find nearest neighbors across space and time. We demonstrate the generalizability of the representation -- without finetuning -- across a range of visual correspondence tasks, including video object segmentation, keypoint tracking, and optical flow. Our approach outperforms previous self-supervised methods and performs competitively with strongly supervised methods.</abstract><pub>IEEE</pub><doi>10.1109/CVPR.2019.00267</doi><tpages>11</tpages></addata></record>
fulltext fulltext_linktorsrc
identifier EISSN: 2575-7075
ispartof 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, p.2561-2571
issn 2575-7075
language eng
recordid cdi_ieee_primary_8954240
source IEEE Xplore All Conference Series
subjects Computer vision
Motion and Tracking
Object segmentation
Optical flow
Pattern recognition
Representation Learning
Task analysis
Training
Video Analytics
Visualization
title Learning Correspondence From the Cycle-Consistency of Time
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-26T21%3A30%3A30IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-ieee_CHZPO&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=proceeding&rft.atitle=Learning%20Correspondence%20From%20the%20Cycle-Consistency%20of%20Time&rft.btitle=2019%20IEEE/CVF%20Conference%20on%20Computer%20Vision%20and%20Pattern%20Recognition%20(CVPR)&rft.au=Wang,%20Xiaolong&rft.date=2019-06&rft.spage=2561&rft.epage=2571&rft.pages=2561-2571&rft.eissn=2575-7075&rft_id=info:doi/10.1109/CVPR.2019.00267&rft.eisbn=9781728132938&rft.eisbn_list=1728132932&rft_dat=%3Cieee_CHZPO%3E8954240%3C/ieee_CHZPO%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c249t-cb5ce4cf019c4c04a39e291e21da9402b9638c9c23f268ddb38bda14192413ac3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rft_ieee_id=8954240&rfr_iscdi=true