Localizing Objects with Self-Supervised Transformers and no Labels

Localizing objects in image collections without supervision can help to avoid expensive annotation campaigns. We propose a simple approach to this problem, that leverages the activation features of a vision transformer pre-trained in a self-supervised manner. Our method, LOST, does not require any e...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-09
Main Authors: Siméoni, Oriane, Puy, Gilles, Vo, Huy V, Roburin, Simon, Gidaris, Spyros, Bursuc, Andrei, Pérez, Patrick, Renaud Marlet, Ponce, Jean
Format: Article
Language:English
Subjects:
Annotations
Object recognition
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Localizing objects in image collections without supervision can help to avoid expensive annotation campaigns. We propose a simple approach to this problem, that leverages the activation features of a vision transformer pre-trained in a self-supervised manner. Our method, LOST, does not require any external object proposal nor any exploration of the image collection; it operates on a single image. Yet, we outperform state-of-the-art object discovery methods by up to 8 CorLoc points on PASCAL VOC 2012. We also show that training a class-agnostic detector on the discovered objects boosts results by another 7 points. Moreover, we show promising results on the unsupervised object discovery task. The code to reproduce our results can be found at https://github.com/valeoai/LOST.
ISSN:2331-8422