Unsupervised object discovery with pseudo label generated using K-means and self-supervised transformer

• Published in: Neurocomputing (Amsterdam) 2023-08, Vol.545, p.126326, Article 126326
• Main Authors: Lim, SeongTaek, Park, JaeEon, Lee, MinYoung, Lee, HongChul
• Format: Article
• Language: English
• Subjects: K-means; Object mask discovery; Self-supervised transformer
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2023.126326

•A simple yet effective framework that generates pseudo labels for unsupervised class agnostic instance segmentation is proposed.•Cosine distance-based K-means can separate foreground and background with components of self-supervised transformer.•We report state-of-the-art unsupervised class agnostic instance segmentation performance on two challenge datasets; further, we report state-of-the-art unsupervised class agnostic object detection performance on COCO val2017. Instance segmentation is a fundamental task in the field of computer vision. It aims to assign every pixel to an appropriate class and localize objects within bounding boxes. However, expensive pixel-level segmentation labels are essential for training current state-of-the-art instance segmentation models. In this paper, we propose IMST, a simple method for generating instance-level pseudo-object mask labels without any form of human annotation. IMST leverages the fact that self-supervised transformers embed background and foreground into distinct cluster spaces. This characteristic allows us to discover class-agnostic object masks from unlabeled image datasets using cosine distance K-means clustering. We also present an object mask refinement method that employs ensemble results of K-means in a single image. Despite its simplicity, IMST achieves state-of-the-art performance in the unsupervised object mask discovery task. In unsupervised class-agnostic instance segmentation, IMST outperforms concurrent works by margins of 5.3 AP and 4.0 AP, respectively, on COCO20k and COCO val2017. Our proposed method can be extended to object box discovery tasks, such as unsupervised class-agnostic object detection and unsupervised single object discovery, surpassing previous works.