NudgeSeg: Zero-Shot Object Segmentation by Repeated Physical Interaction

Recent advances in object segmentation have demonstrated that deep neural networks excel at object segmentation for specific classes in color and depth images. However, their performance is dictated by the number of classes and objects used for training, thereby hindering generalization to never see...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-09
Main Authors: Singh, Chahat Deep, Sanket, Nitin J, Parameshwara, Chethan M, Fermüller, Cornelia, Aloimonos, Yiannis
Format: Article
Language:English
Subjects:
Artificial neural networks
Geometric constraints
Image segmentation
Moving object recognition
Pattern matching
Pattern recognition
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recent advances in object segmentation have demonstrated that deep neural networks excel at object segmentation for specific classes in color and depth images. However, their performance is dictated by the number of classes and objects used for training, thereby hindering generalization to never seen objects or zero-shot samples. To exacerbate the problem further, object segmentation using image frames rely on recognition and pattern matching cues. Instead, we utilize the 'active' nature of a robot and their ability to 'interact' with the environment to induce additional geometric constraints for segmenting zero-shot samples. In this paper, we present the first framework to segment unknown objects in a cluttered scene by repeatedly 'nudging' at the objects and moving them to obtain additional motion cues at every step using only a monochrome monocular camera. We call our framework NudgeSeg. These motion cues are used to refine the segmentation masks. We successfully test our approach to segment novel objects in various cluttered scenes and provide an extensive study with image and motion segmentation methods. We show an impressive average detection rate of over 86% on zero-shot objects.
ISSN:2331-8422