Cloth manipulation based on category classification and landmark detection

• Published in: International journal of advanced robotic systems 2022-07, Vol.19 (4)
• Main Authors: Gustavsson, Oscar, Ziegler, Thomas, Welle, Michael C, Bütepage, Judith, Varava, Anastasiia, Kragic, Danica
• Format: Article
• Language: English
• Subjects: classification; Cloth; data augmentation; Datasets; Deep learning; Evaluation; garment manipulation; Garments; Image classification; Image manipulation; Machine learning; vision for robotics
• ISSN: 1729-8806; 1729-8814; 1729-8814
• DOI: 10.1177/17298806221110445

Cloth manipulation remains a challenging problem for the robotic community. Recently, there has been an increased interest in applying deep learning techniques to problems in the fashion industry. As a result, large annotated data sets for cloth category classification and landmark detection were created. In this work, we leverage these advances in deep learning to perform cloth manipulation. We propose a full cloth manipulation framework that, performs category classification and landmark detection based on an image of a garment, followed by a manipulation strategy. The process is performed iteratively to achieve a stretching task where the goal is to bring a crumbled cloth into a stretched out position. We extensively evaluate our learning pipeline and show a detailed evaluation of our framework on different types of garments in a total of 140 recorded and available experiments. Finally, we demonstrate the benefits of training a network on augmented fashion data over using a small robotic-specific data set.