Deep Learning-Based Instance Segmentation for Feature Extraction of Branched Deformable Linear Objects for Robotic Manipulation

• Main Authors: Zurn, Manuel, Kienzlen, Annika, Klingel, Lars, Lechler, Armin, Verl, Alexander, Ren, Shiyi, Xu, Weiliang
• Format: Conference Proceeding
• Language: English
• Subjects: Annotations; Automation; Branched Deformable Linear Object; Connectors; Deep Learning; Deformable models; Image segmentation; Machine Vision; Robotic Manipulation; Training; Wire Harness Feature Extraction; Wires
• ISSN: 2161-8089
• DOI: 10.1109/CASE56687.2023.10260646

Branched deformable linear objects, like wire harnesses, are vastly used in the electrical industry to interconnect multiple electronic components while allowing small relative motions. Their deformable nature gives them an infinite-dimensional configuration space, which is challenging for robotic manipulation and requires sensor feedback about the actual configuration. Cameras have been used before in automation attempts as sensor feedback of branched deformable linear objects. However, image processing is sensible to occlusions and unstructured environments. This paper presents a method of using deep learning-based instance segmentation networks to extract topological features like connectors, clips, branch points and overlapping points on the branched deformable linear object. The topological features originate from features that can be used for matching a stretched representation of the branched deformable linear object to its sensor representation. Our results show that the network is capable of extracting the features for different datasets with high accuracy. Furthermore, this paper evaluates the use of preliminary results in the training process to facilitate annotation by predicting the labels. For now, this paper only uses 2D images of the branched deformable linear objects. Future work will also use depth images, such that individual deformable linear object features, such as length or radius, can be extracted from the images. We anticipate the usage of wire harness automation with robotic manipulation, augmented reality support of the worker or quality control of wire harnesses.